native-inference

Here are 4 public repositories matching this topic...

RocketWill / ByteWhisperer

Cross-language reference for native inference SDK integration across C++, Python, and C#, focused on FFI, structure layout, buffer ownership, and resource lifecycle.

python windows c-plus-plus csharp ffi pinvoke interoperability ctypes model-deployment onnx openvino sdk-integration native-inference

Updated Aug 9, 2026
C#

propardhu / Quant_test_LLM

Star

Native llama.cpp orchestration for quantized LLM evaluation, drift analysis, and reproducible model comparison.

python performance-analysis quantization model-evaluation model-comparison pydantic apple-silicon typer-cli llm llmops llama-cpp local-ai gguf reproducible-ai inference-evaluation quantization-drift sementic-preservation native-inference

Updated May 31, 2026
Python

dangkhoa2016 / Mage-Flow-Turbo-Native-Inference

Star

Portable native inference stack for Mage-Flow-Turbo using stable-diffusion.cpp/sd-cli, Q8_0 DiT GGUF, Qwen3-VL-4B Q4_K_M and a dedicated VAE, with CPU/CUDA backends, CLI, REST API, model verification and Kaggle integration.

Updated Sep 11, 2026
Python

zbiljic / needle-go

Star

Go library for running the Cactus Needle on-device tool-calling model.

go golang needle cactus structured-output on-device-ai llm function-calling local-ai tool-calling native-inference

Updated Sep 14, 2026
Go

Add this topic to your repo

To associate your repository with the native-inference topic, visit your repo's landing page and select "manage topics."

Learn more