LLM speculative inference server for consumer hardware & heterogeneous computing
An agent harness that compiles a model into one provably-correct, self-retargeting CUDA megakernel and self-tunes it past cuBLAS at batch-1 LLM decode, paper: https://arxiv.org/abs/2606.09682
A light, transparent, and modular inference & quantization engine for studying LLMs.
Air.rs 70B+ inference on consumer GPU, LLM inference in Rust
Single-launch batch-1 decode engine for PrismML Bonsai 27B (ternary and 1-bit) on NVIDIA GPUs. 1.76x the vendor llama.cpp fork on H100, same outputs.
Persistent-kernel CPU→GPU command bus. 64-byte packets, 0.5 µs enqueue, ~96 ns dispatch — no cudaLaunchKernel. C++/CUDA, 2 files.
Vendor-portable GPU decoders for quantum LDPC codes — Triton min-sum BP & Relay-BP on NVIDIA (CUDA), AMD (ROCm), and Apple-silicon (Metal), consuming any stim DetectorErrorModel.
Add a description, image, and links to the megakernel topic page so that developers can more easily learn about it.
To associate your repository with the megakernel topic, visit your repo's landing page and select "manage topics."