heterogeneous-inference

Here are 2 public repositories matching this topic...

chadhurley25075-png / pd-bridge

Heterogeneous prefill/decode for DeepSeek-V4-Flash: CUDA prefill (DGX Spark, vLLM) -> Metal decode (Mac Studio, oMLX) over plain 10GbE

mlx kv-cache apple-silicon vllm llm-inference deepseek dgx-spark omlx prefill-decode heterogeneous-inference

Updated Sep 8, 2026
Python

manishklach / kv-attention-sram-memory-orchestrator

Star

KV Memory Orchestrator — a browser-native research simulator for deterministic KV-cache residency, SRAM/HBM tiering, DMA scheduling, and memory-fabric orchestration in future transformer inference systems. Patent No. 202641062302, filed in India.

sram memory-hierarchy kv-cache ai-inference systems-simulation llm-inference transformer-inference heterogeneous-inference patent-demo

Updated May 17, 2026
JavaScript

Add this topic to your repo

To associate your repository with the heterogeneous-inference topic, visit your repo's landing page and select "manage topics."

Learn more