Popular repositories Loading

1. leanctx leanctx Public

   Drop-in prompt compression for production LLM apps. Cut your token bill 40-60% without changing your code. Python SDK, LLMLingua-2, MIT.

   Python 316 3

2. kube-gpu-top kube-gpu-top Public

   htop for GPU pods on Kubernetes — per-pod GPU utilization, memory, temperature, power, and waste detection

   Go 48 4

3. samza samza Public

   Forked from apache/samza

   Mirror of Apache Samza

   Java

4. vllm vllm Public

   Forked from vllm-project/vllm

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python

5. semantic-router semantic-router Public

   Forked from vllm-project/semantic-router

   Intelligent Mixture-of-Models Router for Efficient LLM Inference

   Python

6. sglang sglang Public

   Forked from sgl-project/sglang

   SGLang is a fast serving framework for large language models and vision language models.

   Python