Popular repositories Loading

1. vllm vllm Public

   Forked from vllm-project/vllm

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python

2. llm-compressor llm-compressor Public

   Forked from vllm-project/llm-compressor

   Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

   Python

3. rtp-llm rtp-llm Public

   Forked from alibaba/rtp-llm

   RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.

   Cuda 1

4. sglang sglang Public

   Forked from sgl-project/sglang

   SGLang is a high-performance serving framework for large language models and multimodal models.

   Python

5. aiter aiter Public

   Forked from ROCm/aiter

   AI Tensor Engine for ROCm

   Python

6. InferenceX InferenceX Public

   Forked from SemiAnalysisAI/InferenceX

   Open Source Continuous Inference Benchmark Research Platform — Kimi K3 2.8T, MiniMax M3, DeepSeekv4, GLM5 - GB200 NVL72 vs MI355X vs B200 vs GB300 NVL72 & soon™ TPUv6e/v7/Trainium2/3 | 开源持续推理基准研究平台…

   Python