Pinned Loading

1. yali yali Public

   Speed-of-Light SW efficiency by using ultra low-latency primitives for comms collectives

   Cuda 6

2. vllm-project/aibrix vllm-project/aibrix Public

   Cost-efficient and pluggable Infrastructure components for GenAI inference

   Go 4.5k 516

3. ai-dynamo/dynamo ai-dynamo/dynamo Public

   A Datacenter Scale Distributed Inference Serving Framework

   Rust 5.8k 789

4. sgl-project/sglang sgl-project/sglang Public

   SGLang is a high-performance serving framework for large language models and multimodal models.

   Python 22.6k 4.1k

5. tokasaurus tokasaurus Public

   Forked from ScalingIntelligence/tokasaurus

   Python

6. LMCache LMCache Public

   Forked from LMCache/LMCache

   Supercharge Your LLM with the Fastest KV Cache Layer

   Python