Popular repositories Loading

1. Sequoia Sequoia Public

   scalable and robust tree-based speculative decoding algorithm

   Python 345 37

2. TriForce TriForce Public

   [COLM 2024] TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding

   Python 251 17

3. MagicPIG MagicPIG Public

   [ICLR2025 Spotlight] MagicPIG: LSH Sampling for Efficient LLM Generation

   Python 212 16

4. MagicDec MagicDec Public

   [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding

   Python 116 9

5. UMbreLLa UMbreLLa Public

   LLM Inference on consumer devices

   Python 115 15

6. gsm_infinite gsm_infinite Public

   Python 45 3