Popular repositories Loading

1. llm-compressor llm-compressor Public

   Forked from vllm-project/llm-compressor

   Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

   Python

2. xoscar xoscar Public

   Forked from xorbitsai/xoscar

   Python actor framework for heterogeneous computing.

   Python

3. inference inference Public

   Forked from xorbitsai/inference

   Replace OpenAI GPT with another LLM in your app by changing a single line of code. Xinference gives you the freedom to use any LLM you need. With Xinference, you're empowered to run inference with …

   Python

4. mmengine mmengine Public

   Forked from open-mmlab/mmengine

   OpenMMLab Foundational Library for Training Deep Learning Models

   Python

5. GPTQModel GPTQModel Public

   Forked from ModelCloud/GPTQModel

   GPTQ based LLM model compression/quantization toolkit with accelerated inference support for both cpu/gpu via HF, vLLM, and SGLang.

   Python

6. vllm vllm Public

   Forked from vllm-project/vllm

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python