Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
williamFalcon authored Apr 16, 2024
1 parent 412e0fe commit 03cb166
Showing 1 changed file with 9 additions and 8 deletions.
17 changes: 9 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,11 @@

# ⚡ LitGPT

**Pretrain, finetune, deploy 20+ LLMs on your own data**
**Pretrain, finetune, evaluate, and deploy 20+ LLMs on your own data**

Uses the latest state-of-the-art techniques:

✅ fp4/8/16/32     ✅ LoRA, QLoRA, Adapter (v1, v2)     ✅ flash attention     ✅ FSDP     ✅ 1-1000+ GPUs/TPUs
flash attention    fp4/8/16/32     ✅ LoRA, QLoRA, Adapter (v1, v2)     ✅ FSDP     ✅ 1-1000+ GPUs/TPUs

---

Expand All @@ -32,14 +32,15 @@ Uses the latest state-of-the-art techniques:

 

## Finetune, pretrain and deploy AI models Lightning fast ⚡⚡
LitGPT is a command-line tool to use, pretrain, finetune and deploy LLMs. It is based on configs with highly-optimized recipes for training the world's largest, most powerful open-source LLMs.
## Finetune, pretrain and deploy LLMs Lightning fast ⚡⚡
LitGPT is a command-line tool designed to easily [finetune](#finetune-an-llm), [pretrain](#pretrain-an-llm), [evaluate](#use-an-llm), and deploy [20+ LLMs](#choose-from-20-llms) **on your own data**. It features highly-optimized [training recipes](#training-recipes) for the world's most powerful open-source large-language-models (LLMs).

We've reimplemented all the model architectures and training recipes for 3 reasons:
We reimplemented all model architectures and training recipes from scratch for 4 reasons:

1. Remove all abstraction layers and have single file implementations.
2. Guarantee Apache 2.0 compliance to enable enterprise use without limits.
3. Optimized every detail of every model to get the fastest performance possible to lower cost and training speeds.
1. Remove all abstraction layers and have single file implementations.
2. Guarantee Apache 2.0 compliance to enable enterprise use without limits.
3. Optimized every detail of every model to get the fastest performance possible to lower cost and training speeds.
4. Config based for highly-optimized recipes.

 

Expand Down

0 comments on commit 03cb166

Please sign in to comment.