Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
williamFalcon authored Apr 19, 2024
1 parent 79b46b1 commit 71ed4de
Showing 1 changed file with 13 additions and 11 deletions.
24 changes: 13 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,11 @@ Uses the latest state-of-the-art techniques:
<p align="center">
<a href="https://lightning.ai/">Lightning AI</a> •
<a href="#choose-from-20-llms">Models</a> •
<a href="#install-litgpt">Install</a> •
<a href="#get-started">Get started</a> •
<a href="#use-an-llm">Evaluate</a> •
<a href="#quick-start">Quick start</a> •
<a href="#use-an-llm-for-inference">Inference</a> •
<a href="#finetune-an-llm">Finetune</a> •
<a href="#finetune-an-llm">Pretrain</a> •
<a href="#deploy-an-llm">Deploy</a> •
<a href="#state-of-the-art-features">Features</a> •
<a href="#training-recipes">Training recipes (YAML)</a>
</p>
Expand All @@ -32,7 +32,7 @@ Uses the latest state-of-the-art techniques:
&nbsp;

# Finetune, pretrain and deploy LLMs Lightning fast ⚡⚡
LitGPT is a command-line tool designed to easily [finetune](#finetune-an-llm), [pretrain](#pretrain-an-llm), [evaluate](#use-an-llm), and deploy [20+ LLMs](#choose-from-20-llms) **on your own data**. It features highly-optimized [training recipes](#training-recipes) for the world's most powerful open-source large language models (LLMs).
LitGPT is a command-line tool designed to easily [finetune](#finetune-an-llm), [pretrain](#pretrain-an-llm), [evaluate](#use-an-llm), and [deploy](#deploy-an-llm) [20+ LLMs](#choose-from-20-llms) **on your own data**. It features highly-optimized [training recipes](#training-recipes) for the world's most powerful open-source large language models (LLMs).

We reimplemented all model architectures and training recipes from scratch for 4 reasons:

Expand Down Expand Up @@ -112,7 +112,7 @@ pip install -e '.[all]'

---

# Get started
# Quick start
After installing LitGPT, select the model and action you want to take on that model (finetune, pretrain, evaluate, deploy, etc...):

```bash
Expand All @@ -126,7 +126,8 @@ litgpt serve mistralai/Mistral-7B-Instruct-v0.2

&nbsp;

### Use an LLM
### Use an LLM for inference
Use LLMs for inference to test its chatting capabilities, run evaluations, or extract embeddings, etc...
Here's an example showing how to use the Mistral 7B LLM.

<a target="_blank" href="https://lightning.ai/lightning-ai/studios/litgpt-chat">
Expand Down Expand Up @@ -251,8 +252,7 @@ litgpt chat \
&nbsp;

### Deploy an LLM

This example illustrates how to deploy an LLM using LitGPT.
Once you're ready to deploy a finetuned LLM, run this command:

<a target="_blank" href="https://lightning.ai/lightning-ai/studios/litgpt-serve">
<img src="https://pl-bolts-doc-images.s3.us-east-2.amazonaws.com/app-2/studio-badge.svg" alt="Open In Studio"/>
Expand All @@ -261,13 +261,15 @@ This example illustrates how to deploy an LLM using LitGPT.
&nbsp;

```bash
# 1) Download a pretrained model (alternatively, use your own finetuned model)
litgpt download --repo_id microsoft/phi-2
# locate the checkpoint to your finetuned or pretrained model and call the `serve` command:
litgpt serve --checkpoint_dir path/to/your/checkpoint/microsoft/phi-2

# 2) Start the server
# Alternative: if you haven't finetuned, download any checkpoint to deploy it:
litgpt download --repo_id microsoft/phi-2
litgpt serve --checkpoint_dir checkpoints/microsoft/phi-2
```

Test the server in a separate terminal and integrate the model API into your AI product:
```python
# 3) Use the server (in a separate session)
import requests, json
Expand Down

0 comments on commit 71ed4de

Please sign in to comment.