Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
tikikun authored Nov 14, 2023
1 parent 52dc578 commit d28291e
Showing 1 changed file with 9 additions and 7 deletions.
16 changes: 9 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,13 +60,6 @@ Download a llama model to try running the llama C++ integration. You can find a

Double-click on Nitro to run it. After downloading your model, make sure it's saved to a specific path. Then, make an API call to load your model into Nitro.

***OPTIONAL***: You can run Nitro on a different port like 5000 instead of 3928 by running it manually in terminal
```zsh
./nitro 1 127.0.0.1 5000 ([thread_num] [host] [port])
```
- thread_num : the number of thread that nitro webserver needs to have
- host : host value normally 127.0.0.1 or 0.0.0.0
- port : the port that nitro got deployed onto

```zsh
curl -X POST 'http://localhost:3928/inferences/llamacpp/loadmodel' \
Expand Down Expand Up @@ -98,6 +91,15 @@ Table of parameters
| `system_prompt` | String | The prompt to use for system rules. |
| `pre_prompt` | String | The prompt to use for internal configuration. |


***OPTIONAL***: You can run Nitro on a different port like 5000 instead of 3928 by running it manually in terminal
```zsh
./nitro 1 127.0.0.1 5000 ([thread_num] [host] [port])
```
- thread_num : the number of thread that nitro webserver needs to have
- host : host value normally 127.0.0.1 or 0.0.0.0
- port : the port that nitro got deployed onto

**Step 4: Perform Inference on Nitro for the First Time**

```zsh
Expand Down

0 comments on commit d28291e

Please sign in to comment.