Skip to content

Commit

Permalink
Merge pull request #181 from janhq/hotfix-docs
Browse files Browse the repository at this point in the history
Update docs
  • Loading branch information
tikikun authored Nov 23, 2023
2 parents 64cc293 + 07f6ac1 commit 9fee2cb
Show file tree
Hide file tree
Showing 7 changed files with 84 additions and 65 deletions.
24 changes: 24 additions & 0 deletions docs/docs/demos/chatbox-vid.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
---
title: Run local chatbox under 1 minute on MacOS with Nitro
---

<iframe width="675" height="380" src="https://www.youtube.com/embed/apfI13ijvjc" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

## Links

- [Download Nitro](https://github.com/janhq/nitro/releases)
- [Download Chatbox](https://github.com/Bin-Huang/chatbox)

## Commands

```bash title="Load model"
curl http://localhost:3928/inferences/llamacpp/loadmodel \
-H 'Content-Type: application/json' \
-d '{
"llama_model_path": "model/llama-2-7b-chat.Q5_K_M.gguf",
"ctx_len": 512,
"ngl": 100,
}'
```

For more information, please refer to the [Nitro with Chatbox](examples/chatbox.md) documentation.
24 changes: 18 additions & 6 deletions docs/docs/examples/chatbox.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,16 +16,25 @@ To download and install Chatbox, follow the instructions available at this [link

## Using Nitro as a Backend

1. Start Nitro server
**1. Start Nitro server**

Open your command line tool and enter:
```
nitro
```

> Ensure you are using the latest version of [Nitro](new/install.md)
**2. Download Model**

2. Run the Model
Use these commands to download and save the [Llama2 7B chat model](https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGUF/tree/main):

```bash
mkdir model && cd model
wget -O llama-2-7b-model.gguf https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGUF/resolve/main/llama-2-7b-chat.Q5_K_M.gguf?download=true
```

> For more GGUF model, please look at [The Bloke](https://huggingface.co/TheBloke).
**3. Run the Model**

To load the model, use the following command:

Expand All @@ -39,13 +48,16 @@ curl http://localhost:3928/inferences/llamacpp/loadmodel \
}'
```

3. Config chatbox
**4. Config chatbox**

Adjust the `settings` in Chatbox to connect with Nitro. Change your settings to match the configuration shown in the image below:

![Settings](img/chatbox.PNG)

4. Chat with the Model
**5. Chat with the Model**

Once the setup is complete, you can start chatting with the model using Chatbox. All functions of Chatbox are now enabled with Nitro as the backend.

## Video demo
## Futher Usage

For convenient usage, you can utilize [Jan](https://jan.ai/), as it is integrated with Nitro.
18 changes: 18 additions & 0 deletions docs/docs/examples/jan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
---
title: Nitro with Jan
---

You can effortlessly utilize Nitro through [Jan](https://jan.ai/), as it is fully integrated with all its functions. With Jan, using Nitro becomes straightforward without the need for any coding.


<iframe width="675" height="380" src="https://www.youtube.com/embed/hw9w8_54OIc" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

## What is Jan?

Jan is a ChatGPT-alternative that runs on your own computer, with a local API server.

Jan uses open-source AI models, stores data in open file formats, is highly customizable via extensions.

For additional details, please consult the [Jan Documenation](https://jan.ai/docs).

> [Download Jan](https://jan.ai/)
54 changes: 0 additions & 54 deletions docs/docs/examples/llm.md

This file was deleted.

2 changes: 1 addition & 1 deletion docs/docs/new/about.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: About Nitro
slug: /about
slug: /docs
---

Nitro is a high-efficiency C++ inference engine for edge computing, powering [Jan](https://jan.ai/). It is lightweight and embeddable, ideal for product integration.
Expand Down
4 changes: 2 additions & 2 deletions docs/docs/new/build-source.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,13 +72,13 @@ Time to build Nitro!
- **On Linux:**

```bash
make -j $(%NUMBER_OF_PROCESSORS%)
make -j $(nproc)
```

- **On Windows:**

```bash
cmake --build . --config Release
make -j $(%NUMBER_OF_PROCESSORS%)
```

## Start process
Expand Down
23 changes: 21 additions & 2 deletions docs/sidebars.js
Original file line number Diff line number Diff line change
Expand Up @@ -33,10 +33,19 @@ const sidebars = {
{
type: "category",
label: "Features",
collapsible: false,
collapsed: false,
link: { type: "doc", id: "features/feat" },
items: [
"features/chat",
"features/embed",
],
},
{
type: "category",
label: "Advanced Features",
link: { type: "doc", id: "features/feat" },
items: [
"features/multi-thread",
"features/cont-batch",
"features/load-unload",
Expand All @@ -46,10 +55,11 @@ const sidebars = {
},
{
type: "category",
label: "Guides",
label: "Integrations",
collapsible: false,
collapsed: false,
items: [
"examples/jan",
"examples/chatbox",
"examples/openai-node",
"examples/openai-python",
Expand All @@ -62,7 +72,16 @@ const sidebars = {
// collapsed: false,
// items: [{ type: "doc", id: "new/architecture", label: "Architecture" }],
// },
"new/faq"
{
type: "category",
label: "Demos",
collapsible: true,
collapsed: true,
items: [
"demos/chatbox-vid",
],
},
"new/faq",
],

apiSidebar: [
Expand Down

0 comments on commit 9fee2cb

Please sign in to comment.