refactor: Resolve accidental merge change

README.md

@@ -60,6 +60,7 @@ Download a llama model to try running the llama C++ integration. You can find a
 
 Double-click on Nitro to run it. After downloading your model, make sure it's saved to a specific path. Then, make an API call to load your model into Nitro.
 
+
 ```zsh
 curl -X POST 'http://localhost:3928/inferences/llamacpp/loadmodel' \
   -H 'Content-Type: application/json' \
@@ -90,6 +91,15 @@ Table of parameters
 | `system_prompt`    | String  | The prompt to use for system rules.                          |
 | `pre_prompt`    | String  | The prompt to use for internal configuration.                          |
 
+
+***OPTIONAL***: You can run Nitro on a different port like 5000 instead of 3928 by running it manually in terminal
+```zsh
+./nitro 1 127.0.0.1 5000 ([thread_num] [host] [port])
+```
+- thread_num : the number of thread that nitro webserver needs to have
+- host : host value normally 127.0.0.1 or 0.0.0.0
+- port : the port that nitro got deployed onto
+
 **Step 4: Perform Inference on Nitro for the First Time**
 
 ```zsh

docs/docs/examples/llm.md

@@ -0,0 +1,54 @@
+---
+title: Simple chatbot with Nitro
+---
+
+This guide provides instructions to create a chatbot powered by Nitro using the GGUF model.
+
+## Step 1: Download the Model
+
+First, you'll need to download the chatbot model.
+
+1. **Navigate to the Models Folder**
+   - Open your project directory.
+   - Locate and open the `models` folder within the directory.
+
+2. **Select a GGUF Model**
+   - Visit the Hugging Face repository at [TheBloke's Models](https://huggingface.co/TheBloke).
+   - Browse through the available models.
+   - Choose the model that best fits your needs.
+
+3. **Download the Model**
+   - Once you've selected a model, download it using a command like the one below. Replace `<llama_model_path>` with the path of your chosen model.
+
+
+```bash title="Downloading Zephyr 7B Model"
+wget https://huggingface.co/TheBloke/zephyr-7B-beta-GGUF/resolve/main/zephyr-7b-beta.Q5_K_M.gguf?download=true
+```
+
+## Step 2: Load model
+Now, you'll set up the model in your application.
+
+1. **Open `app.py` File**
+
+    - In your project directory, find and open the app.py file.
+
+2. **Configure the Model Path**
+
+    - Modify the model path in app.py to point to your downloaded model.
+    - Update the configuration parameters as necessary.
+
+```bash title="Example Configuration" {2}
+dat = {
+    "llama_model_path": "nitro/interface/models/zephyr-7b-beta.Q5_K_M.gguf",
+    "ctx_len": 2048,
+    "ngl": 100,
+    "embedding": True,
+    "n_parallel": 4,
+    "pre_prompt": "A chat between a curious user and an artificial intelligence",
+    "user_prompt": "USER: ",
+    "ai_prompt": "ASSISTANT: "}
+```
+
+Congratulations! Your Nitro chatbot is now set up. Feel free to experiment with different configuration parameters to tailor the chatbot to your needs.
+
+For more information on parameter settings and their effects, please refer to Run Nitro(using-nitro) for a comprehensive parameters table.

docs/docs/features/chat.md

@@ -0,0 +1,172 @@
+---
+title: Chat Completion
+---
+
+The Chat Completion feature in Nitro provides a flexible way to interact with any local Large Language Model (LLM).
+
+## Single Request Example
+
+To send a single query to your chosen LLM, follow these steps:
+
+<div style={{ width: '50%', float: 'left', clear: 'left' }}>
+
+```bash title="Nitro"
+curl http://localhost:3928/inferences/llamacpp/chat_completion \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "",
+    "messages": [
+      {
+        "role": "user",
+        "content": "Hello"
+      },
+    ]
+  }'
+
+```
+</div>
+
+<div style={{ width: '50%', float: 'right', clear: 'right' }}>
+
+```bash title="OpenAI"
+curl https://api.openai.com/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -H "Authorization: Bearer $OPENAI_API_KEY" \
+  -d '{
+    "model": "gpt-3.5-turbo",
+    "messages": [
+      {
+        "role": "user",
+        "content": "Hello"
+      }
+    ]
+  }'
+```
+</div>
+
+This command sends a request to your local LLM, querying about the winner of the 2020 World Series.
+
+### Dialog Request Example
+
+For ongoing conversations or multiple queries, the dialog request feature is ideal. Here’s how to structure a multi-turn conversation:
+
+<div style={{ width: '50%', float: 'left', clear: 'left' }}>
+
+```bash title="Nitro"
+curl http://localhost:3928/inferences/llamacpp/chat_completion \
+  -H "Content-Type: application/json" \
+  -d '{
+    "messages": [
+      {
+        "role": "system",
+        "content": "You are a helpful assistant."
+      },
+      {
+        "role": "user",
+        "content": "Who won the world series in 2020?"
+      },
+      {
+        "role": "assistant",
+        "content": "The Los Angeles Dodgers won the World Series in 2020."
+      },
+      {
+        "role": "user",
+        "content": "Where was it played?"
+      }
+    ]
+  }'
+
+```
+</div>
+
+<div style={{ width: '50%', float: 'right', clear: 'right' }}>
+
+```bash title="OpenAI"
+curl https://api.openai.com/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -H "Authorization: Bearer $OPENAI_API_KEY" \
+  -d '{
+    "messages": [
+      {
+        "role": "system",
+        "content": "You are a helpful assistant."
+      },
+      {
+        "role": "user",
+        "content": "Who won the world series in 2020?"
+      },
+      {
+        "role": "assistant",
+        "content": "The Los Angeles Dodgers won the World Series in 2020."
+      },
+      {
+        "role": "user",
+        "content": "Where was it played?"
+      }
+    ]
+  }'
+```
+</div>
+
+### Chat Completion Response
+
+Below are examples of responses from both the Nitro server and OpenAI:
+
+<div style={{ width: '50%', float: 'left', clear: 'left' }}>
+
+```js title="Nitro"
+{
+  "choices": [
+    {
+      "finish_reason": null,
+      "index": 0,
+      "message": {
+        "content": "Hello, how may I assist you this evening?",
+        "role": "assistant"
+      }
+    }
+  ],
+  "created": 1700215278,
+  "id": "sofpJrnBGUnchO8QhA0s",
+  "model": "_",
+  "object": "chat.completion",
+  "system_fingerprint": "_",
+  "usage": {
+    "completion_tokens": 13,
+    "prompt_tokens": 90,
+    "total_tokens": 103
+  }
+}
+```
+</div>
+
+<div style={{ width: '50%', float: 'right', clear: 'right' }}>
+
+```js title="OpenAI"
+{
+  "choices": [
+    {
+      "finish_reason": "stop"
+      "index": 0,
+      "message": {
+        "role": "assistant",
+        "content": "Hello there, how may I assist you today?",
+      }
+    }
+  ],
+  "created": 1677652288,
+  "id": "chatcmpl-123",
+  "model": "gpt-3.5-turbo-0613",
+  "object": "chat.completion",
+  "system_fingerprint": "fp_44709d6fcb",
+  "usage": {
+    "completion_tokens": 12,
+    "prompt_tokens": 9,
+    "total_tokens": 21
+  }
+}
+```
+</div>
+
+
+The chat completion feature in Nitro showcases compatibility with OpenAI, making the transition between using OpenAI and local AI models more straightforward. For further details and advanced usage, please refer to the [API reference](https://nitro.jan.ai/api).