Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: second half of "import model docs" PR #1021

Merged
merged 4 commits into from
Dec 15, 2023
Merged

docs: second half of "import model docs" PR #1021

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
80b6c01
nits
Dec 15, 2023
f942560
nits
Dec 15, 2023
473f99e
deconflict
Dec 15, 2023
0ecd06e
Merge branch 'main' into importModelDocs
hahuyhoang411 Dec 15, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 17 additions & 11 deletions docs/docs/guides/quickstart.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@ Navigate to `~/jan/models` folder on your computer.

In `App Settings`, go to `Advanced`, then `Open App Directory`.

Or, you can directly cd into:

```sh
# Windows
C:/Users/<your_user_name>/jan/models
Expand All @@ -35,21 +37,21 @@ jan/models
In the `models` folder, create a folder with the name of the model.

```sh
mkdir pandora-v1-q4
mkdir trinity-v1-7b
```

## 2. Create a model JSON

Jan follows a standardized model template, called a `model.json`. This allows for easy model configurations, exporting, and sharing.
Jan follows a folder-based, [standard model template](/specs/models) called a `model.json`. This allows for easy model configurations, exporting, and sharing.

```sh
cd pandora-v1-q4
cd trinity-v1-7b
touch model.json
```

The following is an example template for `model.json`
Copy the following into the `model.json`

```
```js
{
"source_url": "https://huggingface.co/janhq/trinity-v1-GGUF/resolve/main/trinity-v1.Q4_K_M.gguf",
"id": "trinity-v1-7b",
Expand All @@ -71,21 +73,25 @@ The following is an example template for `model.json`
"size": 4370000000
},
"engine": "nitro"
}
}
```

:::caution
Ensure the `source_url` is the link to download model
Ensure the `source_url` property is the direct binary download link ending in `.gguf`. Find the links in Huggingface > `Files and versions`.

Ensure the `id` is the same with the new created folder
Ensure the `id` property is the model foldername

Ensure to choose right `prompt_template`
Ensure you are using the correct `prompt_template`
:::

# 3. Use your model
## 3. Download your model binary

Restart the Jan application and look for your model in the Hub.

Click the green `download` button to download your actual model binary from the URL you provided.

![image](https://hackmd.io/_uploads/HJLAqvwI6.png)

There you go. If you have any questions or want to request for more preconfigured GGUF models, please message us on [Jan Discord](https://discord.gg/Dt7MxDyNNZ).
There you go! You are ready to use your model.

If you have any questions or want to request for more preconfigured GGUF models, please message us in [Discord](https://discord.gg/Dt7MxDyNNZ).