forked from jupyterlab/jupyter-ai
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add documentation on how to use Amazon Bedrock (jupyterlab#936)
* Bedrock usage documentation Added a new section to user documentation for the use of Amazon Bedrock models, both base models and customized models. * Bedrock docs * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix saving chat settings (jupyterlab#935) * fix settings save * show placeholder message when API keys section is empty * Bedrock usage documentation Added a new section to user documentation for the use of Amazon Bedrock models, both base models and customized models. * Bedrock docs * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Subpage for Bedrock use Added a link in the section on Amazon Bedrock Usage in the docs to a subpage titled `Using Amazon Bedrock with Jupyter AI` which offers a detailed workflow for using Bedrock models, fine tuning them, and calling them in Jupyter AI. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Updates to user docs Addresses comment by Jason Weill * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: david qiu <[email protected]>
- Loading branch information
1 parent
348d857
commit e8cffcb
Showing
11 changed files
with
91 additions
and
5 deletions.
There are no files selected for viewing
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -57,3 +57,5 @@ | |
}, | ||
], | ||
} | ||
|
||
html_sidebars = {"**": []} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,60 @@ | ||
# Using Amazon Bedrock with Jupyter AI | ||
|
||
[(Return to Chat Interface page for Bedrock)](index.md#amazon-bedrock-usage) | ||
|
||
Bedrock supports many language model providers such as AI21 Labs, Amazon, Anthropic, Cohere, Meta, and Mistral AI. To use the base models from any supported provider make sure to enable them in Amazon Bedrock by using the AWS console. Go to Amazon Bedrock and select `Model Access` as shown here: | ||
|
||
<img src="../_static/bedrock-model-access.png" | ||
width="75%" | ||
alt='Screenshot of the left panel in the AWS console where Bedrock model access is provided.' | ||
class="screenshot" /> | ||
|
||
Click through on `Model Access` and follow the instructions to grant access to the models you wish to use, as shown below. Make sure to accept the end user license (EULA) as required by each model. You may need your system administrator to grant access to your account if you do not have authority to do so. | ||
|
||
<img src="../_static/bedrock-model-select.png" | ||
width="75%" | ||
alt='Screenshot of the Bedrock console where models may be selected.' | ||
class="screenshot" /> | ||
|
||
You should also select embedding models in addition to language completion models if you intend to use retrieval augmented generation (RAG) on your documents. | ||
|
||
You may now select a chosen Bedrock model from the drop-down menu box title `Completion model` in the chat interface. If RAG is going to be used then pick an embedding model that you chose from the Bedrock models as well. An example of these selections is shown below: | ||
|
||
<img src="../_static/bedrock-chat-basemodel.png" | ||
width="50%" | ||
alt='Screenshot of the Jupyter AI chat panel where the base language model and embedding model is selected.' | ||
class="screenshot" /> | ||
|
||
Bedrock also allows custom models to be trained from scratch or fine-tuned from a base model. Jupyter AI enables a custom model to be called in the chat panel using its `arn` (Amazon Resource Name). As with custom models, you can also call a base model by its `model id` or its `arn`. An example of using a base model with its `model id` through the custom model interface is shown below: | ||
|
||
<img src="../_static/bedrock-chat-basemodel-modelid.png" | ||
width="75%" | ||
alt='Screenshot of the Jupyter AI chat panel where the base model is selected using model id.' | ||
class="screenshot" /> | ||
|
||
An example of using a base model using its `arn` through the custom model interface is shown below: | ||
|
||
<img src="../_static/bedrock-chat-basemodel-arn.png" | ||
width="75%" | ||
alt='Screenshot of the Jupyter AI chat panel where the base model is selected using its ARN.' | ||
class="screenshot" /> | ||
|
||
To train a custom model in Amazon Bedrock, select `Custom models` in the Bedrock console as shown below, and then you may customize a base model by fine-tuning it or continuing to pre-train it: | ||
|
||
<img src="../_static/bedrock-custom-models.png" | ||
width="75%" | ||
alt='Screenshot of the Bedrock custom models access in the left panel of the Bedrock console.' | ||
class="screenshot" /> | ||
|
||
For details on fine-tuning a base model from Bedrock, see this [reference](https://aws.amazon.com/blogs/aws/customize-models-in-amazon-bedrock-with-your-own-data-using-fine-tuning-and-continued-pre-training/); with related [documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/custom-models.html). | ||
|
||
Once the model is fine-tuned, it will have its own `arn`, as shown below: | ||
|
||
<img src="../_static/bedrock-finetuned-model.png" | ||
width="75%" | ||
alt='Screenshot of the Bedrock fine-tuned model ARN in the Bedrock console.' | ||
class="screenshot" /> | ||
|
||
As seen above, you may click on `Purchase provisioned throughput` to buy inference units with which to call the custom model's API. Enter the model's `arn` in Jupyter AI's Language model user interface to use the provisioned model. | ||
|
||
[(Return to Chat Interface page for Bedrock)](index.md#amazon-bedrock-usage) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters