Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs[minor]: Update bedrock llm integration doc #6331

Merged
merged 4 commits into from
Aug 2, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion deno.json
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,13 @@
"youtube-transcript": "npm:/youtube-transcript",
"neo4j-driver": "npm:/neo4j-driver",
"axios": "npm:/axios",
"@mendable/firecrawl-js": "npm:/@mendable/firecrawl-js"
"@mendable/firecrawl-js": "npm:/@mendable/firecrawl-js",
"@aws-crypto/sha256-js": "npm:/@aws-crypto/sha256-js",
"@aws-sdk/credential-provider-node": "npm:/@aws-sdk/credential-provider-node",
"@smithy/protocol-http": "npm:/@smithy/protocol-http",
"@smithy/signature-v4": "npm:/@smithy/signature-v4",
"@smithy/eventstream-codec": "npm:/@smithy/eventstream-codec",
"@smithy/util-utf8": "npm:/@smithy/util-utf8",
"@aws-sdk/types": "npm:/@aws-sdk/types"
}
}
284 changes: 284 additions & 0 deletions docs/core_docs/docs/integrations/llms/bedrock.ipynb
Original file line number Diff line number Diff line change
@@ -0,0 +1,284 @@
{
"cells": [
{
"cell_type": "raw",
"id": "67db2992",
"metadata": {},
"source": [
"---\n",
"sidebar_label: Bedrock\n",
"---"
]
},
{
"cell_type": "markdown",
"id": "9597802c",
"metadata": {},
"source": [
"# Bedrock\n",
"\n",
"```{=mdx}\n",
"\n",
":::caution\n",
"You are currently on a page documenting the use of Amazon Bedrock models as [text completion models](/docs/concepts/#llms). Many popular models available on Bedrock are [chat completion models](/docs/concepts/#chat-models).\n",
"\n",
"You may be looking for [this page instead](/docs/integrations/chat/bedrock/).\n",
":::\n",
"\n",
"```\n",
"\n",
"> [Amazon Bedrock](https://aws.amazon.com/bedrock/) is a fully managed service that makes Foundation Models (FMs)\n",
"> from leading AI startups and Amazon available via an API. You can choose from a wide range of FMs to find the model that is best suited for your use case.\n",
"\n",
"This will help you get started with Bedrock completion models (LLMs) using LangChain. For detailed documentation on `Bedrock` features and configuration options, please refer to the [API reference](https://api.js.langchain.com/classes/langchain_community_llms_bedrock.Bedrock.html).\n",
"\n",
"## Overview\n",
"### Integration details\n",
"\n",
"- TODO: Fill in table features.\n",
"- TODO: Remove JS support link if not relevant, otherwise ensure link is correct.\n",
"- TODO: Make sure API reference links are correct.\n",
"\n",
"| Class | Package | Local | Serializable | [PY support](https://python.langchain.com/docs/integrations/llms/bedrock) | Package downloads | Package latest |\n",
"| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n",
"| [Bedrock](https://api.js.langchain.com/classes/langchain_community_llms_bedrock.Bedrock.html) | [@langchain/community](https://api.js.langchain.com/modules/langchain_community_llms_bedrock.html) | ❌ | ✅ | ✅ | ![NPM - Downloads](https://img.shields.io/npm/dm/@langchain/community?style=flat-square&label=%20&) | ![NPM - Version](https://img.shields.io/npm/v/@langchain/community?style=flat-square&label=%20&) |\n",
"\n",
"## Setup\n",
"\n",
"To access Bedrock models you'll need to create an AWS account, get an API key, and install the `@langchain/community` integration, along with a few peer dependencies.\n",
"\n",
"### Credentials\n",
"\n",
"Head to [aws.amazon.com](https://aws.amazon.com) to sign up to AWS Bedrock and generate an API key. Once you've done this set the environment variables:\n",
"\n",
"```bash\n",
"export BEDROCK_AWS_REGION=\"your-region-url\"\n",
"export BEDROCK_AWS_ACCESS_KEY_ID=\"your-access-key-id\"\n",
"export BEDROCK_AWS_SECRET_ACCESS_KEY=\"your-secret-access-key\"\n",
"```\n",
"\n",
"If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:\n",
"\n",
"```bash\n",
"# export LANGCHAIN_TRACING_V2=\"true\"\n",
"# export LANGCHAIN_API_KEY=\"your-api-key\"\n",
"```\n",
"\n",
"### Installation\n",
"\n",
"The LangChain Bedrock integration lives in the `@langchain/community` package:\n",
"\n",
"```{=mdx}\n",
"import IntegrationInstallTooltip from \"@mdx_components/integration_install_tooltip.mdx\";\n",
"import Npm2Yarn from \"@theme/Npm2Yarn\";\n",
"\n",
"<IntegrationInstallTooltip></IntegrationInstallTooltip>\n",
"\n",
"<Npm2Yarn>\n",
" @langchain/community\n",
"</Npm2Yarn>\n",
"\n",
"And install the peer dependencies:\n",
"\n",
"<Npm2Yarn>\n",
" @aws-crypto/sha256-js @aws-sdk/credential-provider-node @smithy/protocol-http @smithy/signature-v4 @smithy/eventstream-codec @smithy/util-utf8 @aws-sdk/types\n",
"</Npm2Yarn>\n",
"\n",
"You can also use Bedrock in web environments such as Edge functions or Cloudflare Workers by omitting the `@aws-sdk/credential-provider-node` dependency\n",
"and using the `web` entrypoint:\n",
"\n",
"<Npm2Yarn>\n",
" @aws-crypto/sha256-js @smithy/protocol-http @smithy/signature-v4 @smithy/eventstream-codec @smithy/util-utf8 @aws-sdk/types\n",
"</Npm2Yarn>\n",
"\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "0a760037",
"metadata": {},
"source": [
"## Instantiation\n",
"\n",
"Now we can instantiate our model object and generate chat completions:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "093ae37f",
"metadata": {},
"outputs": [],
"source": [
"// @lc-docs-hide-cell\n",
"// Deno requires these imports, and way of loading env vars.\n",
"// we don't want to expose in the docs.\n",
"// Below this cell we have a typescript markdown codeblock with\n",
"// the node code.\n",
"import \"@aws-sdk/credential-provider-node\";\n",
"import \"@smithy/protocol-http\";\n",
"import \"@aws-crypto/sha256-js\";\n",
"import \"@smithy/protocol-http\";\n",
"import \"@smithy/signature-v4\";\n",
"import \"@smithy/eventstream-codec\";\n",
"import \"@smithy/util-utf8\";\n",
"import \"@aws-sdk/types\";\n",
"import { Bedrock } from \"@langchain/community/llms/bedrock\"\n",
"import { getEnvironmentVariable } from \"@langchain/core/utils/env\";\n",
"\n",
"const llm = new Bedrock({\n",
" model: \"anthropic.claude-v2\",\n",
" region: \"us-east-1\",\n",
" // endpointUrl: \"custom.amazonaws.com\",\n",
" credentials: {\n",
" accessKeyId: getEnvironmentVariable(\"BEDROCK_AWS_ACCESS_KEY_ID\"),\n",
" secretAccessKey: getEnvironmentVariable(\"BEDROCK_AWS_SECRET_ACCESS_KEY\"),\n",
" },\n",
" temperature: 0,\n",
" maxTokens: undefined,\n",
" maxRetries: 2,\n",
" // other params...\n",
"})"
]
},
{
"cell_type": "markdown",
"id": "a0562a13",
"metadata": {},
"source": [
"```typescript\n",
"import { Bedrock } from \"@langchain/community/llms/bedrock\"\n",
"\n",
"const llm = new Bedrock({\n",
" model: \"anthropic.claude-v2\",\n",
" region: process.env.BEDROCK_AWS_REGION ?? \"us-east-1\",\n",
" // endpointUrl: \"custom.amazonaws.com\",\n",
" credentials: {\n",
" accessKeyId: process.env.BEDROCK_AWS_ACCESS_KEY_ID,\n",
" secretAccessKey: process.env.BEDROCK_AWS_SECRET_ACCESS_KEY,\n",
" },\n",
" temperature: 0,\n",
" maxTokens: undefined,\n",
" maxRetries: 2,\n",
" // other params...\n",
"})\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "0ee90032",
"metadata": {},
"source": [
"## Invocation\n",
"\n",
"Note that some models require specific prompting techniques. For example, Anthropic's Claude-v2 model will throw an error if\n",
"the prompt does not start with `Human: `."
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "035dea0f",
"metadata": {
"tags": []
},
"outputs": [
{
"data": {
"text/plain": [
"\u001b[32m\" Here are a few key points about Bedrock AI:\\n\"\u001b[39m +\n",
" \u001b[32m\"\\n\"\u001b[39m +\n",
" \u001b[32m\"- Bedrock was founded in 2021 and is based in San Fran\"\u001b[39m... 116 more characters"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"const inputText = \"Human: Bedrock is an AI company that\\nAssistant: \"\n",
"\n",
"const completion = await llm.invoke(inputText)\n",
"completion"
]
},
{
"cell_type": "markdown",
"id": "add38532",
"metadata": {},
"source": [
"## Chaining\n",
"\n",
"We can [chain](/docs/how_to/sequence/) our completion model with a prompt template like so:"
]
},
{
"cell_type": "code",
"execution_count": 21,
"id": "078e9db2",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"\u001b[32m' Here is how to say \"I love programming\" in German:\\n'\u001b[39m +\n",
" \u001b[32m\"\\n\"\u001b[39m +\n",
" \u001b[32m\"Ich liebe das Programmieren.\"\u001b[39m"
]
},
"execution_count": 21,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import { PromptTemplate } from \"@langchain/core/prompts\"\n",
"\n",
"const prompt = PromptTemplate.fromTemplate(\"Human: How to say {input} in {output_language}:\\nAssistant:\")\n",
"\n",
"const chain = prompt.pipe(llm);\n",
"await chain.invoke(\n",
" {\n",
" output_language: \"German\",\n",
" input: \"I love programming.\",\n",
" }\n",
")"
]
},
{
"cell_type": "markdown",
"id": "e9bdfcef",
"metadata": {},
"source": [
"## API reference\n",
"\n",
"For detailed documentation of all Bedrock features and configurations head to the API reference: https://api.js.langchain.com/classes/langchain_community_llms_bedrock.Bedrock.html"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Deno",
"language": "typescript",
"name": "deno"
},
"language_info": {
"file_extension": ".ts",
"mimetype": "text/x.typescript",
"name": "typescript",
"nb_converter": "script",
"pygments_lexer": "typescript",
"version": "5.3.3"
},
"vscode": {
"interpreter": {
"hash": "e971737741ff4ec9aff7dc6155a1060a59a8a6d52c757dbbe66bf8ee389494b1"
}
}
},
"nbformat": 4,
"nbformat_minor": 5
}
43 changes: 0 additions & 43 deletions docs/core_docs/docs/integrations/llms/bedrock.mdx

This file was deleted.

4 changes: 1 addition & 3 deletions libs/langchain-scripts/src/cli/docs/templates/llms.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -160,9 +160,7 @@
"source": [
"import { PromptTemplate } from \"@langchain/core/prompts\"\n",
"\n",
"const prompt = new PromptTemplate({\n",
" template: \"How to say {input} in {output_language}:\\n\",\n",
"})\n",
"const prompt = PromptTemplate.fromTemplate(\"How to say {input} in {output_language}:\\n\")\n",
"\n",
"const chain = prompt.pipe(llm);\n",
"await chain.invoke(\n",
Expand Down
Loading