-
Notifications
You must be signed in to change notification settings - Fork 2.3k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge remote-tracking branch 'origin/feat/deepinfra-llm-and-chat' int…
…o feat/deepinfra-llm-and-chat
- Loading branch information
Showing
114 changed files
with
3,292 additions
and
1,129 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,92 @@ | ||
name: Standard Tests (Integration) | ||
|
||
on: | ||
workflow_dispatch: | ||
schedule: | ||
- cron: '0 13 * * *' | ||
|
||
jobs: | ||
standard-tests: | ||
runs-on: ubuntu-latest | ||
strategy: | ||
matrix: | ||
package: [anthropic, cohere, google-genai, groq, mistralai] | ||
steps: | ||
- uses: actions/checkout@v4 | ||
- name: Use Node.js 18.x | ||
uses: actions/setup-node@v3 | ||
with: | ||
node-version: 18.x | ||
cache: "yarn" | ||
- name: Install dependencies | ||
run: yarn install --immutable --mode=skip-build | ||
- name: Run standard tests (integration) for ${{ matrix.package }} | ||
run: yarn test:standard:int --filter=@langchain/${{ matrix.package }} | ||
env: | ||
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} | ||
COHERE_API_KEY: ${{ secrets.COHERE_API_KEY }} | ||
GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }} | ||
GROQ_API_KEY: ${{ secrets.GROQ_API_KEY }} | ||
MISTRAL_API_KEY: ${{ secrets.MISTRAL_API_KEY }} | ||
|
||
# The `@langchain/openai` package contains standard tests for ChatOpenAI and AzureChatOpenAI | ||
# We want to run these separately, so we need to pass the exact path for each test, which means | ||
# we need separate jobs for each test. | ||
standard-tests-openai: | ||
runs-on: ubuntu-latest | ||
steps: | ||
- uses: actions/checkout@v4 | ||
- name: Use Node.js 18.x | ||
uses: actions/setup-node@v3 | ||
with: | ||
node-version: 18.x | ||
cache: "yarn" | ||
- name: Install dependencies | ||
run: yarn install --immutable --mode=skip-build | ||
- name: Build `@langchain/openai` | ||
run: yarn build --filter=@langchain/openai | ||
- name: Run standard tests (integration) for ChatOpenAI | ||
run: yarn workspace @langchain/openai test:single src/tests/chat_models.standard.int.test.ts | ||
env: | ||
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }} | ||
|
||
standard-tests-azure-openai: | ||
runs-on: ubuntu-latest | ||
steps: | ||
- uses: actions/checkout@v4 | ||
- name: Use Node.js 18.x | ||
uses: actions/setup-node@v3 | ||
with: | ||
node-version: 18.x | ||
cache: "yarn" | ||
- name: Install dependencies | ||
run: yarn install --immutable --mode=skip-build | ||
- name: Build `@langchain/openai` | ||
run: yarn build --filter=@langchain/openai | ||
- name: Run standard tests (integration) for `@langchain/openai` AzureChatOpenAI | ||
run: yarn workspace @langchain/openai test:single src/tests/azure/chat_models.standard.int.test.ts | ||
env: | ||
AZURE_OPENAI_API_KEY: ${{ secrets.AZURE_OPENAI_API_KEY }} | ||
AZURE_OPENAI_API_DEPLOYMENT_NAME: "chat" | ||
AZURE_OPENAI_API_VERSION: ${{ secrets.AZURE_OPENAI_API_VERSION }} | ||
AZURE_OPENAI_BASE_PATH: ${{ secrets.AZURE_OPENAI_BASE_PATH }} | ||
|
||
standard-tests-bedrock: | ||
runs-on: ubuntu-latest | ||
steps: | ||
- uses: actions/checkout@v4 | ||
- name: Use Node.js 18.x | ||
uses: actions/setup-node@v3 | ||
with: | ||
node-version: 18.x | ||
cache: "yarn" | ||
- name: Install dependencies | ||
run: yarn install --immutable --mode=skip-build | ||
- name: Build `@langchain/community` | ||
run: yarn build --filter=@langchain/community | ||
- name: Run standard tests (integration) for `@langchain/community` BedrockChat | ||
run: yarn workspace @langchain/community test:single src/chat_models/tests/chatbedrock.standard.int.test.ts | ||
env: | ||
BEDROCK_AWS_REGION: "us-east-1" | ||
BEDROCK_AWS_SECRET_ACCESS_KEY: ${{ secrets.BEDROCK_AWS_SECRET_ACCESS_KEY }} | ||
BEDROCK_AWS_ACCESS_KEY_ID: ${{ secrets.BEDROCK_AWS_ACCESS_KEY_ID }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
# How to combine results from multiple retrievers | ||
|
||
:::info Prerequisites | ||
|
||
This guide assumes familiarity with the following concepts: | ||
|
||
- [Documents](/docs/concepts#document) | ||
- [Retrievers](/docs/concepts#retrievers) | ||
|
||
::: | ||
|
||
The [EnsembleRetriever](https://api.js.langchain.com/classes/langchain_retrievers_ensemble.EnsembleRetriever.html) supports ensembling of results from multiple retrievers. It is initialized with a list of [BaseRetriever](https://api.js.langchain.com/classes/langchain_core_retrievers.BaseRetriever.html) objects. EnsembleRetrievers rerank the results of the constituent retrievers based on the [Reciprocal Rank Fusion](https://plg.uwaterloo.ca/~gvcormac/cormacksigir09-rrf.pdf) algorithm. | ||
|
||
By leveraging the strengths of different algorithms, the `EnsembleRetriever` can achieve better performance than any single algorithm. | ||
|
||
One useful pattern is to combine a keyword matching retriever with a dense retriever (like embedding similarity), because their strengths are complementary. This can be considered a form of "hybrid search". The sparse retriever is good at finding relevant documents based on keywords, while the dense retriever is good at finding relevant documents based on semantic similarity. | ||
|
||
Below we demonstrate ensembling of a [simple custom retriever](/docs/how_to/custom_retriever/) that simply returns documents that directly contain the input query with a retriever derived from a [demo, in-memory, vector store](https://api.js.langchain.com/classes/langchain_vectorstores_memory.MemoryVectorStore.html). | ||
|
||
import CodeBlock from "@theme/CodeBlock"; | ||
import Example from "@examples/retrievers/ensemble_retriever.ts"; | ||
|
||
<CodeBlock language="typescript">{Example}</CodeBlock> | ||
|
||
## Next steps | ||
|
||
You've now learned how to combine results from multiple retrievers. | ||
Next, check out some other retrieval how-to guides, such as how to [improve results using multiple embeddings per document](/docs/how_to/multi_vector) | ||
or how to [create your own custom retriever](/docs/how_to/custom_retriever). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,7 @@ | ||
--- | ||
keywords: [bedrock] | ||
--- | ||
|
||
# AWS | ||
|
||
All functionality related to [Amazon AWS](https://aws.amazon.com/) platform | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,7 @@ | ||
--- | ||
keywords: [azure] | ||
--- | ||
|
||
import CodeBlock from "@theme/CodeBlock"; | ||
|
||
# Microsoft | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,7 @@ | ||
--- | ||
keywords: [openai] | ||
--- | ||
|
||
# OpenAI | ||
|
||
All functionality related to OpenAI | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.