Skip to content

Commit

Permalink
improve integration docs
Browse files Browse the repository at this point in the history
  • Loading branch information
hwchase17 committed Nov 29, 2024
1 parent c2f1d02 commit ce6e4bb
Show file tree
Hide file tree
Showing 2 changed files with 157 additions and 18 deletions.
71 changes: 53 additions & 18 deletions docs/docs/contributing/how_to/integrations/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,16 @@ pagination_next: contributing/how_to/integrations/package

# Contribute Integrations

LangChain integrations are packages that provide access to language models, vector stores, and other components that can be used in LangChain.
Integrations are a core component of LangChain.
LangChain provides standard interfaces for several different components (language models, vector stores, etc) that are crucial when building LLM applications.

This guide will walk you through how to contribute new integrations to LangChain, by
publishing an integration package to PyPi, and adding documentation for it
to the LangChain Monorepo.

These instructions will evolve over the next few months as we improve our integration
processes.
## Why contribute an integration to LangChain?

- **Discoverability:** LangChain is the most used framework for building LLM applications, with over 20 million monthly downloads. LangChain integrations are discoverable by a large community of GenAI builders.
- **Interoptability:** LangChain components expose a standard interface, allowing developers to easily swap them for each other. If you implement a LangChain integration, any developer using a different component will easily be able to swap yours in.
- **Best Practices:** Through their standard interface, LangChain components encourage and facilitate best practices (streaming, async, etc)


## Components to Integrate

Expand All @@ -22,8 +24,7 @@ supported in LangChain

:::

While any component can be integrated into LangChain, at this time we are only accepting
new integrations in the docs of the following kinds:
While any component can be integrated into LangChain, there are specific types of integrations we encourage more:

<table>
<tr>
Expand Down Expand Up @@ -60,18 +61,52 @@ new integrations in the docs of the following kinds:

## How to contribute an integration

The only step necessary to "be" a LangChain integration is to add documentation
that will render on this site (https://python.langchain.com/).
In order to contribute an integration, you should follow these steps:

1. Confirm that your integration is in the [list of components](#components-to-integrate) we are currently encouraging.
2. Implement your integration
3. Implement the standard tests ([see below](#standard-tests)) for your integration and successfully run them.
4. Publish your integration in a Python package to PyPi.
5. [Optional] Open and merge a PR to add documentation for your integration to the official LangChain docs.
6. [Optional] Engage with the LangChain team for joint co-marketing ([see below](#co-marketing)).


## Standard Tests

Testing is a critical part of the development process that ensures your code works as expected and meets the desired quality standards.
In the LangChain ecosystem, we have 2 main types of tests: **unit tests** and **integration tests**.
These standard tests help maintain compatibility between different components and ensure reliability.

**Unit Tests**: Unit tests are designed to validate the smallest parts of your code—individual functions or methods—ensuring they work as expected in isolation. They do not rely on external systems or integrations.

**Integration Tests**: Integration tests validate that multiple components or systems work together as expected. For tools or integrations relying on external services, these tests often ensure end-to-end functionality.

Each type of integration has its own set of standard tests. Please see the correct guide for setting up standard tests for your integration:

- [Chat Models]
- [Tools]
- [Toolkits]
- [Retrievers]
- [Document Loaders]
- [Retrievers]
- [Document Loaders]
- [Vector Stores]
- [Embedding Models]

## Co-Marketing

As a prerequisite to adding your integration to our documentation, you must:
With over 20 million monthly downloads, LangChain has a large audience of developers building LLM applications.
Besides just adding integrations, we also like to show them examples of cool tools or APIs they can use.

1. Confirm that your integration is in the [list of components](#components-to-integrate) we are currently accepting.
2. [Implement your package](./package.mdx) and publish it to a public github repository.
3. [Implement the standard tests](./standard_tests) for your integration and successfully run them.
4. [Publish your integration](./publish.mdx) by publishing the package to PyPi and add docs in the `docs/docs/integrations` directory of the LangChain monorepo.
While traditionally called "co-marketing", we like to thing of this more as "co-education".
For that reason, while we are happy to highlight your integration through our social media channels, we prefer to highlight examples that also serve some educational purpose.
Our main social media channels are Twitter and LinkedIn.

Once you have completed these steps, you can submit a PR to the LangChain monorepo to add your integration to the documentation.
Here are some heuristics for types of content we are excited to promote:

## Further Reading
- **Integration announcement:** If you announce the integration with a link to the LangChain documentation page, we are happy to re-tweet/re-share on Twitter/LinkedIn.
- **Educational content:** We highlight good educational content on the weekends - if you write a good blog or make a good YouTube video, we are happy to share there! Note that we prefer content that is NOT framed as "here's how to use integration XYZ", but rather "here's how to do ABC", as we find that is more educational and helpful for developers.
- **End-to-end applications:** End-to-end applications are great resources for developers looking to build. We prefer to highlight applications that are more complex/agentic in nature, and that use [LangGraph](https://github.com/langchain-ai/langgraph) as the orchestration framework. We get particularly excited about anything involving long-term memory, human-in-the-loop interaction patterns, or multi-agent architectures.
- **Research:** We love highlighting novel research! Whether it is research built on top of LangChain or that integrates with it.

To get started, let's learn [how to bootstrap a new integration package](./package.mdx) for LangChain.
// TODO: set up some form to gather these requests.
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
# Standard Tests (Chat Model)

This guide walks through how to create standard tests for a custom Chat Model that you have developed.

## Setup

First we need to install certain dependencies. These include:

- `pytest`: For running tests
- `pytest-socket`: For running unit tests
- `pytest-asyncio`: For testing async functionality
- `langchain-tests`: For importing standard tests
- `langchain-core`: This should already be installed, but is needed to define our integration.

```shell
pip install -U langchain-core pytest pytest-socket pytest-asyncio langchain-tests
```

## Add and configure standard tests
There are 2 namespaces in the langchain-tests package:

[unit tests](../../../concepts/testing.mdx#unit-tests) (langchain_tests.unit_tests): designed to be used to test the component in isolation and without access to external services
[integration tests](../../../concepts/testing.mdx#integration-tests) (langchain_tests.integration_tests): designed to be used to test the component with access to external services (in particular, the external service that the component is designed to interact with).

Both types of tests are implemented as [pytest class-based test suites](https://docs.pytest.org/en/7.1.x/getting-started.html#group-multiple-tests-in-a-class).

By subclassing the base classes for each type of standard test (see below), you get all of the standard tests for that type, and you can override the properties that the test suite uses to configure the tests.

Here's how you would configure the standard unit tests for the custom chat model:

```python
# title="tests/unit_tests/test_chat_models.py"
from typing import Type

from my_package.chat_models import MyChatModel
from langchain_tests.unit_tests import ChatModelUnitTests


class TestChatParrotLinkUnit(ChatModelUnitTests):
@property
def chat_model_class(self) -> Type[MyChatModel]:
return MyChatModel

@property
def chat_model_params(self) -> dict:
# These should be parameters used to initialize your integration for testing
return {
"model": "bird-brain-001",
"temperature": 0,
"parrot_buffer_length": 50,
}
```

```python
# title="tests/integration_tests/test_chat_models.py"
from typing import Type

from my_package.chat_models import MyChatModel
from langchain_tests.integration_tests import ChatModelIntegrationTests


class TestChatParrotLinkIntegration(ChatModelIntegrationTests):
@property
def chat_model_class(self) -> Type[MyChatModel]:
return MyChatModel

@property
def chat_model_params(self) -> dict:
# These should be parameters used to initialize your integration for testing
return {
"model": "bird-brain-001",
"temperature": 0,
"parrot_buffer_length": 50,
}
```

## Run standard tests

After setting tests up, you would run these with the following commands from your project root:

```shell
# run unit tests without network access
pytest --disable-socket --allow-unix-socket --asyncio-mode=auto tests/unit_tests

# run integration tests
pytest --asyncio-mode=auto tests/integration_tests
```

## Test suite information and troubleshooting

What tests are run to test this integration?

If a test fails, what does that mean?

You can find information on the tests run for this integration in the [Standard Tests API Reference](https://python.langchain.com/api_reference/standard_tests/index.html).

// TODO: link to exact page for this integration test suite information

## Skipping tests

Sometimes you do not need your integration to pass all tests, as you only rely on specific subsets of functionality.
If this is the case, you can turn off specific tests by...

// TODO: add instructions for how to turn off tests

0 comments on commit ce6e4bb

Please sign in to comment.