Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

INTPYTHON-377 langgraph-checkpoint-mongodb #7

Merged
Show file tree
Hide file tree
Changes from 60 commits
Commits
Show all changes
76 commits
Select commit Hold shift + click to select a range
1b1130c
Added new package langgraph-checkpoint-mongodb
caseyclements Oct 31, 2024
fefa0e1
Copies from langgraph everything to run test_pregel
caseyclements Nov 1, 2024
4aca7e5
Add snapshot files (run tests with --snapshot-update)
caseyclements Nov 1, 2024
8373eda
Test cleanup. There are 5 of 123 failing
caseyclements Nov 1, 2024
b3277a7
Marked points of failure and added a couple notes
caseyclements Nov 2, 2024
3ded014
Fixed test_pregel::test_root_graph. Related to deterministic_uuids fe…
caseyclements Nov 3, 2024
0d38a26
Replace ALL_CHECKPOINTERS_SYNC in test_root_graph
caseyclements Nov 3, 2024
c2dc0ce
Fixed tests related to subgraphs and checkpoint_ns. One left that may…
caseyclements Nov 3, 2024
9a815f2
linting
caseyclements Nov 4, 2024
94bab6f
Added pytest-repeat
caseyclements Nov 4, 2024
8794bd9
Turned async tests backed on
caseyclements Nov 4, 2024
a9a5a4b
Applied WRITES_IDX_MAP.get and checkpoint_ns fixes to AsyncMongoDBSaver
caseyclements Nov 4, 2024
d328a68
Updated poetry.lock after adding pytest-repeat
caseyclements Nov 4, 2024
64fa394
Added snapshots for test_pregel_async
caseyclements Nov 4, 2024
8daded4
Updated setup of _checkpointer_mongodb_aio
caseyclements Nov 4, 2024
fe6cd14
Added missing pending_writes in alist
caseyclements Nov 4, 2024
c0481e2
Addd TODO to test as it might be another red herring
caseyclements Nov 4, 2024
730cae6
Following advice from Actions, updated libs in check_diff
caseyclements Nov 4, 2024
bdf4b2b
Adjusting pyproject.toml to see if we can use existing Github Actions
caseyclements Nov 4, 2024
070ac48
Added empty test_integration group in pyproject for CI
caseyclements Nov 4, 2024
313a5c2
Update content-hash of poetry.lock
caseyclements Nov 4, 2024
2b1de87
formatting
caseyclements Nov 4, 2024
960d773
Had forgotten to await checkpointer.clxn_chkpnt.delete_many
caseyclements Nov 4, 2024
111b4c9
Add blocking methods to AsyncMongoDBSaver
caseyclements Nov 4, 2024
edd1969
Added compatability methods async and aiter for py3.9
caseyclements Nov 5, 2024
0658d00
formatting
caseyclements Nov 5, 2024
f8f602f
Updated logic in check_diff to avoid linting langgraph-checkpoint-mon…
caseyclements Nov 5, 2024
5ba00f3
Typo
caseyclements Nov 5, 2024
2940c3c
Updated logic in check_diff to avoid linting langgraph-checkpoint-mon…
caseyclements Nov 5, 2024
eed64e9
Temporarily turn off make lint_test.
caseyclements Nov 5, 2024
594642e
Set mongodb client hosts for CI
caseyclements Nov 6, 2024
7285839
Added placeholder used for compiling integration tests without runnin…
caseyclements Nov 6, 2024
d6b85da
Added compile and requires to available pytest markers
caseyclements Nov 6, 2024
9197152
Updated db_name to langgraph_checkpoints_db for clarity
caseyclements Nov 6, 2024
706fbd9
Removed 3.9 from python version matrix to see id anext workaround was…
caseyclements Nov 6, 2024
f9b3142
Merge branch 'main' into INTPYTHON-377-LANGGRAPH-CHECKPOINTER
caseyclements Nov 7, 2024
a307500
Added supercharge github action to mongodb demon instead of a remote …
caseyclements Nov 11, 2024
b4c62d0
Revert "Removed 3.9 from python version matrix to see id anext workar…
caseyclements Nov 11, 2024
71b90a9
Found hardcoded string
caseyclements Nov 12, 2024
38e7ff8
Allow override of uri and db_name via environ for testing
caseyclements Nov 12, 2024
d940864
pytest skipped: test_prebuilt_tool_chat as a fix should be in another PR
caseyclements Nov 12, 2024
1766750
Fixes to AsyncMongoDBSaver docstrings
caseyclements Nov 12, 2024
87069e8
Added a couple lines to the README. It will be properly expanded in a…
caseyclements Nov 12, 2024
e647dba
Revert "pytest skipped: test_prebuilt_tool_chat as a fix should be in…
caseyclements Nov 13, 2024
5219d3e
Addressed comments by @vbarda
caseyclements Nov 14, 2024
5ebc64c
relax the requirement for having a thread ID in the config for alist
caseyclements Nov 14, 2024
0871d6e
Reapply "pytest skipped: test_prebuilt_tool_chat so that we start wit…
caseyclements Nov 14, 2024
5545743
formatting
caseyclements Nov 14, 2024
5ef0954
Removed tests previously copied from langraph/tests
caseyclements Nov 20, 2024
aac8872
Made fix in put_writes
caseyclements Nov 20, 2024
e984a53
Copied latest langraph/test (test_pregel) into integration_tests
caseyclements Nov 20, 2024
7bfe7b5
Updated pyproject.toml
caseyclements Nov 20, 2024
4ba694b
Updated langgraph and langraph-checkpoint to main branch of langchain…
caseyclements Nov 20, 2024
87906b7
Actually updated langgraph and langraph-checkpoint to main branch of …
caseyclements Nov 20, 2024
5651c9f
Updated poetry.lock
caseyclements Nov 20, 2024
d504de4
Adjusted integration_tests to use relative imports
caseyclements Nov 20, 2024
2a0d986
Formatted
caseyclements Nov 20, 2024
6528cfd
Linting
caseyclements Nov 20, 2024
1f728d8
Temporary change to poetry install with groups in tests
caseyclements Nov 20, 2024
2fbf0aa
Temporarily removed libs/mongodb from CI to focus on langgraph-checkp…
caseyclements Nov 20, 2024
cd28d9a
Revert "Temporarily removed libs/mongodb from CI to focus on langgrap…
caseyclements Nov 21, 2024
f872518
Revert "Temporary change to poetry install with groups in tests"
caseyclements Nov 21, 2024
754b9c8
Removed compile-integration-tests
caseyclements Nov 21, 2024
68c924f
Renamed langchain_mongodb github actions
caseyclements Nov 21, 2024
8c143cd
Merge branch 'main' into INTPYTHON-377-LANGGRAPH-CHECKPOINTER
caseyclements Nov 21, 2024
17890aa
Updated check_diff script to take a specific lib, and added libs/mong…
caseyclements Nov 21, 2024
037ce12
Unified style of pyproject with dev,test,lint,test_integration
caseyclements Nov 21, 2024
069738e
Simplified project dependency groups and unified tests
caseyclements Nov 22, 2024
e953e26
Typos
caseyclements Nov 22, 2024
c0ade10
Typo in LIB_DIRS
caseyclements Nov 22, 2024
ab1c3c2
Remove irrelevant tests
caseyclements Nov 22, 2024
1f9247c
Removed code duplication, using utils method prepare_metadata
caseyclements Nov 22, 2024
4716887
Linting
caseyclements Nov 22, 2024
69313b7
For linting, revert to using separate loads/dumps metadata methods, b…
caseyclements Nov 22, 2024
6ee7287
Unused import
caseyclements Nov 22, 2024
d8d7d5c
nit fix
caseyclements Nov 22, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions .github/scripts/check_diff.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
import sys
from typing import Dict

LIB_DIRS = ["libs/mongodb"]
LIB_DIRS = ["libs/langgraph-checkpoint-mongodb"]

if __name__ == "__main__":
files = sys.argv[1:]
Expand Down Expand Up @@ -33,14 +33,16 @@
for dir_ in LIB_DIRS:
if file.startswith(dir_):
dirs_to_run["test"].add(dir_)
if "langgraph-checkpoint-mongodb/tests" not in file:
dirs_to_run["lint"].add(dir_)
elif file.startswith("libs/"):
raise ValueError(
f"Unknown lib: {file}. check_diff.py likely needs "
"an update for this new library!"
)

outputs = {
"dirs-to-lint": list(dirs_to_run["lint"] | dirs_to_run["test"]),
"dirs-to-lint": list(dirs_to_run["lint"]),
"dirs-to-test": list(dirs_to_run["test"]),
}
for key, value in outputs.items():
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/_compile_integration_test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ jobs:

- name: Install integration dependencies
shell: bash
run: poetry install --with=test_integration,test
run: poetry install --with dev

- name: Check integration tests compile
shell: bash
Expand Down
12 changes: 6 additions & 6 deletions .github/workflows/_lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ jobs:
# It doesn't matter how you change it, any change will cause a cache-bust.
working-directory: ${{ inputs.working-directory }}
run: |
poetry install --with lint,typing
poetry install --with dev

- name: Get .mypy_cache to speed up mypy
uses: actions/cache@v4
Expand All @@ -85,7 +85,7 @@ jobs:
- name: Install unit+integration test dependencies
working-directory: ${{ inputs.working-directory }}
run: |
poetry install --with test,test_integration
poetry install --with dev

- name: Get .mypy_cache_test to speed up mypy
uses: actions/cache@v4
Expand All @@ -96,7 +96,7 @@ jobs:
${{ env.WORKDIR }}/.mypy_cache_test
key: mypy-test-${{ runner.os }}-${{ runner.arch }}-py${{ matrix.python-version }}-${{ inputs.working-directory }}-${{ hashFiles(format('{0}/poetry.lock', inputs.working-directory)) }}

- name: Analysing the code with our lint
working-directory: ${{ inputs.working-directory }}
run: |
make lint_tests
# - name: Analysing the code with our lint
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be re-enabled?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I made that change, maybe. But it picks up, oh, 1909 errors that aren't related to my changes.

Found 1909 errors in 7 files (checked 13 source files)
make: *** [Makefile:28: lint_tests] Error 1

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I opened https://jira.mongodb.org/browse/INTPYTHON-412 to follow up with restoring the lint.

# working-directory: ${{ inputs.working-directory }}
# run: |
# make lint_tests
2 changes: 1 addition & 1 deletion .github/workflows/_release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,7 @@ jobs:
poetry run python -c "import $IMPORT_NAME; print(dir($IMPORT_NAME))"

- name: Import test dependencies
run: poetry install --with test,test_integration
run: poetry install --with dev
working-directory: ${{ inputs.working-directory }}

# Overwrite the local version of the package with the test PyPI version.
Expand Down
5 changes: 4 additions & 1 deletion .github/workflows/_test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,10 @@ jobs:

- name: Install dependencies
shell: bash
run: poetry install --with test
run: poetry install --with dev

- name: Start MongoDB
uses: supercharge/[email protected]

- name: Run core tests
shell: bash
Expand Down
8 changes: 6 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# 🦜️🔗 LangChain MongoDB

This repository contains 1 package with MongoDB integrations with LangChain:
This is a Monorepo containing partner packages of MongoDB and LangChainAI.
It includes integrations between MongoDB, Atlas, LangChain, and LangGraph.

- [langchain-mongodb](https://pypi.org/project/langchain-mongodb/)
It contains the following packages.

- `langchain-mongodb` ([PyPI](https://pypi.org/project/langchain-mongodb/))
- `langgraph-checkpoint-mongodb`
vbarda marked this conversation as resolved.
Show resolved Hide resolved
42 changes: 42 additions & 0 deletions libs/langgraph-checkpoint-mongodb/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
.PHONY: test lint format help

######################
# TESTING AND COVERAGE
######################

test tests:
poetry run pytest tests

######################
# LINTING AND FORMATTING
######################

# Define a variable for Python and notebook files.
PYTHON_FILES=.
MYPY_CACHE=.mypy_cache
lint format: PYTHON_FILES=.
lint_diff format_diff: PYTHON_FILES=$(shell git diff --name-only --relative --diff-filter=d main . | grep -E '\.py$$|\.ipynb$$')
lint_package: PYTHON_FILES=langgraph
lint_tests: PYTHON_FILES=tests
lint_tests: MYPY_CACHE=.mypy_cache_test

lint lint_diff lint_package lint_tests:
poetry run ruff check .
[ "$(PYTHON_FILES)" = "" ] || poetry run ruff format $(PYTHON_FILES) --diff
[ "$(PYTHON_FILES)" = "" ] || poetry run ruff check --select I $(PYTHON_FILES)
[ "$(PYTHON_FILES)" = "" ] || mkdir -p $(MYPY_CACHE)
[ "$(PYTHON_FILES)" = "" ] || poetry run mypy $(PYTHON_FILES) --cache-dir $(MYPY_CACHE)

format format_diff:
poetry run ruff format $(PYTHON_FILES)
poetry run ruff check --select I --fix $(PYTHON_FILES)

######################
# HELP
######################

help:
@echo '----'
@echo 'format - run code formatters'
@echo 'lint - run linters'
@echo 'test - run unit tests'
95 changes: 95 additions & 0 deletions libs/langgraph-checkpoint-mongodb/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
# LangGraph Checkpoint MongoDB

Implementation of LangGraph CheckpointSaver that uses MongoDB.

## Usage

```python
from langgraph.checkpoint.mongodb import MongoDBSaver

write_config = {"configurable": {"thread_id": "1", "checkpoint_ns": ""}}
read_config = {"configurable": {"thread_id": "1"}}

MONGODB_URI = "mongodb://localhost:27017"
DB_NAME = "checkpoint_example"

with MongoDBSaver.from_conn_string(MONGODB_URI, DB_NAME) as checkpointer:
# call .setup() the first time you're using the checkpointer
checkpointer.setup()
checkpoint = {
"v": 1,
"ts": "2024-07-31T20:14:19.804150+00:00",
"id": "1ef4f797-8335-6428-8001-8a1503f9b875",
"channel_values": {
"my_key": "meow",
"node": "node"
},
"channel_versions": {
"__start__": 2,
"my_key": 3,
"start:node": 3,
"node": 3
},
"versions_seen": {
"__input__": {},
"__start__": {
"__start__": 1
},
"node": {
"start:node": 2
}
},
"pending_sends": [],
}

# store checkpoint
checkpointer.put(write_config, checkpoint, {}, {})

# load checkpoint
checkpointer.get(read_config)

# list checkpoints
list(checkpointer.list(read_config))
```

### Async

```python
from langgraph.checkpoint.pymongo import AsyncMongoDBSaver

async with AsyncMongoDBSaver.from_conn_string(MONGODB_URI) as checkpointer:
checkpoint = {
"v": 1,
"ts": "2024-07-31T20:14:19.804150+00:00",
"id": "1ef4f797-8335-6428-8001-8a1503f9b875",
"channel_values": {
"my_key": "meow",
"node": "node"
},
"channel_versions": {
"__start__": 2,
"my_key": 3,
"start:node": 3,
"node": 3
},
"versions_seen": {
"__input__": {},
"__start__": {
"__start__": 1
},
"node": {
"start:node": 2
}
},
"pending_sends": [],
}

# store checkpoint
await checkpointer.aput(write_config, checkpoint, {}, {})

# load checkpoint
await checkpointer.aget(read_config)

# list checkpoints
[c async for c in checkpointer.alist(read_config)]
```
Loading
Loading