langchain: enable weaviate v4 and its usage as self-query retriever

[xintoteai]

• Weaviate v4 could not be used with SelfQueryRetriever. I added this capability in libs/langchain/retrievers/self_query/base.py, while adding a test_base.py script under libs/langchain/integration_tests/
• I created docker-compose/weaviate.yml for the Weaviate V4 usage inside libs/langchain/tests/integration_tests
• Issues with test_weaviate.py inside libs/commmunity/integration_tests: embedding_openai fixture was missing; certainty is almost 1 but can be off by a tiny amount. I updated them.
• Added docs/docs/integrations/ example
• (Almost) successfully ran make format, make lint and make test from the root of both libs/langchain and libs/community. There were still several test errors from the parts of the code base that I didn't touch.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
langchain	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Dec 9, 2024 8:55pm

[vercel]

Deployment failed with the following error:

The provided GitHub repository does not contain the requested branch or commit reference. Please ensure the repository is not empty.

[efriis]

hey there! Could you work with @hsm207 to put this where the weaviate team would like this (most likely in the https://github.com/langchain-ai/langchain-weaviate repo)

Once it's in, feel free to reopen with just the docs file change!

[xintoteai]

I understand your point. However, what I've found is what is missing is not langchain-weaviate. Its stuff works. It is inside the langchain repo, where a critical import syntax is missing.

I screenshot this piece of code changes inside: libs/langchain/langchain/retrievers/self_query/base.py. Without such a change in important, the new weaviate v4 won't work for self query retriever - my main contribution to this repo. The rest of the PR is mainly to provide a test case inside the integration_tests/ directory.

[efriis]

Got it. This PR unfortunately wasn't mergable in its current state for a variety of reasons. feel free to put up a pr more like https://github.com/langchain-ai/langchain/pull/28292/files

In particular, please avoid adding any dependencies. You're welcome to skip the integration tests if @hsm207 confirms the implementation it's importing should work with the self query retriever here.

[xintoteai]

I got your point. My takeaway:

• I can supply a new but much smaller PR similar to your example.
• I have or had no way to test the import it when I was trying to follow the langchain instructions. That is why I wrote those tests. Perhaps, I can constrain myself to submit only 1 test
• Understand your point of not adding dependencies.