End-to-end RAG tutorial using source file, PyAirbyte and Pinecone (using Langchain)

[aaronsteers]

Summary

The goal is to show users how to use PyAirbyte to load data from file source, store the data in Pinecone vector store and perform a short RAG demo (using OpenAI/LangChain) on the loaded data in Pinecone.

Deliverable

Python notebook illustrating end to end RAG using PyAirbyte, file as source, and Pinecone as destination. Please add a "What / Why / How" blurb at the top of the notebook to explain what the code is doing.

Resources to Assist

• PyAirbyte tutorials: https://github.com/airbytehq/quickstarts/tree/main/pyairbyte_notebooks
• PyAirbyte Documentaton: https://docs.airbyte.com/using-airbyte/pyairbyte/getting-started
• PyAirbyte API Reference: https://docs.airbyte.com/using-airbyte/pyairbyte/getting-started#api-reference
• File source documentation: https://docs.airbyte.com/integrations/sources/file
• LangChain Pinecone support: https://python.langchain.com/v0.1/docs/integrations/vectorstores/pinecone/

[arice77]

Hey @aaronsteers,I would love to create a tutorial around this topic 👍

[Harmaton]

I have done this before, I would like to write about it :)

[siddhant3030]

I'd like to work on this with real use case for a client

[marcosmarxm]

@siddhant3030 It is yours. @Harmaton I assigned other to you.

[marcosmarxm]

@siddhant3030 do you need any help herE?

[siddhant3030]

hey. No. I'm working on it. Thanks

[marcosmarxm]

@siddhant3030 can you give us an update about this task?

[pangeran-bottor]

Hi, I would love to take this if it becomes available

[pangeran-bottor]

@siddhant3030 It will benefit all of us if you reply and update or share the problem here so others and the Airbyte team can also check for transparency. From the outside, this seems inactive for three weeks, so others probably can pick up this issue

[gupta-arpan]

@siddhant3030 If you have been unable to address the issue, could you please transfer it to me?

[siddhant3030]

@gupta-arpan please go ahead. I can't reassign it

[bala-ceg]

I would like to take this issue, just in case if it is available

[pangeran-bottor]

@marcosmarxm, per your last information, can I take the issue if this is available?

[gupta-arpan]

@bindipankhudi, As @siddhant3030 gave me a gohead signal. I have made a PR airbytehq/quickstarts#124 for this issue please review it, Thanks.

[marcosmarxm]

@gupta-arpan the user @pangeran-bottor requested this issue before. It is his right to work on this.

[pangeran-bottor]

Hi guys @bindipankhudi @aaronsteers @marcosmarxm
Kindly need your help to review this airbytehq/quickstarts#127