Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add ChatMessage store support (including reader+writer) #47

Closed
wants to merge 9 commits into from

Conversation

vblagoje
Copy link
Member

@vblagoje vblagoje commented Jul 29, 2024

Why:

Enhances Haystack chat use cases by adding an in-memory store for chat messages along with components for message retrieval and writing.

What:

  • Added InMemoryChatMessageStore to store chat messages in memory.
  • Introduced ChatMessageStore protocol to define a standard interface for chat message storage.
  • Created ChatMessageRetriever and ChatMessageWriter components for fetching and storing chat messages, respectively.
  • Extended test coverage with unit tests for the new chat message store, retriever, and writer components.

How can it be used:

The following examples demonstrate how to use the newly added components:

  • Retrieving Chat Messages:

    from haystack_experimental.components.retrievers import ChatMessageRetriever
    from haystack_experimental.chat_message_stores.in_memory import InMemoryChatMessageStore
    
    store = InMemoryChatMessageStore()
    
    retriever = ChatMessageRetriever(message_store=store)
    retrieved_messages = retriever.run()["messages"]
  • Writing Chat Messages:

    from haystack_experimental.components.writers import ChatMessageWriter
    from haystack_experimental.chat_message_stores.in_memory import InMemoryChatMessageStore
    
    store = InMemoryChatMessageStore()
    
    writer = ChatMessageWriter(message_store=store)
    new_messages = [ChatMessage.from_user("I have a question about Python.")]
    writer.run(messages=new_messages)
  • Colab notebook (pipeline usage):

    • Here is the notebook demoing these classes in a concrete chat+RAG use case

How did you test it:

  • Implemented unit tests for InMemoryChatMessageStore to ensure proper initialization, message storage behaviors, and serialization/deserialization functionality.
  • Added unit tests for ChatMessageRetriever and ChatMessageWriter to validate interaction with the message store, including message retrieval and writing capabilities.
  • Validated all tests to confirm they pass and accurately reflect component behaviors.

Notes for the reviewer:

  • The ChatMessageRetriever of this initial version does not have filters and top_k run method parameters intentionally, as they will be added in subsequent PR.
  • Please review the implementation of the ChatMessageStore protocol hierarchy to ensure it aligns well with our design principles

@coveralls
Copy link

coveralls commented Jul 29, 2024

Pull Request Test Coverage Report for Build 10147173400

Details

  • 221 of 236 (93.64%) changed or added relevant lines in 12 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.3%) to 91.132%

Changes Missing Coverage Covered Lines Changed/Added Lines %
haystack_experimental/chat_message_stores/in_memory/chat_message_store.py 23 24 95.83%
haystack_experimental/components/retrievers/chat_message_retriever.py 30 34 88.24%
haystack_experimental/components/writers/chat_message_writer.py 31 35 88.57%
haystack_experimental/chat_message_stores/types/protocol.py 12 18 66.67%
Totals Coverage Status
Change from base Build 10145832341: 0.3%
Covered Lines: 1973
Relevant Lines: 2165

💛 - Coveralls

@vblagoje vblagoje force-pushed the chat_message_store branch from 8d82af6 to 762c938 Compare July 29, 2024 14:13
@vblagoje vblagoje marked this pull request as ready for review July 29, 2024 15:23
@vblagoje vblagoje requested a review from a team as a code owner July 29, 2024 15:23
@vblagoje vblagoje requested review from julian-risch and removed request for a team July 29, 2024 15:23
@vblagoje vblagoje changed the title draft: Add ChatMessage store support (including reader+writer) feat: Add ChatMessage store support (including reader+writer) Jul 29, 2024
@vblagoje vblagoje requested a review from shadeMe July 29, 2024 15:26
@julian-risch
Copy link
Member

@vblagoje Could you please pick a different reviewer for this as I was involved in writing this?
I would suggest to separate the store, retriever and writer into separate PRs. Makes it easier to review. ~700 lines are a lot.

@vblagoje
Copy link
Member Author

Ok deal @julian-risch

@vblagoje vblagoje closed this Jul 29, 2024
@vblagoje vblagoje deleted the chat_message_store branch August 12, 2024 12:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants