You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
Not a but per-se, but currently the behaviour of some document stores prevents pipeline's validation and serialisation.
The way Haystack pipelines validate and serialise requires all the components to be able to initialise fast, hence any operation loading or downloading data is demanded to the warm_up method of the component classes (when needed).
This should be the case for document stores too, for example:
We don't necessarily want to reach the network and connect to a database only in order to create (and validate) a pipeline
If we're building a pipeline in order to serialise it and send it over the wire, the database might not be available at "pipeline-build time"
Contrary to regular components, a warm_up strategy wouldn't be effective, because we don't have a single point in the codebase where the pipeline can call document_store.warm_up(). In fact, a document store can be passed to a writer, to a reader, or even used standalone.
For the time being, we decided to roll out a better init strategy for document stores without introducing new abstractions nor breaking changes. This means that every document store, for convention and where it makes sense, will defer the database connection until really needed.
This is the list of document stores affected that we need to change:
The content you are editing has changed. Please copy your edits and refresh the page.
Describe the bug
Not a but per-se, but currently the behaviour of some document stores prevents pipeline's validation and serialisation.
The way Haystack pipelines validate and serialise requires all the components to be able to initialise fast, hence any operation loading or downloading data is demanded to the
warm_up
method of the component classes (when needed).This should be the case for document stores too, for example:
Contrary to regular components, a
warm_up
strategy wouldn't be effective, because we don't have a single point in the codebase where the pipeline can calldocument_store.warm_up()
. In fact, a document store can be passed to a writer, to a reader, or even used standalone.For the time being, we decided to roll out a better init strategy for document stores without introducing new abstractions nor breaking changes. This means that every document store, for convention and where it makes sense, will defer the database connection until really needed.
This is the list of document stores affected that we need to change:
Tasks
AstraDocumentStore
: avoid connecting to the server in the class constructor #767ElasticsearchDocumentStore
: avoid connecting to the server in the class constructor #765MongoDBAtlasDocumentStore
: avoid connecting to the server in the class constructor #768OpenSearchDocumentStore
: avoid connecting to the server in the class constructor #749PgvectorDocumentStore
: avoid connecting to the server in the class constructor #771PineconeDocumentStore
: avoid connecting to the server in the class constructor #776QdrantDocumentStore
: avoid connecting to the server in the class constructor #747WeaviateDocumentStore
: avoid connecting to the server in the class constructor #762The text was updated successfully, but these errors were encountered: