Skip to content

Commit

Permalink
Merge pull request #148 from deepset-ai/metadata-enrichment
Browse files Browse the repository at this point in the history
adding the tables to all the cookbooks
  • Loading branch information
TuanaCelik authored Nov 1, 2024
2 parents 23e6ade + 93b39c6 commit fd6f603
Show file tree
Hide file tree
Showing 4 changed files with 41 additions and 2 deletions.
10 changes: 9 additions & 1 deletion notebooks/extracting_metadata_filters_from_a_user_query.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,15 @@
"\n",
"*Notebook by [David Batista](https://www.linkedin.com/in/dsbatista)*\n",
"\n",
"> 📚 This cookbook has an accompanying article with a complete walkthrough [\"Advanced Retrieval: Extract Metadata from Queries to Improve Retrieval\"](https://haystack.deepset.ai/blog/extracting-metadata-filter)\n",
"> This is part one of the **Advanced Use Cases** series:\n",
">\n",
"> 1️⃣ **Extract Metadata from Queries to Improve Retrieval & the [full article](/blog/extracting-metadata-filter)**\n",
">\n",
"> 2️⃣ Query Expansion [cookbook](/cookbook/query-expansion) & [full article](/blog/query-expansion)\n",
">\n",
"> 3️⃣ Query Decomposition [cookbook](/cookbook/query_decomposition) & the [full article](/blog/query-decomposition)\n",
">\n",
"> 4️⃣ [Automated Metadata Enrichment](/cookbook/metadata_enrichment)\n",
"\n",
"In this notebook, we'll discuss how to implement a custom component, `QueryMetadataExtractor`, that extracts entities from the query and formulates the corresponding metadata filter.\n",
"\n",
Expand Down
10 changes: 10 additions & 0 deletions notebooks/metadata_enrichment.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,16 @@
"\n",
"by Tuana Celik ([LI](https://www.linkedin.com/in/tuanacelik/), [Twitter](https://x.com/tuanacelik))\n",
"\n",
"> This is part one of the **Advanced Use Cases** series:\n",
">\n",
"> 1️⃣ Extract Metadata from Queries to Improve Retrieval [cookbook](/cookbook/extracting_metadata_filters_from_a_user_query) & [full article](/blog/extracting-metadata-filter)\n",
">\n",
"> 2️⃣ Query Expansion [cookbook](/cookbook/query-expansion) & [full article](/blog/query-expansion)\n",
">\n",
"> 3️⃣ Query Decomposition [cookbook](/cookbook/query_decomposition) & the [full article](/blog/query-decomposition)\n",
">\n",
"> 4️⃣ **Automated Metadata Enrichment**\n",
"\n",
"In this example, you'll see how you can make use of structured outputs which is an option for some LLMs, and a custom Haystack component, to automate the enrichment of metadata from documents.\n",
"\n",
"You will see how you can define your own metadata fields as a Pydantic Model, as well as the data types each field should have. Finally, you will get a custom `MetadataEnricher` to extract the required fields and add them to the document meta information.\n",
Expand Down
13 changes: 12 additions & 1 deletion notebooks/query-expansion.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,16 @@
"# Advanced RAG: Query Expansion\n",
"_by Tuana Celik ([LI](https://www.linkedin.com/in/tuanacelik/), [Twitter/X](https://x.com/tuanacelik))_\n",
"\n",
"> This is part one of the **Advanced Use Cases** series:\n",
">\n",
"> 1️⃣ Extract Metadata from Queries to Improve Retrieval [cookbook](/cookbook/extracting_metadata_filters_from_a_user_query) & [full article](/blog/extracting-metadata-filter)\n",
">\n",
"> 2️⃣ **Query Expansion & the [full article](/blog/query-expansion)**\n",
">\n",
"> 3️⃣ Query Decomposition [cookbook](/cookbook/query_decomposition) & [full article](/blog/query-decomposition)\n",
">\n",
"> 4️⃣ Automated Metadata Enrichment](/cookbook/metadata_enrichment)\n",
"\n",
"In this cookbook, you'll learn how to implement query expansion for RAG. Query expansion consists of asking an LLM to produce a number of similar queries to a user query. We are then able to use each of these queries in the retrieval process, increasing the number and relevance of retrieved documents.\n",
"\n",
"📚 [Read the full article](https://haystack.deepset.ai/blog/query-expansion)"
Expand Down Expand Up @@ -649,7 +659,8 @@
"provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"display_name": "deeplearning",
"language": "python",
"name": "python3"
},
"language_info": {
Expand Down
10 changes: 10 additions & 0 deletions notebooks/query_decomposition.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,16 @@
"\n",
"by Tuana Celik ([LI](https://www.linkedin.com/in/tuanacelik/), [Twitter](https://x.com/tuanacelik))\n",
"\n",
"> This is part one of the **Advanced Use Cases** series:\n",
">\n",
"> 1️⃣ Extract Metadata from Queries to Improve Retrieval [cookbook](/cookbook/extracting_metadata_filters_from_a_user_query)& [full article](/blog/extracting-metadata-filter)\n",
">\n",
"> 2️⃣ Query Expansion [cookbook](/cookbook/query-expansion) & [full article](/blog/query-expansion)\n",
">\n",
"> 3️⃣ **Query Decomposition** & the [full article](/blog/query-decomposition)\n",
">\n",
"> 4️⃣ [Automated Metadata Enrichment](/cookbook/metadata_enrichment)\n",
"\n",
"Query decomposition is a technique we can use to decompose complex queries into simpler steps, answering each sub-question, and getting an LLM to reason about the final answer based on the answers to the sub-questions.\n",
"\n",
"In this recipe, we're using the structured output functionality (currently in beta) by OpenAI to construct `Questions` which lists the sub-questions based on the original question, as well as keeping track of the intermediate answers to each question.\n",
Expand Down

0 comments on commit fd6f603

Please sign in to comment.