Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for filters in Vector and VectorCypher retrievers #29

Merged
merged 41 commits into from
May 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
e9786f4
Support for filters in Vector and VectorCypher retrievers
stellasia May 6, 2024
b7ef345
Ruff
stellasia May 6, 2024
0c918cd
Back to the normal dimension size in e2e tests
stellasia May 6, 2024
a6cf577
Improved docstrings + include an example
stellasia May 6, 2024
25688fe
Re-add tests for the _get_query_tail function (deleted by mistake)
stellasia May 7, 2024
77c0679
Update docstrings, move filters file, rename function
stellasia May 7, 2024
f2a830e
Support for filters in Vector and VectorCypher retrievers
stellasia May 6, 2024
fb342a5
Ruff
stellasia May 6, 2024
f3c0dab
Back to the normal dimension size in e2e tests
stellasia May 6, 2024
bea058d
Improved docstrings + include an example
stellasia May 6, 2024
90e3191
Re-add tests for the _get_query_tail function (deleted by mistake)
stellasia May 7, 2024
d52da87
Update docstrings, move filters file, rename function
stellasia May 7, 2024
962849a
Merge remote-tracking branch 'origin/pre-filters' into pre-filters
stellasia May 7, 2024
0475f38
More unit tests
stellasia May 7, 2024
24d29d4
More unit tests for filters
stellasia May 7, 2024
0a4711e
Increase test coverage for queries
stellasia May 7, 2024
0fd442f
Simplification, formatting
stellasia May 7, 2024
085ffa3
Merge branch 'main' into pre-filters
willtai May 9, 2024
87e9ded
Add try catch for create_index and rename imports of neo4j (#30)
willtai May 13, 2024
3a9de9c
Support for filters in Vector and VectorCypher retrievers
stellasia May 6, 2024
6e90039
Ruff
stellasia May 6, 2024
d3106af
Back to the normal dimension size in e2e tests
stellasia May 6, 2024
53d622f
Improved docstrings + include an example
stellasia May 6, 2024
c9bd3db
Re-add tests for the _get_query_tail function (deleted by mistake)
stellasia May 7, 2024
7851f2d
Update docstrings, move filters file, rename function
stellasia May 7, 2024
e9e3211
Support for filters in Vector and VectorCypher retrievers
stellasia May 6, 2024
f9ea4e7
Ruff
stellasia May 6, 2024
1e556a0
Back to the normal dimension size in e2e tests
stellasia May 6, 2024
cad9d4c
Improved docstrings + include an example
stellasia May 6, 2024
b1bf005
Update docstrings, move filters file, rename function
stellasia May 7, 2024
ae072fb
More unit tests
stellasia May 7, 2024
6ee5ab2
More unit tests for filters
stellasia May 7, 2024
2daa2a3
Increase test coverage for queries
stellasia May 7, 2024
a5e8980
Simplification, formatting
stellasia May 7, 2024
6d19919
Fix path after merge
stellasia May 13, 2024
361b176
Merge remote-tracking branch 'origin/pre-filters' into pre-filters
stellasia May 13, 2024
2b89aff
Use backticks only if field is non identifier
stellasia May 13, 2024
6379a84
ruff
stellasia May 13, 2024
37b75a0
Replace field name validity check by regex
stellasia May 14, 2024
235edaa
Update docstring
stellasia May 14, 2024
ff25b93
ruff
stellasia May 14, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 72 additions & 0 deletions examples/vector_search_with_filters.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
from neo4j import GraphDatabase
from neo4j_genai import VectorRetriever

import random
import string
from neo4j_genai.embedder import Embedder
from neo4j_genai.indexes import create_vector_index


URI = "neo4j://localhost:7687"
AUTH = ("neo4j", "password")

INDEX_NAME = "embedding-name"
DIMENSION = 1536

# Connect to Neo4j database
driver = GraphDatabase.driver(URI, auth=AUTH)


# Create Embedder object
class CustomEmbedder(Embedder):
def embed_query(self, text: str) -> list[float]:
return [random.random() for _ in range(DIMENSION)]


# Generate random strings
def random_str(n: int) -> str:
return "".join([random.choice(string.ascii_letters) for _ in range(n)])


embedder = CustomEmbedder()

# Creating the index
create_vector_index(
driver,
INDEX_NAME,
label="Document",
property="propertyKey",
dimensions=DIMENSION,
similarity_fn="euclidean",
)

# Initialize the retriever
retriever = VectorRetriever(driver, INDEX_NAME, embedder)

# Upsert the query
vector = [random.random() for _ in range(DIMENSION)]
insert_query = (
"MERGE (doc:Document {id: $id})"
"ON CREATE SET doc.int_property = $id, "
" doc.short_text_property = toString($id)"
"WITH doc "
"CALL db.create.setNodeVectorProperty(doc, 'propertyKey', $vector)"
"WITH doc "
"MERGE (author:Author {name: $authorName})"
"MERGE (doc)-[:AUTHORED_BY]->(author)"
"RETURN doc, author"
)
parameters = {
"id": random.randint(0, 10000),
"vector": vector,
"authorName": random_str(10),
}
driver.execute_query(insert_query, parameters)

# Perform the search
query_text = "Find me a book about Fremen"
print(
retriever.search(
query_text=query_text, top_k=1, filters={"int_property": {"$gt": 100}}
)
)
Loading
Loading