Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add NVIDIARerank connector, a BaseDocumentCompressor, with support for NVIDIA API Catalog #19

Merged
merged 26 commits into from
Apr 23, 2024

Conversation

mattf
Copy link
Collaborator

@mattf mattf commented Apr 12, 2024

introduce NVIDIARerank, a BaseDocumentCompressor, supporting NVIDIA API Catalog, e.g.

from langchain_nvidia_ai_endpoints import NVIDIARerank
from langchain_core.documents import Document

query = "which way should i go?"
passages = [
	"two roads diverged in a yellow wood, and sorry i could not travel both and be one traveler, long i stood and looked down one as far as i could to where it bent in the undergrowth;",
	"then took the other, as just as fair, and having perhaps the better claim because it was grassy and wanted wear, though as for that the passing there had worn them really about the same,",
	"and both that morning equally lay in leaves no step had trodden black. oh, i marked the first for another day! yet knowing how way leads on to way i doubted if i should ever come back.",
	"i shall be telling this with a sigh somewhere ages and ages hense: two roads diverged in a wood, and i, i took the one less traveled by, and that has made all the difference."
]
documents = [Document(page_content=passage) for passage in passages]

# available models
models = NVIDIARerank.get_available_models()

# API Catalog with default model
ranker = NVIDIARerank()

# API Catalog with selected model
ranker = NVIDIARerank(model=models[0].id)

# control number of output documents
ranker.top_n = 2

# perform reranking compression
ranked_documents = ranker.compress_documents(documents=documents, query=query)

# examine ranking relevance score
assert ranked_documents[0].metadata["relevance_score"] > ranked_documents[1].metadata["relevance_score"]

@mattf mattf self-assigned this Apr 12, 2024
@mattf mattf force-pushed the mattf/add-reranking-api-catalog branch from 5586cde to 6662c10 Compare April 17, 2024 15:30
Zenodia and others added 25 commits April 20, 2024 07:10
current:
 {"model": "...",
  "query": {"text": "hello world?"},
  "passages": [
   {"text": "passage one"},
   {"text": "passage two"},
   {"text": "passage three"}
  ],
  "logits": True
 }
-> [{"index": 1, "score": 1.0, "logit":  1.234},
    {"index": 2, "score": 0.5, "logit":  0.910},
    {"index": 0, "score": 0.0, "logit": -5.678}]

change:
 - remove logit option
 - return logits instead of normalized scores

new:
 {"model": "...",
  "query": {"text": "hello world?"},
  "passages": [
   {"text": "passage one"},
   {"text": "passage two"},
   {"text": "passage three"}
  ]
 }
-> [{"index": 1, "logit":  1.234},
    {"index": 2, "logit":  0.910},
    {"index": 0, "logit": -5.678}]

user impact:
 - user must perform their own normalization, if desired
 - no impact: results are still returned in sorted order
 - no impact: users can still produce a total order across batches
doing so would require selection of a bogus logit, which will break
the ability of a user to call compress_documents with disjoint sets of
documents and reconstruct an ordering. at least in the degenerate case
of 1 document per batch.
@mattf mattf force-pushed the mattf/add-reranking-api-catalog branch from 6662c10 to d95fd5a Compare April 20, 2024 12:33
@chantal-rose
Copy link

Looks good @mattf

@mattf mattf merged commit edb8b9b into main Apr 23, 2024
12 checks passed
@mattf mattf deleted the mattf/add-reranking-api-catalog branch April 23, 2024 19:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants