Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] sparse: use staged cursor and upper_bound for WAND #970

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

sparknack
Copy link
Contributor

@sparknack sparknack commented Dec 3, 2024

Use staged cursor and upper_bound to avoid recalculating the score from cursor 0 every time in the loop. And modify some variable names to be more readable.

Simple benchmark result(time used to search all queries once on the full dataset) with drop ratio build 0.32 and drop ratio search 0.6:

BM25 Before After Improvement %
bigann SPLADE vectors of MSMarco 81,980ms 66,205ms 19.2%
Milvus 2.5 Full Text Search Vectors of MSMarco 1,005ms 1,034ms ~
IP Before After Improvement %
bigann SPLADE vectors of MSMarco 85,527ms 75,366ms 12%
Milvus 2.5 Full Text Search Vectors of MSMarco(using IP/TF-IDF metric though, not BM25) 2,839ms 3,191ms -12%

This optimization seems to have an improvement on learned sparse embedding like SPLADE, which is denser, but leads to a little bit of performance degration on more sparse embeddings.

Hold the review of this PR for now until more investigation.

@sre-ci-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: sparknack
To complete the pull request process, please assign hhy3 after the PR has been reviewed.
You can assign the PR to them by writing /assign @hhy3 in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@sre-ci-robot
Copy link
Collaborator

Welcome @sparknack! It looks like this is your first PR to zilliztech/knowhere 🎉

Copy link

mergify bot commented Dec 3, 2024

@sparknack 🔍 Important: PR Classification Needed!

For efficient project management and a seamless review process, it's essential to classify your PR correctly. Here's how:

  1. If you're fixing a bug, label it as kind/bug.
  2. For small tweaks (less than 20 lines without altering any functionality), please use kind/improvement.
  3. Significant changes that don't modify existing functionalities should be tagged as kind/enhancement.
  4. Adjusting APIs or changing functionality? Go with kind/feature.

For any PR outside the kind/improvement category, ensure you link to the associated issue using the format: “issue: #”.

Thanks for your efforts and contribution to the community!.

@sparknack
Copy link
Contributor Author

/kind improvement

@sparknack
Copy link
Contributor Author

issue: #967

@zhengbuqian
Copy link
Collaborator

/hold

@sparknack sparknack changed the title sparse: use staged cursor and upper_bound for WAND [WIP] sparse: use staged cursor and upper_bound for WAND Dec 3, 2024
Copy link

codecov bot commented Dec 3, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 74.39%. Comparing base (3c46f4c) to head (f0d8927).
Report is 262 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff            @@
##           main     #970       +/-   ##
=========================================
+ Coverage      0   74.39%   +74.39%     
=========================================
  Files         0       82       +82     
  Lines         0     6690     +6690     
=========================================
+ Hits          0     4977     +4977     
- Misses        0     1713     +1713     

see 82 files with indirect coverage changes

@mergify mergify bot added the ci-passed label Dec 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants