Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Product Quantization #79

Merged
merged 26 commits into from
Feb 21, 2024
Merged

Product Quantization #79

merged 26 commits into from
Feb 21, 2024

Commits on Feb 19, 2024

  1. Implement product quantization in lantern cli

    @var77
    var77 committed Feb 19, 2024
    Configuration menu
    Copy the full SHA
    59f17e3 View commit details
    Browse the repository at this point in the history

  2. Process all splits in parallel

    @var77
    var77 committed Feb 19, 2024
    Configuration menu
    Copy the full SHA
    3a6fd65 View commit details
    Browse the repository at this point in the history

  3. Add subvector-id argument and ability to horizontally scale the runni… 

    …ng job among vms
    @var77
    var77 committed Feb 19, 2024
    Configuration menu
    Copy the full SHA
    3a32b9f View commit details
    Browse the repository at this point in the history

  4. Fix progress tracking for pq

    @var77
    var77 committed Feb 19, 2024
    Configuration menu
    Copy the full SHA
    dfb41c7 View commit details
    Browse the repository at this point in the history

  5. Fix indexing bug

    @var77
    var77 committed Feb 19, 2024
    Configuration menu
    Copy the full SHA
    6ccc001 View commit details
    Browse the repository at this point in the history

  6. Parallelize vector compression

    @var77
    var77 committed Feb 19, 2024
    Configuration menu
    Copy the full SHA
    ffa9a49 View commit details
    Browse the repository at this point in the history

  7. Parallelize data fetching and export

    @var77
    var77 committed Feb 19, 2024
    Configuration menu
    Copy the full SHA
    41c12ec View commit details
    Browse the repository at this point in the history

  8. Refactor and separate code parts

    @var77
    var77 committed Feb 19, 2024
    Configuration menu
    Copy the full SHA
    16011ba View commit details
    Browse the repository at this point in the history

  9. Refactor code, pack arguments in a struct

    @var77
    var77 committed Feb 19, 2024
    Configuration menu
    Copy the full SHA
    d2fbdf8 View commit details
    Browse the repository at this point in the history

  10. Add gcp batch job flow

    @var77
    var77 committed Feb 19, 2024
    Configuration menu
    Copy the full SHA
    606ac54 View commit details
    Browse the repository at this point in the history

  11. Add tests for lantern_pq

    @var77
    var77 committed Feb 19, 2024
    Configuration menu
    Copy the full SHA
    dc6ef0f View commit details
    Browse the repository at this point in the history

  12. Add action to push cli image to GCR

    @var77
    var77 committed Feb 19, 2024
    Configuration menu
    Copy the full SHA
    cfa1dab View commit details
    Browse the repository at this point in the history

  13. Remove unnecessary arguments

    @var77
    var77 committed Feb 19, 2024
    Configuration menu
    Copy the full SHA
    4cc5977 View commit details
    Browse the repository at this point in the history

  14. Rename codebook table and params to match lantern pq

    @var77
    var77 committed Feb 19, 2024
    Configuration menu
    Copy the full SHA
    1155b57 View commit details
    Browse the repository at this point in the history

Commits on Feb 20, 2024

  1. Fix naming issues, add --dataset-limit argument

    @var77
    var77 committed Feb 20, 2024
    Configuration menu
    Copy the full SHA
    50fdb62 View commit details
    Browse the repository at this point in the history

  2. Conditionaliy publish latest tag for cli docker image

    @var77
    var77 committed Feb 20, 2024
    Configuration menu
    Copy the full SHA
    d3e4575 View commit details
    Browse the repository at this point in the history

Commits on Feb 21, 2024

  1. Use renamed lantern access method

    @Ngalstyan4 @var77
    Ngalstyan4 authored and var77 committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    000e628 View commit details
    Browse the repository at this point in the history

  2. Release v0.2.0

    @Ngalstyan4 @var77
    Ngalstyan4 authored and var77 committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    92aeef0 View commit details
    Browse the repository at this point in the history

  3. Temporarily change lantern tag for testing before lantern is released

    @Ngalstyan4 @var77
    Ngalstyan4 authored and var77 committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    2a342e6 View commit details
    Browse the repository at this point in the history

  4. Implement pq-quantization in external index construction

    @Ngalstyan4 @var77
    Ngalstyan4 authored and var77 committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    70f284c View commit details
    Browse the repository at this point in the history

  5. Fix codebook offset bug

    @Ngalstyan4 @var77
    Ngalstyan4 authored and var77 committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    19b7ba8 View commit details
    Browse the repository at this point in the history

  6. set pq parameter in index construction when importing

    @Ngalstyan4 @var77
    Ngalstyan4 authored and var77 committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    5adcf10 View commit details
    Browse the repository at this point in the history

  7. Fix codebook lifetime bug in rust<->C interface

    @Ngalstyan4 @var77
    Ngalstyan4 authored and var77 committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    a467cb0 View commit details
    Browse the repository at this point in the history

  8. Prepare for release

    @Ngalstyan4 @var77
    Ngalstyan4 authored and var77 committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    d107f81 View commit details
    Browse the repository at this point in the history

  9. Fix naming for uppercase table names, check if codebook table exists … 

    …before processing
    @var77
    var77 committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    dcdfcd9 View commit details
    Browse the repository at this point in the history

  10. Add pq argument for external index reindexing

    @var77
    var77 committed Feb 21, 2024
    Configuration menu
    Copy the full SHA
    2dc901f View commit details
    Browse the repository at this point in the history