Code

feat: Implement bloom_filter_agg#987

Code

Merged

andygrove merged 28 commits intoapache:mainapache/datafusion-comet:mainfrom mbutrovich:bloom_field_aggmbutrovich/datafusion-comet:bloom_field_aggCopy head branch name to clipboardOct 18, 2024

+637-20

Commits

Commits on Sep 25, 2024

• Add test that invokes bloom_filter_agg.

  mbutrovich

  committed
  e662814

• QueryPlanSerde support for BloomFilterAgg.

  mbutrovich

  committed
  20f6e67

Commits on Sep 27, 2024

• Add bloom_filter_agg based on sample UDAF. planner instantiates it now. Added spark_bit_array_tests.

  mbutrovich

  committed
  1ec31a2

• Partial work on Accumulator. Need to finish merge_batch and state.

  mbutrovich

  committed
  3965dc4

Commits on Sep 30, 2024

• BloomFilterAgg state, merge_state, and evaluate. Need more tests.

  mbutrovich

  committed
  62e656c

• Matches Spark behavior. Need to clean up the code quite a bit, and do cargo clippy.

  mbutrovich

  committed
  33ef47d

• Merge branch 'apache:main' into bloom_field_agg

  mbutrovich

  authored
  2040c76

• Remove old comment.

  mbutrovich

  committed
  599a8f9

• Clippy. Increase bloom filter size back to Spark's default.

  mbutrovich

  committed
  a2a8cf3

• API cleanup.

  mbutrovich

  committed
  22aedd9

Commits on Oct 1, 2024

• API cleanup.

  mbutrovich

  committed
  bf22902

Commits on Oct 2, 2024

• Merge branch 'apache:main' into bloom_field_agg

  mbutrovich

  authored
  4b7000c

• Add BloomFilterAgg benchmark to CometExecBenchmark

  mbutrovich

  committed
  88adc75

• Docs.

  mbutrovich

  committed
  a21e0e3

• API cleanup, fix merge_bits to update cardinality.

  mbutrovich

  committed
  5c5d0f9

• Refactor merge_bits to update bit_count with the bit merging.

  mbutrovich

  committed
  cd107e3

• Remove benchmark results file.

  mbutrovich

  committed
  4f06098

• Docs.

  mbutrovich

  committed
  79f6468

• Add native side benchmarks.

  mbutrovich

  committed
  57fe742

• Adjust benchmark parameters to match Spark defaults.

  mbutrovich

  committed
  ec64e4c

• Address review feedback.

  mbutrovich

  committed
  7a81f35

Commits on Oct 3, 2024

• Merge branch 'apache:main' into bloom_field_agg

  mbutrovich

  authored
  013513e

Commits on Oct 4, 2024

• Add assertion to merge_batch.

  mbutrovich

  committed
  3347923

Commits on Oct 9, 2024

• Merge branch 'apache:main' into bloom_field_agg

  mbutrovich

  authored
  5c82f24

Commits on Oct 13, 2024

• Merge branch 'apache:main' into bloom_field_agg

  mbutrovich

  authored
  c39ff1d

Commits on Oct 17, 2024

• Address some review feedback.

  mbutrovich

  committed
  1ed99e3

Commits on Oct 18, 2024

• Only generate native BloomFilterAgg if child has LongType.

  mbutrovich

  committed
  d41a9d2

• Add TODO with GitHub issue link.

  mbutrovich

  committed
  6d13890