Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Iceberg:utilize iceberg zonemap statistics to perform data skipping #18645

Closed
chenzl25 opened this issue Sep 23, 2024 · 1 comment
Closed
Assignees
Milestone

Comments

@chenzl25
Copy link
Contributor

Is your feature request related to a problem? Please describe.

https://github.com/apache/iceberg-rust/blob/cbbd086c92ef5a93d7dc98bbb1b26b9cf044910a/crates/iceberg/src/scan.rs#L98-L103

Iceberg-rust SDK has a method called with_filter for planning. We can use this method to pass predicates to perform data skipping.

Describe the solution you'd like

No response

Describe alternatives you've considered

No response

Additional context

No response

@kwannoel
Copy link
Contributor

  1. impl predicate pushdown interface for iceberg scan.
  2. Convert the predicate from rw to iceberg-rs predicate.
  3. Pass the predicate to the sdk using with-filter.
  4. They may use or / and conditions. First implementation just focus on and case. The or condition might be much more difficult to support.
  5. Can also try directly convert the conditions: https://github.com/apache/iceberg-rust/blob/15b2a7aeea577b84044bda43e37bd7bc07c4a82c/crates/iceberg/src/expr/predicate.rs#L301

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants