#26395 Disabling possibly unnecessary prefetching during GroupIntoBatches by using an experimental flag #26618
Conversation
|
seemingly irrelevant failure during |
nbali
force-pushed
the
fix_for_26395_by_using_an_experiment
branch
from
1f6eefe to
14c6555
Compare
…ntoBatches by using an experimental flag
nbali
force-pushed
the
fix_for_26395_by_using_an_experiment
branch
from
14c6555 to
1d62a65
Compare
|
Run Java_Pulsar_IO_Direct PreCommit |
|
Assigning reviewers. If you would like to opt out of this review, comment R: @robertwb for label java. Available commands:
The PR bot will only process comments in the main thread (not review comments). |
|
Run Java PreCommit |
| @@ -110,6 +112,10 @@ | |||
| public class GroupIntoBatches<K, InputT> | |||
| extends PTransform<PCollection<KV<K, InputT>>, PCollection<KV<K, Iterable<InputT>>>> { | |||
|
|
|||
| /** Experiment to "avoid possibly unnecessary prefetching". */ | |||
| public static final String AVOID_POSSIBLY_UNNECESSARY_PREFETCHING = | |||
| "avoid_possibly_unnecessary_prefetching"; | |||
There was a problem hiding this comment.
consider a descriptive naming: disable_groupintobatches_preferch
|
Tested and found this might not be the cause of data processed being much larger than the actual data size: #26395 (comment) |
|
Reminder, please take a look at this pr: @robertwb |
|
waiting on author |
github-actions
bot
added
Next Action: Author
and removed
Next Action: Reviewers
slow-review
labels
|
FYI I'm trying to make time for this this week, but it's most likely it won't happen. If there is a way to make the reminders snooze for 1-2 weeks feel free to do so. |
|
Reminder, please take a look at this pr: @robertwb |
|
waiting on author |
github-actions
bot
added
Next Action: Author
and removed
Next Action: Reviewers
slow-review
labels
|
This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the [email protected] list. Thank you for your contributions. |
|
This pull request has been closed due to lack of activity. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time. |
|
added this to [Parent issue] Support for Apache Pulsar #31078 |
|
@hpvd I'm pretty sure it's not related to Apache Pulsar |
We haven't reached a verdict on the proper way to implement #26395, so I created a PR for all the alternatives. The chosen one should be reviewed+merged, the rest should be closed. (alternative PR: #26619)
closes #26395
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, commentfixes #<ISSUE NUMBER>instead.[ ] UpdateCHANGES.mdwith noteworthy changes.[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)
See CI.md for more information about GitHub Actions CI.