-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Flaky fuzz tests for filtered outer SortMergeJoin #12359
Comments
The repro case is to run the same test 1000 times and depending on data distribution between batches the problem can arise |
fyi @viirya |
I believe all of them flaky because of flaky LeftAnti join, all Left, Right, Full depends on LeftAnti starting from #10892 PR. |
Simple test case for LeftOuter
@viirya cc |
For LeftOuter/RightOuter and FullJoin the rootcause is the same. SMJ emits a null row if the filtered match is not found. Given that SMJ emits rows in
|
Related apache/datafusion-comet#398 |
Describe the bug
After replacing the filter for join fuzz tests with the selective one (that doesn't return 100% of input rows), it turned out that following tests may periodically fail for HashJoin vs SortMergeJoin cases:
To Reproduce
To reproduce this one should add
JoinTestType::HjSmj
to mentioned above test cases and run them (likely a couple of times)Expected behavior
SMJ and HJ should return equal resultsets for these test cases
Additional context
No response
The text was updated successfully, but these errors were encountered: