Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: [ffigen] Trying to fix global_test flake #1674

Draft
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

liamappelbe
Copy link
Contributor

@liamappelbe liamappelbe commented Oct 23, 2024

global_test and global_native_test are exactly the same, except that one uses ordinary bindings, and the other uses @Native style bindings. When both tests are run in the same suite, occasionally they will both fail. It's always the case that both fail or both pass.

The flake appears to be in the build, not the test. If I change the workflow to build once and run the tests 10 times (and count the failures instead of stopping on failure), then it's always the case that there are either 0 or 10 failures: https://github.com/dart-lang/native/actions/runs/11471378290/job/31980751070?pr=1674

I hit a bit of a wall at this point, because I'm not sure how to get the bad build artifacts from github. So I've just changed these two tests to share the same native code. They still have their own ffigen bindings, but the underlying so file only has one copy of the global variables. Just have to make sure to reset the test state after each test, so they don't interfere with each other.

This seems to have fixed the flake, because there's no longer a conflict between the tests native artifacts. To verify the fix I changed the workflow to run the build/test 10 times, and there were no flakes: https://github.com/dart-lang/native/actions/runs/12386195112/job/34573726773

Still flaky.

@github-actions github-actions bot added the type-infra A repository infrastructure change or enhancement label Oct 23, 2024
@liamappelbe liamappelbe changed the title Still debugging global flake [WIP] Still debugging global flake Oct 24, 2024
@coveralls
Copy link

coveralls commented Dec 18, 2024

Coverage Status

coverage: 87.958%. remained the same
when pulling 02de189 on more_global_flake
into f2a0e28 on main.

@github-actions github-actions bot removed the type-infra A repository infrastructure change or enhancement label Dec 18, 2024
@liamappelbe liamappelbe changed the title [WIP] Still debugging global flake [ffigen] Fix global_test flake (finally!) Dec 18, 2024
@liamappelbe liamappelbe marked this pull request as ready for review December 18, 2024 04:54
@liamappelbe liamappelbe requested a review from dcharkes December 18, 2024 04:54
@liamappelbe
Copy link
Contributor Author

Of course, as soon as I put this PR out for review, it starts flaking again 😩

@liamappelbe liamappelbe removed the request for review from dcharkes December 18, 2024 05:12
@liamappelbe liamappelbe marked this pull request as draft December 18, 2024 05:12
@liamappelbe liamappelbe changed the title [ffigen] Fix global_test flake (finally!) WIP: [ffigen] Trying to fix global_test flake Dec 18, 2024
@github-actions github-actions bot added the type-infra A repository infrastructure change or enhancement label Dec 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
package:ffigen type-infra A repository infrastructure change or enhancement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants