Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Install Beam from wheels in Dependency Compat Test Suite. #31308

Merged
merged 1 commit into from
May 16, 2024

Conversation

tvalentyn
Copy link
Contributor

Dependency postcommit suite runs multiple small suites of unit tests. Installing beam takes 3-4 minutes to build a wheel. Using a wheel to install beam should speed up the suite, since the wheel will be built once.


Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
  • Update CHANGES.md with noteworthy changes.
  • If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

Build python source distribution and wheels
Python tests
Java tests
Go tests

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

@github-actions github-actions bot added the build label May 15, 2024
@tvalentyn
Copy link
Contributor Author

Currently testing this on https://github.com/apache/beam/actions/runs/9101961896

@tvalentyn tvalentyn marked this pull request as draft May 15, 2024 20:05
@tvalentyn tvalentyn requested a review from Abacn May 15, 2024 21:16
@tvalentyn tvalentyn marked this pull request as ready for review May 15, 2024 21:16
Copy link
Contributor

Assigning reviewers. If you would like to opt out of this review, comment assign to next reviewer:

R: @damccorm for label build.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

@Abacn
Copy link
Contributor

Abacn commented May 15, 2024

if I remember correctly, useWheelDistribution is only effective for Dataflow test (it changes sdk_location pipeline option to wheel). However these tests are running on direct runner. This change alone is a no-op yet?

@tvalentyn
Copy link
Contributor Author

tvalentyn commented May 15, 2024

I thought so too, but after you dug up the PR for me (thanks!) that added this option, i looked closer in https://github.com/apache/beam/blob/6cb30cc5c86ce394ca558c431af55a935b8d9d6b/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy#L3078C121-L3078C136
and it does make a difference for tox . Note that in https://github.com/apache/beam/actions/runs/9101961896 , the runtime is 1hr 5 min, even though I added a bit more tests to that suite (see: #31305). On the other hand, exact same suite without this change times out after 2hrs: https://github.com/apache/beam/actions/runs/9101921047/job/25020312963?pr=31305.

@tvalentyn
Copy link
Contributor Author

exact same suite without this change times out after 2hrs: apache/beam/actions/runs/9101921047/job/25020312963?pr=31305.

reminder that due to nature of GHA permissions, the signal on the PR #31305 itself is not accurate, even though it includes a commit from this PR, GHA still runs long because it hasn't taken effect yet.

@Abacn
Copy link
Contributor

Abacn commented May 16, 2024

I thought so too, but after you dug up the PR for me (thanks!) that added this option, i looked closer in https://github.com/apache/beam/blob/6cb30cc5c86ce394ca558c431af55a935b8d9d6b/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy#L3078C121-L3078C136 and it does make a difference for tox . Note that in https://github.com/apache/beam/actions/runs/9101961896 , the runtime is 1hr 5 min, even though I added a bit more tests to that suite (see: #31305). On the other hand, exact same suite without this change times out after 2hrs: https://github.com/apache/beam/actions/runs/9101921047/job/25020312963?pr=31305.

Oh, I see, yeah, the functionality is added in #28385 making useWheelDistribution working for tox

@Abacn Abacn merged commit 084f23b into apache:master May 16, 2024
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants