Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Temporarily fix pip check failure on xgboost and grpcio #32432

Merged
merged 7 commits into from
Sep 12, 2024

Conversation

shunping
Copy link
Contributor

@shunping shunping commented Sep 11, 2024

Originally, this PR is to pin pip to version 24.1, so that pip check would not fail on xgboost-1.7.6 and grpcio-1.66.1.

After some attempts, it seems it is a bit inconvenient to enforce pip version, especially inside the venv set up by tox.

Therefore, we decide to make the following changes

  1. for xgboost, since we don't have any plan to invest further on it, we are going to remove it from deps. After that, the xgboost related tests in the ML test suite will be skipped. We can put it back in the future if we decide to support xgboost again ([Feature request]: Support XGBoost 2.x #31252) .

  2. for grpcio, the failure only occurs on MacOS, so we decide to skip pip check on MacOS until pip check failed on grpcio in macos-arm64 grpc/grpc#37660 is fixed.

#fixes #31285

@github-actions github-actions bot added the build label Sep 11, 2024
Copy link
Contributor

Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment assign set of reviewers

@shunping shunping marked this pull request as draft September 11, 2024 23:32
@shunping shunping marked this pull request as draft September 11, 2024 23:32
@shunping
Copy link
Contributor Author

Run Python_ML PreCommit 3.8

@shunping shunping changed the title Pin pip version to 24.1 so pip check wont't fail Temporarily fix pip check failure on xgboost and grpcio Sep 12, 2024
@shunping shunping marked this pull request as ready for review September 12, 2024 01:32
@shunping
Copy link
Contributor Author

shunping commented Sep 12, 2024

R: @tvalentyn @Abacn

Copy link
Contributor

Assigning reviewers. If you would like to opt out of this review, comment assign to next reviewer:

R: @damccorm for label python.
R: @Abacn for label build.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

@Abacn
Copy link
Contributor

Abacn commented Sep 12, 2024

Note that the PR triggered workflow won't run on the change in .github/, which means the passed Python tests / Python Unit Tests (macos-latest, ...) is passed without the changes of .github/workflows/python_tests.yml (and sdks/python/tox.ini) in this PR. Do we still need them?

@shunping
Copy link
Contributor Author

shunping commented Sep 12, 2024

Note that the PR triggered workflow won't run on the change in .github/, which means the passed Python tests / Python Unit Tests (macos-latest, ...) is passed without the changes of .github/workflows/python_tests.yml (and sdks/python/tox.ini) in this PR. Do we still need them?

Are you sure? I think the tests are run with my changes in .github/, since I see the new tox environment I created for macos is used in the tests. For example:
https://github.com/apache/beam/actions/runs/10821274842/job/30022967680?pr=32432

@Abacn
Copy link
Contributor

Abacn commented Sep 12, 2024

ah you are right, I remembered there is this kind of limitation but looks like it does not applies to this specific test. Turns out the limitation applies to pull_request_target but not pull_request trigger

Copy link
Contributor

@Abacn Abacn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm for python_test. For dependency change defer to ml owner

Copy link
Contributor

@damccorm damccorm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks - lets just try to revert the mac pieces once grpc is ready

@damccorm damccorm merged commit 17718a9 into apache:master Sep 12, 2024
112 checks passed
@shunping shunping deleted the pin-pip-version branch September 12, 2024 13:40
@shunping
Copy link
Contributor Author

Thanks! I have created the task for reverting the mac change #32431.

reeba212 pushed a commit to reeba212/beam that referenced this pull request Dec 4, 2024
* Pin pip version to 24.1 so pip check wont't fail

* Trigger precomit python ml tests to verify the fix.

* Pin pip version in pyproject.toml

* Try to enforce pip version in tox.

* Disable xgboost tests by removing xgboost dependency.

* Add tox env for macos

* Revert some unnecessary changes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

The PreCommit Python ML tests with ML deps installed job is flaky
3 participants