Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove Python 3.8 Support from Apache Beam #32643

Merged
merged 4 commits into from
Oct 21, 2024

Conversation

jrmccluskey
Copy link
Contributor

@jrmccluskey jrmccluskey commented Oct 3, 2024

Removes Python 3.8 support from Apache Beam and cleans up a number of references to the version. Also stops building 3.8 snapshot containers.

DO NOT MERGE while 2.60 release is in-flight

Part of #31192


Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
  • Update CHANGES.md with noteworthy changes.
  • If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

Build python source distribution and wheels
Python tests
Java tests
Go tests

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

Copy link
Contributor

github-actions bot commented Oct 3, 2024

Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment assign set of reviewers

@jrmccluskey
Copy link
Contributor Author

R: @tvalentyn @damccorm

Since this also impacts things like the build wheels workflow this should be held until the release is further along

Copy link
Contributor

github-actions bot commented Oct 3, 2024

Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control. If you'd like to restart, comment assign set of reviewers

Copy link
Contributor

@damccorm damccorm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good - lets leave it until the release is finalized

@@ -75,12 +75,6 @@
'This version of Apache Beam has not been sufficiently tested on '
'Python %s.%s. You may encounter bugs or missing features.' %
(sys.version_info.major, sys.version_info.minor))
elif sys.version_info.minor == 8:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should also change the boundaries of the above condition to include 3.8

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

@@ -97,15 +97,15 @@ LogOutput "Installing python and dependencies."
export DEBIAN_FRONTEND=noninteractive
apt install -y apt-transport-https ca-certificates software-properties-common curl unzip apt-utils > /dev/null 2>&1
add-apt-repository -y ppa:deadsnakes/ppa > /dev/null 2>&1 && apt update > /dev/null 2>&1
apt install -y python3.8 python3.8-distutils python3-pip > /dev/null 2>&1
apt install -y --reinstall python3.8-distutils > /dev/null 2>&1
apt install -y python3.9 python3.9-distutils python3-pip > /dev/null 2>&1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI, I don't know if python3.9-distutils is available (deadsnakes/issues#150), and not sure if this package is still necessary.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this code exercised by some IT suite?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe these scripts are invoked for playground deployment, not exercised by a test. Looks like we can get away with python3-distutils

@@ -155,7 +155,7 @@ def cythonize(*args, **kwargs):
# Exclude 1.5.0 and 1.5.1 because of
# https://github.com/pandas-dev/pandas/issues/45725
dataframe_dependency = [
'pandas>=1.4.3,!=1.5.0,!=1.5.1,<2.3;python_version>="3.8"',
'pandas>=1.4.3,!=1.5.0,!=1.5.1,<2.3;python_version>="3.9"',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can omit python_version>="3.9" since now it includes all the versions we support

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

conditions like this make sense when we install different dependency versions depending on Python version, for example, pandas 1.x for Python 3.7, and Pandas 2.x for all other versions.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

@@ -271,18 +271,13 @@ def get_portability_package_data():
return files


python_requires = '>=3.8'
python_requires = '>=3.9'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(drive by): line 276 should have >= 13 or > 12

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

Copy link
Contributor

@tvalentyn tvalentyn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM otherwise

@jrmccluskey
Copy link
Contributor Author

The MacOS failures are due to #32811, unrelated to this PR

@jrmccluskey jrmccluskey merged commit 62b0830 into apache:master Oct 21, 2024
118 of 121 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants