Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-50397][CORE] Remove deprecated --ip and -i arguments from Master/Worker #48938

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

dongjoon-hyun
Copy link
Member

@dongjoon-hyun dongjoon-hyun commented Nov 22, 2024

What changes were proposed in this pull request?

This PR aims to remove --ip and -i arguments from Master/Worker from Apache Spark 4.0.0.

Why are the changes needed?

These arguments were deprecated on Apr 15, 2013 before v0.8.0-incubating and have never been used in Apache Spark code base.

Utils.checkHost(value, "ip no longer supported, please use hostname " + value)

Does this PR introduce any user-facing change?

Yes, however, I believe these incubating-era arguments are not only misleading (because IPs are not hostnames) but also not required in Apache Spark 4.0.0 users.

How was this patch tested?

Manual review.

BEFORE

$ sbin/start-master.sh --help
Usage: ./sbin/start-master.sh [options]

Options:
  -i HOST, --ip HOST     Hostname to listen on (deprecated, please use --host or -h)
  -h HOST, --host HOST   Hostname to listen on
  -p PORT, --port PORT   Port to listen on (default: 7077)
  --webui-port PORT      Port for web UI (default: 8080)
  --properties-file FILE Path to a custom Spark properties file.
                         Default is conf/spark-defaults.conf.

AFTER

$ sbin/start-master.sh --help
Usage: ./sbin/start-master.sh [options]

Options:
  -h HOST, --host HOST   Hostname to listen on
  -p PORT, --port PORT   Port to listen on (default: 7077)
  --webui-port PORT      Port for web UI (default: 8080)
  --properties-file FILE Path to a custom Spark properties file.
                         Default is conf/spark-defaults.conf.

Was this patch authored or co-authored using generative AI tooling?

No.

@dongjoon-hyun
Copy link
Member Author

To @mridulm , I saw that these were deprecated by you at 0.8.0-incubating.
It's over 10 years and probably I believe we can let them go from Apache Spark 4.0.0. WDYT?

@dongjoon-hyun
Copy link
Member Author

All tests passed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant