From e8abfd7be4c26ba571080679a79484e5d75c7fc2 Mon Sep 17 00:00:00 2001 From: Ellen Marie Dash Date: Tue, 1 Sep 2020 15:26:28 -0400 Subject: [PATCH 1/7] Add "Bundler version locking" RFC. --- text/0000-bundler-version-locking.md | 142 +++++++++++++++++++++++++++ 1 file changed, 142 insertions(+) create mode 100644 text/0000-bundler-version-locking.md diff --git a/text/0000-bundler-version-locking.md b/text/0000-bundler-version-locking.md new file mode 100644 index 0000000..e9be3cf --- /dev/null +++ b/text/0000-bundler-version-locking.md @@ -0,0 +1,142 @@ +- Feature Name: bundler_version_locking +- Start Date: 2020-09-01 +- RFC PR: (leave this empty) +- Bundler Issue: (leave this empty) + +# Summary + +If a user specifies a required Bundler version in the Gemfile/gemspec, it should be installed and used during the normal `bundle install`/`bundle exec` workflow. + +# Motivation + +There are many times where locking your Bundler version is useful. The existence of `BundlerVersionFinder` shows that, but that approach clearly has not worked. **This RFC assumes that `BundlerVersionFinder` is removed first. The discussion on whether or not to do that should be had elsewhere.** + +# Guide-level explanation + +## Example 1 + +For this example, assume Bundler 2.1.4 is installed but Bundler 2.0.2 is not. + +Gemfile: + +``` +source "https://rubygems.org" + +gem "bundler", "= 2.0.2" +``` + +Output: + +``` +$ bundle install +Bundler 2.1.4 is being run, but "Gemfile" requires version 2.0.2. +Installing Bundler 2.0.2... +(... output of installing Bundler 2.0.2 ...) +Switching to Bundler 2.0.2... +(... rest of output, as normal ...) +$ bundle install +(... normal output ...) +``` + +## Example 2 + +For this example, assume both Bundler 2.0.2 and Bundler 2.1.4 are installed. + +Gemfile: + +``` +source "https://rubygems.org" + +gem "bundler", "~> 2.0" +``` + +Output: + +``` +$ bundle install +Bundler 2.1.4 is being run, but "Gemfile" requires version 2.0.2. +Switching to Bundler 2.0.2... +(... normal output ...) +``` + +## Example 3 + +For this example, assume both Bundler 2.0.2 and Bundler 2.1.4 are installed. + +Gemfile: + +``` +source "https://rubygems.org" + +gemspec +``` + +blah.gemspec: + +``` +<...> + spec.add_development_dependency "bundler", "~> 2.0" +<...> +``` + +Output: + +``` +$ bundle install +Bundler 2.1.4 is being run, but "blah.gemspec" requires version 2.0.2. +Switching to Bundler 2.0.2... +(... normal output ...) +``` + +## Example 4 + +For this example, assume both Bundler 2.0.2 and Bundler 2.1.4 are installed. +Note that since the default behavior is to run the newest version installed, and that matches the requirement, it never needs to switch versions. + +Gemfile: + +``` +source "https://rubygems.org" + +gem "bundler", "~> 2.1" +``` + +Output: + +``` +$ bundle install +(... normal output ...) +``` + +# Reference-level explanation + +First, the `BundlerVersionFinder` would be removed, as mentioned in "Motivation." + +Then, Bundler would do the following when `bundle` is executed: + +1. If the first argument isn't `__` _and_ the Gemfile/gemspec specify a required version of Bundler _and_ the requirement isn't met by the currently-running version: + a. Install the required version of Bundler, if needed. + b. Replace the current process with `bundle __ ` (e.g. something along the lines of `Kernel.exec("bundle", "_#{required_version}_", *args)`) +2. Run as normal. + +# Drawbacks + +TBD. (I'm sure there are some.) + +# Rationale and Alternatives + +The main alternative is a refinement of the BundlerVersionFinder, but I think the approach in this RFC better follows the [principle of least surprise](https://en.wikipedia.org/wiki/Principle_of_least_astonishment) by relying on existing knowledge and assumptions. + +The approach in this RFC tries to ensure: + +1. It is inherently opt-in: it won't do anything if you don't explicitly list Bundler as a dependency. +2. The user has more control: + - It's opt-in, so it won't get in the way if it's not actively wanted. + - By implementing it in terms of the `__` feature, we provide a way for users to override the behavior if needed. +3. It builds on existing conventions: + - Locking the Bundler version is done in the same place and way as any other dependency. + - Changing the locked Bundler version is done the same way as any other dependency. + +# Unresolved questions + +There are many quality-of-life things that could be added, like telling users if they're relying on an outdated Bundler version, but I feel this can be added after the fact. From 2a5c0ba29b18fe7e0f3a4f4c95ce49129419849e Mon Sep 17 00:00:00 2001 From: Ellen Marie Dash Date: Tue, 1 Sep 2020 15:38:08 -0400 Subject: [PATCH 2/7] Make output consistent; add an unresolved question. --- text/0000-bundler-version-locking.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/text/0000-bundler-version-locking.md b/text/0000-bundler-version-locking.md index e9be3cf..4210109 100644 --- a/text/0000-bundler-version-locking.md +++ b/text/0000-bundler-version-locking.md @@ -35,6 +35,8 @@ Installing Bundler 2.0.2... Switching to Bundler 2.0.2... (... rest of output, as normal ...) $ bundle install +Bundler 2.1.4 is being run, but "Gemfile" requires version 2.0.2. +Switching to Bundler 2.0.2... (... normal output ...) ``` @@ -140,3 +142,8 @@ The approach in this RFC tries to ensure: # Unresolved questions There are many quality-of-life things that could be added, like telling users if they're relying on an outdated Bundler version, but I feel this can be added after the fact. + +Things I want to figure out: + +1. If the version it needs is found, should it still print a message explaining what version was ran and which was switched to? Or should it avoid printing anything unless there's a problem? + - In theory, passing `--verbose` includes this, because the first line from `bundle install --verbose` is `Running `bundle install --verbose` with bundler 2.0.2` From 961540036c7018efbdced921b3d0d546b148067b Mon Sep 17 00:00:00 2001 From: Ellen Marie Dash Date: Tue, 1 Sep 2020 15:41:58 -0400 Subject: [PATCH 3/7] Make example output more accurate. --- text/0000-bundler-version-locking.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/text/0000-bundler-version-locking.md b/text/0000-bundler-version-locking.md index 4210109..33c5dd4 100644 --- a/text/0000-bundler-version-locking.md +++ b/text/0000-bundler-version-locking.md @@ -29,13 +29,13 @@ Output: ``` $ bundle install -Bundler 2.1.4 is being run, but "Gemfile" requires version 2.0.2. +Bundler 2.1.4 is being run, but "Gemfile" requires version "= 2.0.2". Installing Bundler 2.0.2... (... output of installing Bundler 2.0.2 ...) Switching to Bundler 2.0.2... (... rest of output, as normal ...) $ bundle install -Bundler 2.1.4 is being run, but "Gemfile" requires version 2.0.2. +Bundler 2.1.4 is being run, but "Gemfile" requires version "= 2.0.2". Switching to Bundler 2.0.2... (... normal output ...) ``` @@ -56,7 +56,7 @@ Output: ``` $ bundle install -Bundler 2.1.4 is being run, but "Gemfile" requires version 2.0.2. +Bundler 2.1.4 is being run, but "Gemfile" requires version "~> 2.0". Switching to Bundler 2.0.2... (... normal output ...) ``` @@ -85,7 +85,7 @@ Output: ``` $ bundle install -Bundler 2.1.4 is being run, but "blah.gemspec" requires version 2.0.2. +Bundler 2.1.4 is being run, but "blah.gemspec" requires version "~> 2.0". Switching to Bundler 2.0.2... (... normal output ...) ``` From ca9c777c9927cb8a68846f810f5e93f28fc50e28 Mon Sep 17 00:00:00 2001 From: Ellen Marie Dash Date: Fri, 20 Nov 2020 14:34:48 -0500 Subject: [PATCH 4/7] Make example output more accurate; remove irrelevant remarks about BundlerVersionFinder being removed. --- text/0000-bundler-version-locking.md | 53 ++++++++++++++++------------ 1 file changed, 30 insertions(+), 23 deletions(-) diff --git a/text/0000-bundler-version-locking.md b/text/0000-bundler-version-locking.md index 33c5dd4..a9f0833 100644 --- a/text/0000-bundler-version-locking.md +++ b/text/0000-bundler-version-locking.md @@ -9,10 +9,14 @@ If a user specifies a required Bundler version in the Gemfile/gemspec, it should # Motivation -There are many times where locking your Bundler version is useful. The existence of `BundlerVersionFinder` shows that, but that approach clearly has not worked. **This RFC assumes that `BundlerVersionFinder` is removed first. The discussion on whether or not to do that should be had elsewhere.** +There are many times where locking your Bundler version is useful. The existence of `BundlerVersionFinder` shows that, but that approach has been confusing for end-users. # Guide-level explanation +If you need to pin the Bundler version, simply specify it in your `Gemfile` or `gemspec and run `bundle install` as usual. + +If the running version doesn't meet the requirements, Bundler will install the specified version of itself, and then re-run itself using that version. + ## Example 1 For this example, assume Bundler 2.1.4 is installed but Bundler 2.0.2 is not. @@ -25,19 +29,22 @@ source "https://rubygems.org" gem "bundler", "= 2.0.2" ``` -Output: +Output, first run: ``` $ bundle install Bundler 2.1.4 is being run, but "Gemfile" requires version "= 2.0.2". -Installing Bundler 2.0.2... -(... output of installing Bundler 2.0.2 ...) -Switching to Bundler 2.0.2... +Fetching bundler-2.0.2.gem +(... rest of output from installing Bundler 2.0.2 ...) +Using bundler 2.0.2 (... rest of output, as normal ...) +``` + +Output, second or later run: +``` $ bundle install -Bundler 2.1.4 is being run, but "Gemfile" requires version "= 2.0.2". -Switching to Bundler 2.0.2... -(... normal output ...) +Using bundler 2.0.2 +(... rest of output, as normal ...) ``` ## Example 2 @@ -56,9 +63,8 @@ Output: ``` $ bundle install -Bundler 2.1.4 is being run, but "Gemfile" requires version "~> 2.0". -Switching to Bundler 2.0.2... -(... normal output ...) +Using bundler 2.0.2 +(... rest of output, as normal ...) ``` ## Example 3 @@ -81,12 +87,20 @@ blah.gemspec: <...> ``` -Output: +Output, first run: ``` $ bundle install Bundler 2.1.4 is being run, but "blah.gemspec" requires version "~> 2.0". -Switching to Bundler 2.0.2... +Using bundler 2.0.2 +(... normal output ...) +``` + +Output, second or later run: + +``` +$ bundle install +Using bundler 2.0.2 (... normal output ...) ``` @@ -112,13 +126,11 @@ $ bundle install # Reference-level explanation -First, the `BundlerVersionFinder` would be removed, as mentioned in "Motivation." - Then, Bundler would do the following when `bundle` is executed: 1. If the first argument isn't `__` _and_ the Gemfile/gemspec specify a required version of Bundler _and_ the requirement isn't met by the currently-running version: a. Install the required version of Bundler, if needed. - b. Replace the current process with `bundle __ ` (e.g. something along the lines of `Kernel.exec("bundle", "_#{required_version}_", *args)`) + b. Replace the current process with `bundle __ ` (e.g. something along the lines of `Kernel.exec($0, "_#{required_version}_", *ARGV)`) 2. Run as normal. # Drawbacks @@ -127,8 +139,6 @@ TBD. (I'm sure there are some.) # Rationale and Alternatives -The main alternative is a refinement of the BundlerVersionFinder, but I think the approach in this RFC better follows the [principle of least surprise](https://en.wikipedia.org/wiki/Principle_of_least_astonishment) by relying on existing knowledge and assumptions. - The approach in this RFC tries to ensure: 1. It is inherently opt-in: it won't do anything if you don't explicitly list Bundler as a dependency. @@ -139,11 +149,8 @@ The approach in this RFC tries to ensure: - Locking the Bundler version is done in the same place and way as any other dependency. - Changing the locked Bundler version is done the same way as any other dependency. +I am not aware of any alternatives that accomplish all of these. + # Unresolved questions There are many quality-of-life things that could be added, like telling users if they're relying on an outdated Bundler version, but I feel this can be added after the fact. - -Things I want to figure out: - -1. If the version it needs is found, should it still print a message explaining what version was ran and which was switched to? Or should it avoid printing anything unless there's a problem? - - In theory, passing `--verbose` includes this, because the first line from `bundle install --verbose` is `Running `bundle install --verbose` with bundler 2.0.2` From c6203b6c268564646ed2982e207e59068b55b944 Mon Sep 17 00:00:00 2001 From: Ellen Marie Dash Date: Fri, 4 Dec 2020 20:00:28 -0500 Subject: [PATCH 5/7] [Bundler Version Locking] Fix formatting issues; remove concrete implementation suggestions since it's clear more discussion needs to happen. --- text/0000-bundler-version-locking.md | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/text/0000-bundler-version-locking.md b/text/0000-bundler-version-locking.md index a9f0833..8508673 100644 --- a/text/0000-bundler-version-locking.md +++ b/text/0000-bundler-version-locking.md @@ -13,7 +13,7 @@ There are many times where locking your Bundler version is useful. The existence # Guide-level explanation -If you need to pin the Bundler version, simply specify it in your `Gemfile` or `gemspec and run `bundle install` as usual. +If you need to pin the Bundler version, simply specify it in your `Gemfile` or `gemspec` file and run `bundle install` as usual. If the running version doesn't meet the requirements, Bundler will install the specified version of itself, and then re-run itself using that version. @@ -129,13 +129,13 @@ $ bundle install Then, Bundler would do the following when `bundle` is executed: 1. If the first argument isn't `__` _and_ the Gemfile/gemspec specify a required version of Bundler _and_ the requirement isn't met by the currently-running version: - a. Install the required version of Bundler, if needed. - b. Replace the current process with `bundle __ ` (e.g. something along the lines of `Kernel.exec($0, "_#{required_version}_", *ARGV)`) + a. Install the required version of Bundler if needed, respecting Gemfile.lock as normal. + b. Replace the current process with the equivalent of `bundle `, using the correct version. 2. Run as normal. # Drawbacks -TBD. (I'm sure there are some.) +This does add localized complexity to part of the codebase, either in the binstub or the `bundle install` process (depending on the implementation). # Rationale and Alternatives @@ -144,7 +144,7 @@ The approach in this RFC tries to ensure: 1. It is inherently opt-in: it won't do anything if you don't explicitly list Bundler as a dependency. 2. The user has more control: - It's opt-in, so it won't get in the way if it's not actively wanted. - - By implementing it in terms of the `__` feature, we provide a way for users to override the behavior if needed. + - By respecting the `__` feature, we provide a way for users to override the behavior if needed. 3. It builds on existing conventions: - Locking the Bundler version is done in the same place and way as any other dependency. - Changing the locked Bundler version is done the same way as any other dependency. @@ -153,4 +153,6 @@ I am not aware of any alternatives that accomplish all of these. # Unresolved questions -There are many quality-of-life things that could be added, like telling users if they're relying on an outdated Bundler version, but I feel this can be added after the fact. +There are many quality-of-life things that could be added, like telling users if they're relying on an outdated Bundler version, but these can be added after the fact. + +The exact implentation is still unclear — it could be part of `bundle install`, or installing the right Bundler version could be handled by the `bundle` binstub. From 04ac4b6c60ecca6245712f0876614dd437df6e14 Mon Sep 17 00:00:00 2001 From: Ellen Marie Dash Date: Sat, 16 Jan 2021 00:04:48 -0500 Subject: [PATCH 6/7] [bundler version locking] Basically rewrote the entire thing based on prior discussions. --- text/0000-bundler-version-locking.md | 67 +++++++++++++++++++++------- 1 file changed, 51 insertions(+), 16 deletions(-) diff --git a/text/0000-bundler-version-locking.md b/text/0000-bundler-version-locking.md index 8508673..830a5a9 100644 --- a/text/0000-bundler-version-locking.md +++ b/text/0000-bundler-version-locking.md @@ -5,17 +5,35 @@ # Summary -If a user specifies a required Bundler version in the Gemfile/gemspec, it should be installed and used during the normal `bundle install`/`bundle exec` workflow. +Bundler should respect the exact version specified for a project. + +If there is a lockfile (typically `Gemfile.lock`) with a `BUNDLED WITH` statement, +it should install and use that Bundler version. + +Otherwise, if there is a dependency on `bundler` (in `Gemfile` or equivalent), +it should install and use that Bundler version. + +This should happen transparently during the normal `bundle install`/`bundle exec` workflow. # Motivation -There are many times where locking your Bundler version is useful. The existence of `BundlerVersionFinder` shows that, but that approach has been confusing for end-users. +There are many times where locking your Bundler version is useful. +The existence of `BundlerVersionFinder` shows that, but the initial attempt +caused innumerable problems. This is an attempt to resolve those problems. # Guide-level explanation -If you need to pin the Bundler version, simply specify it in your `Gemfile` or `gemspec` file and run `bundle install` as usual. +Pin the Bundler version for a project by either: + +1. Commiting `Gemfile.lock`. +2. Adding `gem "bundler", some_version_constraint` to Gemfile. + +If you take approach 1, you can upgrade the locked Bundler version by +running `bundle update --bundler`. + +If you take approach 2, you can upgrade the locked Bundler version by +changing the version constraint and running `bundle install`. -If the running version doesn't meet the requirements, Bundler will install the specified version of itself, and then re-run itself using that version. ## Example 1 @@ -126,12 +144,22 @@ $ bundle install # Reference-level explanation -Then, Bundler would do the following when `bundle` is executed: - -1. If the first argument isn't `__` _and_ the Gemfile/gemspec specify a required version of Bundler _and_ the requirement isn't met by the currently-running version: - a. Install the required version of Bundler if needed, respecting Gemfile.lock as normal. - b. Replace the current process with the equivalent of `bundle `, using the correct version. -2. Run as normal. +When executing a Bundler command, it should do the following: + +1. If the first argument is `__`: + 1. If the specified version is running, skip to step 5. + 2. Install the specified version, if needed. + 3. Re-execute Bundler using the specified version. +2. If there is a lockfile with a `BUNDLED WITH` statement: + 1. If the specified version is running, skip to step 5. + 2. Install the version specified, if needed. + 3. Re-execute Bundler using the specified version. +3. Resolve dependencies. +4. If the resolved dependencies include `bundler`: + 1. If the specified version is running, skip to step 5. + 2. Install the Bundler version specified, if needed. + 3. Re-execute Bundler using the specified version. +5. Run as normal. # Drawbacks @@ -141,9 +169,8 @@ This does add localized complexity to part of the codebase, either in the binstu The approach in this RFC tries to ensure: -1. It is inherently opt-in: it won't do anything if you don't explicitly list Bundler as a dependency. -2. The user has more control: - - It's opt-in, so it won't get in the way if it's not actively wanted. +1. It is transparent about what is occurring. +2. The user stays in control: - By respecting the `__` feature, we provide a way for users to override the behavior if needed. 3. It builds on existing conventions: - Locking the Bundler version is done in the same place and way as any other dependency. @@ -153,6 +180,14 @@ I am not aware of any alternatives that accomplish all of these. # Unresolved questions -There are many quality-of-life things that could be added, like telling users if they're relying on an outdated Bundler version, but these can be added after the fact. - -The exact implentation is still unclear — it could be part of `bundle install`, or installing the right Bundler version could be handled by the `bundle` binstub. +1. There are many quality-of-life things that could be added, like telling + users if they're relying on an outdated Bundler version, but these can be + added after the fact. +2. The exact implentation is still unclear — it could be part of + `bundle install`, or installing the right Bundler version could be handled + by the `bundle` binstub. +3. How do we [preserve the system environment](https://github.com/rubygems/rfcs/pull/29#issuecomment-735416819)? + ``` + $ MY_ENV=a ruby -e 'ENV["MY_ENV"]="b";Kernel.exec("ruby", "-e", "print ENV[\"MY_ENV\"]")' + b + ``` From 75c10612aa58e9c03981579e294b4344c1dff9f7 Mon Sep 17 00:00:00 2001 From: Ellen Marie Dash Date: Fri, 16 Jul 2021 17:57:36 -0400 Subject: [PATCH 7/7] Add more details to Motivation section, based on @deivid-rodriguez's and @zofrex's input. --- text/0000-bundler-version-locking.md | 42 ++++++++++++++++++++++++++-- 1 file changed, 39 insertions(+), 3 deletions(-) diff --git a/text/0000-bundler-version-locking.md b/text/0000-bundler-version-locking.md index 830a5a9..0673a13 100644 --- a/text/0000-bundler-version-locking.md +++ b/text/0000-bundler-version-locking.md @@ -17,9 +17,45 @@ This should happen transparently during the normal `bundle install`/`bundle exec # Motivation -There are many times where locking your Bundler version is useful. -The existence of `BundlerVersionFinder` shows that, but the initial attempt -caused innumerable problems. This is an attempt to resolve those problems. +## Usability + +Bundler is used in a wide variety of scenarios. Experience has proven that +seemingly-minor changes in Bundler can cause things to break in certain +situations, even if the intention was to preserve compatibility. + +By explicitly listing Bundler as a dependency, like it allows you to do with +other dependencies, you can ensure people are using the expected versions. + +To keep the exact version, either use a tight constraint (e.g. `= 2.2.24`) +or use a lockfile. To use a version expected to be compatible, you can +use a `~>` constraint (e.g. `~> 2.2.0`). + +## Source Analysis + +Bundler is currently the only Ruby dependency where you can not reliably +determine from the source code for a project which version will be run. +Every other gem a project depends on is included in the `Gemfile.lock`, +and the Ruby version is in the `Gemfile` and/or `.ruby-version`. + +Even bundler-audit [does not currently check the Bundler version](https://github.com/rubysec/bundler-audit/pull/299) +due to the complexity of doing so. + +Any sufficiently complex software has security vulnerabilities, and Bundler +is no exception. By including the version of Bundler used in Gemfile.lock, +and respecting the version in the Gemfile when Gemfile.lock does not exist, +we bring Bundler in line with the rest of the dependencies. + +## Forcing Bundler Updates + +If Bundler has a bug that causes problems for you, or a vulnerability that +affects you, the best you could currently do is have a script install the +appropriate version. However, this can only work if people remember to run +the script. + +By allowing the Gemfile and Gemfile.lock files to dictate the version of +Bundler that is used, it becomes possible to force Bundler updates to avoid +bugs and vulnerabilities. Running `bundle install` becomes enough to update +_every_ dependency, including Bundler itself. # Guide-level explanation