Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PDEP-17: Backwards compatibility and deprecation policy #59125

Merged
merged 12 commits into from
Dec 23, 2024
Merged

PDEP-17: Backwards compatibility and deprecation policy #59125

Changes from 2 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
bbe15b9
PDEP-17: Backwards compatibility and deprecation policy
Aloqeely Jun 27, 2024
2c9657b
Final revision
Aloqeely Jun 27, 2024
a2476e5
resolve contradiction
Aloqeely Jul 5, 2024
58d34e0
Link to public API reference
Aloqeely Jul 5, 2024
7c5be2a
Review
Aloqeely Aug 9, 2024
130d85b
Merge branch 'main' into pdep-17
Aloqeely Aug 9, 2024
607ef54
explain variable
Aloqeely Aug 9, 2024
5493e31
address review
Aloqeely Sep 4, 2024
0a05348
More reviews
Aloqeely Sep 18, 2024
04ecc26
Update abstract
Aloqeely Sep 18, 2024
472bfb3
Better wording
Aloqeely Sep 18, 2024
13330c5
Update status
Aloqeely Dec 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 84 additions & 0 deletions web/pandas/pdeps/0017-backwards-compatibility-and-deprecation-policy.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
# PDEP-17: Backwards compatibility and deprecation policy

- Created: 27 June 2024
- Status: Under discussion
- Discussion: [#59125](https://github.com/pandas-dev/pandas/issues/59125)
- Author: [Abdulaziz Aloqeely](https://github.com/Aloqeely)
- Revision: 1

## Abstract

This PDEP defines pandas' backwards compatibility and deprecation policy.

## Motivation

Having a clear backwards compatibility and deprecation policy is crucial to having a healthy ecosystem. We want to ensure users can rely on pandas being stable while still allowing the library to evolve.

This policy will ensure that users have enough time to deal with deprecations while also minimizing disruptions on downstream packages' users.

## Scope

This PDEP covers pandas' approach to backwards compatibility and the deprecation and removal process.

The decision making process for deprecating features is out of scope for this PDEP.
Aloqeely marked this conversation as resolved.
Show resolved Hide resolved

## Background

pandas uses a loose variant of semantic versioning.
Aloqeely marked this conversation as resolved.
Show resolved Hide resolved
A pandas release number is written in the format of ``MAJOR.MINOR.PATCH``.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know that MAJOR.MINOR.PATCH is the terminology typically used in semver, but how do people feel about using a terminology like major.feature.bugfix ?

That feels closer to how we also communicate about it (e.g. when announcing it, we didn't call pandas 2.2.0 a "minor" release)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is definitely easier to understand, but don't you think it might be a bit misleading? Because we do release features even in bugfix (patch) releases.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After digging into past release notes, it seems like v2.2.1 is the only patch release with a new 'feature', so maybe that was just an odd case. I'm OK with this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I suppose that was an odd case (and it was also only a packaging feature, i.e. a new extra, not an actual code feature)

There might always be exceptions (when there is a good reason), but I think at least the general rule is that bug-fix / patch releases only contain bug fixes (and then even mostly regression fixes)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe worth clarifying that we consider "features" from the perspective of a user and not a developer? The latter can happen at any time

Copy link
Member

@rhshadrach rhshadrach Sep 12, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find "bugfix" misleading. We typically do not backport mere bugfixes, only those that fix regressions.


For the purposes of this PDEP, the last minor release before a major release will be referred to as the 'final minor version'.

## General policy

This policy applies to the public API. Anything not part of the public API or is marked as "Experimental" may be changed or removed at anytime.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some random thoughts:

  1. We should state here what the public API is - I assume just anything in the pd. namespace. We've talked about that before but not sure we've ever stated officially
  2. We should probably also clarify what "experimental" means in this PDEP. A recent example has been the nullable integer types. Those came out in January 2019 as part of the 0.24 release but are still labeled experimental

For the sake of proposing something on point 2 I'd say we should drop the experimental label when something survives a full major release cycle

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume just anything in the pd. namespace.

I understand what you're going for, but core is in the pd. namespace. I think we'll have to use "documented as public", or explicitly list it out in this PDEP.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found this page in the docs that defines the public API: https://pandas.pydata.org/docs/reference/index.html, going to hyperlink to it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd say we should drop the experimental label when something survives a full major release cycle

Should we really enforce something like this? I feel like the duration depends on the type of functionality and I can see some cases where we'd want to keep something as experimental for a bit longer.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would starting with 2 full release cycles in the proposal mitigate that concern? I don't have a strong point of view on what that duration is; I just think its important to have something that we all agree upon so things don't stay "experimental" forever

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds more reasonable but do we really mind if things stay experimental for a long period? I personally don't think we should have rules on anything experimental as that sort of defeats the point, but will leave it up for discussion.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally, I'd leave some room for ambiguity here.

Maybe we should just say that we should review experimental features every major release to decide whether they should be promoted to a regular feature.


- Breaking backwards compatibility should benefit more than it harms users.
- Breaking changes should go through a deprecation cycle before being implemented.
Aloqeely marked this conversation as resolved.
Show resolved Hide resolved
Aloqeely marked this conversation as resolved.
Show resolved Hide resolved
- Breaking changes should only occur in major releases.
- No deprecations should be introduced in patch releases.
Aloqeely marked this conversation as resolved.
Show resolved Hide resolved

Some bug fixes may require breaking backwards compatibility. In these cases, a deprecation cycle is not necessary. However, bug fixes which have a large impact on users might be treated as a breaking change. Whether or not a change is a bug fix or an API breaking change is a judgement call.

## Deprecation process

Whenever there are plans to break backwards compatibility, a deprecation must be introduced. The deprecation will preserve the existing behavior while emitting a warning that the behavior will be changing in a future version.
Aloqeely marked this conversation as resolved.
Show resolved Hide resolved

A deprecation's warning message should:
- Provide information on what is changing.
- Mention how to achieve similar behavior if an alternative is available.
- Include the version in which the deprecation will be enforced.
Aloqeely marked this conversation as resolved.
Show resolved Hide resolved
- For large-scale deprecations, it is recommended include a reason for the deprecation, alongside a discussion link to get user feedback.
Aloqeely marked this conversation as resolved.
Show resolved Hide resolved

Additionally, when one introduces a deprecation, they should:
- Use the appropriate warning class. More info on this can be found below.
- Add the GitHub issue/PR number as a comment above the warning line.
- Add an entry in the release notes.
- Mention that the functionality is deprecated in the documentation using the ``.. deprecated::`` directive.

### Which warning class to use

Deprecations should initially use ``DeprecationWarning``, and then be switched to ``FutureWarning`` for broader visibility in the final minor version before the major release they are planned to be removed in.
This implementation detail can be ignored by using the appropriate ``PandasDeprecationWarning`` variable.
Aloqeely marked this conversation as resolved.
Show resolved Hide resolved

Not all deprecations have to use ``DeprecationWarning`` but all deprecations should eventually transition to ``FutureWarning``, i.e. deprecations in the final minor version which are planned to be removed in the major release after will immediately use ``FutureWarning``.

It is recommended to not introduce large-scale deprecations in the final minor version which are planned to be removed in the major release after, since that will immediately be using a loud ``FutureWarning`` with not much time between deprecation and removal. Instead, a ``DeprecationWarning`` should be used, and the removal should be scheduled for a later major release.

### Support window for the final minor version

Defining a release policy is not part of this PDEP's scope, but, the final minor version plays a special role in deprecations, since:
- It is the version where deprecations planned for removal in the next major release get switched from a ``DeprecationWarning`` to a more visible ``FutureWarning``.
- ``FutureWarning`` deprecations released in it will immediately be removed in the next major release.

And so, this document recommends a minimum of 6 months between the final minor version and the major release after it.
Aloqeely marked this conversation as resolved.
Show resolved Hide resolved

### Enforcement of deprecations

When one enforces a deprecation, they should:
- Add an entry in the release notes.
- Replace the ``.. deprecated::`` directive in the documentation with a ``.. versioncchanged::`` directive.
Aloqeely marked this conversation as resolved.
Show resolved Hide resolved

### PDEP-17 History

- 27 June 2024: Initial version.