Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DEPR: allowing non-round floats in to_datetime/DatetimeIndex/Timestamp/... #53445

Open
jbrockmendel opened this issue May 29, 2023 · 4 comments
Labels
Bug Deprecate Functionality to remove in pandas Needs Discussion Requires discussion from core team before further action

Comments

@jbrockmendel
Copy link
Member

jbrockmendel commented May 29, 2023

A frequent pain point (#53198, #52286) since we introduced non-nano dt64 support is that to_datetime(x, unit=...).unit does not necessarily mach the user's given unit. This is behavior is documented but still confusing.

We could change/deprecate the behavior so that the output unit matches the input unit. The main things standing in the way of that:

  1. For float inputs that would introduce unwanted rounding. i.e. ATM pd.Timestamp(1.5, unit="s") currently returns the fractional second, but if we changed this would round down (users might be surprised that this always rounds down as opposed to round-nearest)
  2. For units lower than "s" we would have to return closest-supported-unit. We do that in a bunch of places so it isn't a huge deal, but it does mean the policy is more complicated than "output matches input".

The first here seems to be the problematic one. But AFAIK floats just aren't passed that often. When they are, I suspect they are usually round but cast to float so that NaN can be set. Does anyone have a use case for non-round floats in these functions?

If not, we should deprecate the current support, which will un-block allowing "unit" to behave more intuitively.

cc @MarcoGorelli

@jbrockmendel jbrockmendel added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels May 29, 2023
@MarcoGorelli
Copy link
Member

By 'deprecate the current support', do you mean that pandas should error if non-round floats are passed? If so, I'd be very much on board

@MarcoGorelli MarcoGorelli added Deprecate Functionality to remove in pandas Needs Discussion Requires discussion from core team before further action and removed Needs Triage Issue that has not been reviewed by a pandas team member labels May 30, 2023
@jbrockmendel
Copy link
Member Author

raise or round, either way would be OK with me

@MarcoGorelli
Copy link
Member

cool - I'd prefer raising tbh

@jbrockmendel
Copy link
Member Author

jbrockmendel commented Oct 26, 2023

Looks like we get non-round floats from SAS. We call to_datetime from io.sas.sas7bdat._convert_datetimes with floats that can represent fractional seconds.

edit: im having trouble determining what resolution these are supposed to represent in the SAS format. Good reason not to use floats for this!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Deprecate Functionality to remove in pandas Needs Discussion Requires discussion from core team before further action
Projects
None yet
Development

No branches or pull requests

2 participants