Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: tz_localize drops freq from DatetimIndex #36575

Open
1 of 3 tasks
giuliobeseghi opened this issue Sep 23, 2020 · 6 comments
Open
1 of 3 tasks

BUG: tz_localize drops freq from DatetimIndex #36575

giuliobeseghi opened this issue Sep 23, 2020 · 6 comments
Labels
Bug freq retention User expects "freq" attribute to be preserved Frequency DateOffsets Timezones Timezone data dtype

Comments

@giuliobeseghi
Copy link

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • (optional) I have confirmed this bug exists on the master branch of pandas.


Note: Please read this guide detailing how to provide the necessary information for us to reproduce your bug.

Code Sample, a copy-pastable example

import pandas as pd

s = pd.Series(
    [1, 2, 3, 4, 5], index=pd.date_range("2020", periods=5, freq="D")
)
print(s.index.freq)  # <Day>
print(s.tz_localize("europe/london").index.freq)  # None

Problem description

The index should retain the freq attribute despite localization.

Output of pd.show_versions()

It raises

ImportError: Can't determine version for hypothesis
@giuliobeseghi giuliobeseghi added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Sep 23, 2020
@giuliobeseghi
Copy link
Author

Originally posted in #33677

@giuliobeseghi
Copy link
Author

It actually works if tz is "utc"

print(s.tz_localize("europe/London").index.freq)  # None
  • I have checked that this issue has not already been reported.
  • I have confirmed this bug exists on the latest version of pandas.
  • (optional) I have confirmed this bug exists on the master branch of pandas.

Note: Please read this guide detailing how to provide the necessary information for us to reproduce your bug.

Code Sample, a copy-pastable example

import pandas as pd

s = pd.Series(
    [1, 2, 3, 4, 5], index=pd.date_range("2020", periods=5, freq="D")
)
print(s.index.freq)  # <Day>
print(s.tz_localize("europe/london").index.freq)  # None

Problem description

The index should retain the freq attribute despite localization.

Output of pd.show_versions()

@attack68
Copy link
Contributor

attack68 commented Sep 23, 2020

Extended the example to show the impact is solely to Index.

s = pd.Series(
    index=pd.date_range("2020", periods=5, freq="D"),
    data=pd.date_range("2020", periods=5, freq="D"),
    name="date"
)
print(s.index.freq)  
print(s.tz_localize("europe/london").index.freq)  
print(s.dt.tz_localize("europe/london").dt.freq)

On version 1.0.4:

<Day> <Day> D

On version 1.1.2:

<Day> None D

@attack68
Copy link
Contributor

Also link to here: #33940 (comment)

@giuliobeseghi
Copy link
Author

giuliobeseghi commented Sep 23, 2020

Extended the example to show the impact is solely to Index.

s = pd.Series(
    index=pd.date_range("2020", periods=5, freq="D"),
    data=pd.date_range("2020", periods=5, freq="D"),
    name="date"
)
print(s.index.freq)  
print(s.tz_localize("europe/london").index.freq)  
print(s.dt.tz_localize("europe/london").dt.freq)

On version 1.0.4:

<Day> <Day> D

On version 1.1.2:

<Day> None D

Well, that's because the dt accessor always return a frequency when it's inferrable:

import pandas as pd

date_range = pd.date_range("2020", periods=5, freq="D")
date_range.freq = None  # remove frequency from index

s = pd.Series(data=date_range, index=None)
print(s.dt.freq)  # 'D' - dt infers frequency

@mroeschke mroeschke added Frequency DateOffsets Timezones Timezone data dtype and removed Needs Triage Issue that has not been reviewed by a pandas team member labels Aug 13, 2021
@jbrockmendel jbrockmendel added the freq retention User expects "freq" attribute to be preserved label Aug 24, 2023
@veenstrajelmer
Copy link

veenstrajelmer commented Sep 11, 2024

Freq is still being dropped by tz_localize. It is actually maintained if you use tz_localize via dt:

import pandas as pd
s = pd.Series(
    index=pd.date_range("2020", periods=5, freq="D"),
    data=pd.date_range("2020", periods=5, freq="D"),
    name="date"
)
print(s.index.freq)  
print(s.tz_localize("europe/london").index.freq)  
print(s.dt.tz_localize("europe/london").index.freq)

Prints:

<Day>
None
<Day>

However, if you directly work with the index this dt is not available, but you can overwrite the freq from the inferred freq property:

import pandas as pd
s = pd.Series(
    index=pd.date_range("2020", periods=5, freq="D"),
    data=pd.date_range("2020", periods=5, freq="D"),
    name="date"
)
index = s.index
index_aware = index.tz_localize("europe/london")
print(index.freq)  
print(index_aware.freq)  
index_aware.freq = index_aware.inferred_freq
print(index_aware.freq)  

Prints:

<Day>
None
<Day>

No clue how robust this is, so it would be great if the freq can just be passed when applying tz_convert.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug freq retention User expects "freq" attribute to be preserved Frequency DateOffsets Timezones Timezone data dtype
Projects
None yet
Development

No branches or pull requests

5 participants