Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: to_datetime with mixed iso-str and ints #50721

Open
jbrockmendel opened this issue Jan 12, 2023 · 2 comments
Open

BUG: to_datetime with mixed iso-str and ints #50721

jbrockmendel opened this issue Jan 12, 2023 · 2 comments
Assignees
Labels
Bug Constructors Series/DataFrame/Index/pd.array Constructors Datetime Datetime data dtype
Milestone

Comments

@jbrockmendel
Copy link
Member

jbrockmendel commented Jan 12, 2023

objs = ["2016-01-01 01:02:03+05:00", "2016-01-01 01:02:03+06:00", 0]

pd.to_datetime(objs[::-1])  # <- raises with RuntimeError: No active exception to reraise

pd.to_datetime(objs)  # <- raises ValueError

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pandas/core/tools/datetimes.py", line 1077, in to_datetime
    result = convert_listlike(argc, format)
  File "pandas/core/tools/datetimes.py", line 449, in _convert_listlike_datetimes
    return _array_strptime_with_fallback(arg, name, utc, format, exact, errors)
  File "pandas/core/tools/datetimes.py", line 480, in _array_strptime_with_fallback
    result, timezones = array_strptime(arg, fmt, exact=exact, errors=errors, utc=utc)
  File "pandas/_libs/tslibs/strptime.pyx", line 513, in pandas._libs.tslibs.strptime.array_strptime
    raise
  File "pandas/_libs/tslibs/strptime.pyx", line 342, in pandas._libs.tslibs.strptime.array_strptime
    raise ValueError(f"time data \"{val}\" doesn't "
ValueError: time data "0" doesn't match format "%Y-%m-%d %H:%M:%S%z", at position 2

cc @MarcoGorelli

@jbrockmendel jbrockmendel added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Jan 12, 2023
@MarcoGorelli MarcoGorelli self-assigned this Jan 13, 2023
@MarcoGorelli MarcoGorelli added Datetime Datetime data dtype and removed Needs Triage Issue that has not been reviewed by a pandas team member labels Jan 13, 2023
@MarcoGorelli
Copy link
Member

MarcoGorelli commented Jan 13, 2023

thanks! will address


looks like this was present in 1.5.2 too:

In [1]: to_datetime([0, '2016-01-01 01:02:03+06:00', '2016-01-01 01:02:03+05:00'])
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Cell In [1], line 1
----> 1 to_datetime([0, '2016-01-01 01:02:03+06:00', '2016-01-01 01:02:03+05:00'])

File ~/tmp/.venv/lib/python3.8/site-packages/pandas/core/tools/datetimes.py:1100, in to_datetime(arg, errors, dayfirst, yearfirst, utc, format, exact, unit, infer_datetime_format, origin, cache)
   1098         result = _convert_and_box_cache(argc, cache_array)
   1099     else:
-> 1100         result = convert_listlike(argc, format)
   1101 else:
   1102     result = convert_listlike(np.array([arg]), format)[0]

File ~/tmp/.venv/lib/python3.8/site-packages/pandas/core/tools/datetimes.py:438, in _convert_listlike_datetimes(arg, format, name, tz, unit, errors, infer_datetime_format, dayfirst, yearfirst, exact)
    436 assert format is None or infer_datetime_format
    437 utc = tz == "utc"
--> 438 result, tz_parsed = objects_to_datetime64ns(
    439     arg,
    440     dayfirst=dayfirst,
    441     yearfirst=yearfirst,
    442     utc=utc,
    443     errors=errors,
    444     require_iso8601=require_iso8601,
    445     allow_object=True,
    446 )
    448 if tz_parsed is not None:
    449     # We can take a shortcut since the datetime64 numpy array
    450     # is in UTC
    451     dta = DatetimeArray(result, dtype=tz_to_dtype(tz_parsed))

File ~/tmp/.venv/lib/python3.8/site-packages/pandas/core/arrays/datetimes.py:2177, in objects_to_datetime64ns(data, dayfirst, yearfirst, utc, errors, require_iso8601, allow_object, allow_mixed)
   2175 order: Literal["F", "C"] = "F" if flags.f_contiguous else "C"
   2176 try:
-> 2177     result, tz_parsed = tslib.array_to_datetime(
   2178         data.ravel("K"),
   2179         errors=errors,
   2180         utc=utc,
   2181         dayfirst=dayfirst,
   2182         yearfirst=yearfirst,
   2183         require_iso8601=require_iso8601,
   2184         allow_mixed=allow_mixed,
   2185     )
   2186     result = result.reshape(data.shape, order=order)
   2187 except OverflowError as err:
   2188     # Exception is raised when a part of date is greater than 32 bit signed int

File ~/tmp/.venv/lib/python3.8/site-packages/pandas/_libs/tslib.pyx:427, in pandas._libs.tslib.array_to_datetime()

File ~/tmp/.venv/lib/python3.8/site-packages/pandas/_libs/tslib.pyx:711, in pandas._libs.tslib.array_to_datetime()

File ~/tmp/.venv/lib/python3.8/site-packages/pandas/_libs/tslib.pyx:833, in pandas._libs.tslib._array_to_datetime_object()

RuntimeError: No active exception to reraise

In [2]: pd.__version__
Out[2]: '1.5.2'

@jbrockmendel jbrockmendel added the Constructors Series/DataFrame/Index/pd.array Constructors label Oct 28, 2023
@jbrockmendel
Copy link
Member Author

If it isn't addressed before then, this will be closeable in 3.0 when the deprecated _array_to_datetime_object path is removed.

@jbrockmendel jbrockmendel added this to the 3.0 milestone Oct 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Constructors Series/DataFrame/Index/pd.array Constructors Datetime Datetime data dtype
Projects
None yet
Development

No branches or pull requests

2 participants