Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move year to .dt.year, and other namespace-specific function #341

Closed
MarcoGorelli opened this issue Dec 20, 2023 · 1 comment
Closed

Move year to .dt.year, and other namespace-specific function #341

MarcoGorelli opened this issue Dec 20, 2023 · 1 comment

Comments

@MarcoGorelli
Copy link
Contributor

We'd originally decided not to bother with temporal (and other) namespaces, and just have .year be a column method, rather than .dt.year

I'm suggesting to go back on this decision, because of the risk of conflicts - I'll explain

pandas is adding nested datatypes (e.g. pyarrow struct pandas-dev/pandas#54977 , and pyarrow list pandas-dev/pandas#55777)

Some methods on those datatypes will clash with the column/dataframe ones. For example:

  • Column.get_value(2) means "get the value from the second row"
  • Column.list.get_value(2) means "for each row, get the second element in that row's list"

Likewise for Column.mean vs Column.list.mean - the former being a reduction, the latter a transformation (preserves input shape)

Presumably, we'll eventually have nested datatypes in the standard too?

So, I'd suggest we mirror existing dataframe libraries and have namespaces for functionality which is limited to certain datatypes:

  • .dt for temporal functions
  • .str for string manipulation functions
  • later, if/when we have nested datatypes, .list / .struct, and .cat for categorical
@jorisvandenbossche
Copy link
Member

I think there are also argument to consider nested data types separately, and to have those kind of accessors there while not for other data types.

In contrast to primitive / non-nested data types, those nested data types will indeed have a whole category of different methods that specifically deal with their nested nature (access sub-fields, apply a scalar function on each element in the list, apply an aggregation across the list). That might indeed warrant an accessor, because I think the alternative would be to just prefix all those methods with for example list_ (list_mean, list_get, ..), and at that point an accessor indeed makes sense.

But personally I would say the same reasoning doesn't really apply to other data types (e.g. there haven't yet been any methods where we would consider prefixing with datetime_..?)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants