-
-
Notifications
You must be signed in to change notification settings - Fork 27
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Split out graph Expr code from Dataframe Expr code #470
Conversation
from dask_expr import _core as core, _expr as expr
everything = set(dir(object))
c = set(dir(core.Expr)) - everything
d = set(dir(expr.Expr)) - everything - c What's in core.Expr
What's in expr.Expr{'__add__',
'__and__',
'__bool__',
'__getattr__',
'__getitem__',
'__invert__',
'__mul__',
'__neg__',
'__or__',
'__pos__',
'__radd__',
'__rand__',
'__rge__',
'__rgt__',
'__rle__',
'__rlt__',
'__rmul__',
'__ror__',
'__rsub__',
'__rtruediv__',
'__rxor__',
'__sub__',
'__truediv__',
'__xor__',
'_combine_similar',
'_combine_similar_branches',
'_divisions',
'_is_length_preserving',
'abs',
'align',
'all',
'any',
'apply',
'astype',
'clip',
'columns',
'combine_first',
'combine_similar',
'count',
'divisions',
'dtypes',
'fillna',
'idxmax',
'idxmin',
'index',
'isna',
'isnull',
'known_divisions',
'mask',
'max',
'mean',
'memory_usage_per_partition',
'min',
'mode',
'nbytes',
'ndim',
'npartitions',
'nunique_approx',
'optimize',
'prod',
'rename_axis',
'replace',
'round',
'size',
'std',
'sum',
'to_timestamp',
'var',
'where'} |
There are other things we could think about doing, like renaming |
Do you have an idea how we can solve the issue that defining things like addition and multiplication on the expression gets more complex if we split this up? This is why we postponed the initial pr IIRC |
I think that we have array.Add and dataframe.Add. I intentionally didn't move the |
Beforehand meaning before calling this on the expression level or on the expression level? I think this is better suited on the collection level generally speaking. |
All I mean is that the answer with Add and Mul and such is not to share, at least short term. Regardless, I think that this change doesn't get into that question. I think that this is a safe change to make. Maybe we should chat live sometime today? |
Closes #142
Supercedes #158
This is the smallest step I could think of to start this change. I may want to play around a little with arrays this week in anticipation of hanging out with @dcherian next week. I figured it would be good to split this off now if possible.
(sorry for the poor timing, I know it's an active week for this repository)
cc @phofl