Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add first array draft #1090

Merged
merged 30 commits into from
Jun 25, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
0e411ae
Split out graph Expr code from Dataframe Expr code
mrocklin Dec 5, 2023
a9b806a
Clean up a few things while implementing arrays
mrocklin Dec 6, 2023
5c313ae
Add trivial Array implementation
mrocklin Dec 6, 2023
04d78a5
Merge branch 'main' of github.com:dask-contrib/dask-expr into separat…
mrocklin Dec 8, 2023
a0821ac
Merge branch 'main' of github.com:dask-contrib/dask-expr into arrays
mrocklin Dec 8, 2023
f2d0234
Add basic rechunk, also move to a directory
mrocklin Dec 8, 2023
288a1ba
add Rechunk(Rechunk(...)) simplification
mrocklin Dec 8, 2023
aad8e7e
Merge branch 'main' of github.com:dask-contrib/dask-expr into separat…
mrocklin Dec 8, 2023
f13715e
Merge branch 'separate-core-expr' into arrays
mrocklin Dec 8, 2023
d360ca5
Add basic blockwise functionality
mrocklin Dec 8, 2023
e6385dd
add transpose
mrocklin Dec 8, 2023
b9a51e9
Simplify blockwise + rechunk + IO
mrocklin Dec 8, 2023
364db52
Add basic slicing
mrocklin Dec 8, 2023
28fe54f
Support xarray rechunking
mrocklin Dec 9, 2023
673c28f
Some slicing elemwise optimizations
mrocklin Dec 9, 2023
e37cd20
add comment around elemwise future
mrocklin Dec 9, 2023
67635c0
Add simplified random implementation
mrocklin Dec 9, 2023
d2d3343
Build smarter Elemwise class, improve rechunk/slicing optimizations
mrocklin Dec 9, 2023
76d6849
Add basic reduction implementation
mrocklin Dec 9, 2023
8fc48af
Merge branch 'main' of github.com:dask-contrib/dask-expr into arrays
mrocklin Dec 18, 2023
2bb801f
Support compute and persist methods
mrocklin Dec 18, 2023
94d46f9
Add ufuncs, operators, and more reductions
mrocklin Dec 18, 2023
23cd05f
Merge branch 'main' of github.com:dask-contrib/dask-expr into arrays
mrocklin Dec 18, 2023
8d24c4a
Merge branch 'main' of github.com:dask-contrib/dask-expr into arrays
mrocklin Jan 8, 2024
c386a2d
Avoid objects in pprint
mrocklin Jan 18, 2024
e399f40
Merge remote-tracking branch 'origin/main' into pr/mrocklin/471
fjetter Jun 20, 2024
88b9d6c
rebase stuff
fjetter Jun 21, 2024
19b3318
Merge branch 'main' into array_rebase
fjetter Jun 24, 2024
42d1863
Fix remaining tests
phofl Jun 24, 2024
e6f6ef8
Merge branch 'main' into array_rebase
phofl Jun 25, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions ci/environment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ dependencies:
- pyarrow>=7
- pandas>=2
- pre-commit
- xarray
- pip:
- git+https://github.com/dask/distributed
- git+https://github.com/dask/dask
3 changes: 3 additions & 0 deletions dask_expr/_core.py
Original file line number Diff line number Diff line change
Expand Up @@ -150,6 +150,9 @@ def pprint(self):
def __hash__(self):
return hash(self._name)

def __dask_tokenize__(self):
return self._name

def __reduce__(self):
if dask.config.get("dask-expr-no-serialize", False):
raise RuntimeError(f"Serializing a {type(self)} object")
Expand Down
14 changes: 14 additions & 0 deletions dask_expr/array/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
from dask_expr.array import random
from dask_expr.array.core import Array, from_array
from dask_expr.array.reductions import (
mean,
moment,
nanmean,
nanstd,
nansum,
nanvar,
prod,
std,
sum,
var,
)
Loading
Loading