Skip to content

Commit

Permalink
Merge branch 'main' into no-positional-bools
Browse files Browse the repository at this point in the history
  • Loading branch information
flying-sheep authored Oct 22, 2024
2 parents 2854e0a + 6dfe780 commit 17cfd6b
Show file tree
Hide file tree
Showing 3 changed files with 61 additions and 2 deletions.
7 changes: 7 additions & 0 deletions .github/workflows/test-gpu.yml
Original file line number Diff line number Diff line change
Expand Up @@ -85,3 +85,10 @@ jobs:
token: ${{ secrets.CODECOV_TOKEN }}
fail_ci_if_error: true
verbose: true

- name: Remove 'run-gpu-ci' Label
if: always()
uses: actions-ecosystem/action-remove-labels@v1
with:
labels: "run-gpu-ci"
github_token: ${{ secrets.GITHUB_TOKEN }}
4 changes: 2 additions & 2 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
repos:
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.6.8
rev: v0.7.0
hooks:
- id: ruff
types_or: [python, pyi, jupyter]
Expand All @@ -14,7 +14,7 @@ repos:
exclude_types:
- markdown
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.6.0
rev: v5.0.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
Expand Down
52 changes: 52 additions & 0 deletions src/anndata/_io/specs/registry.py
Original file line number Diff line number Diff line change
Expand Up @@ -395,10 +395,62 @@ def read_elem_as_dask(
chunks, optional
length `n`, the same `n` as the size of the underlying array.
Note that the minor axis dimension must match the shape for sparse.
Defaults to `(1000, adata.shape[1])` for CSR sparse,
`(adata.shape[0], 1000)` for CSC sparse,
and the on-disk chunking otherwise for dense.
Returns
-------
DaskArray
Examples
--------
Setting up our example:
>>> from scanpy.datasets import pbmc3k
>>> import tempfile
>>> import anndata as ad
>>> import zarr
>>> tmp_path = tempfile.gettempdir()
>>> zarr_path = tmp_path + "/adata.zarr"
>>> adata = pbmc3k()
>>> adata.layers["dense"] = adata.X.toarray()
>>> adata.write_zarr(zarr_path)
Reading a sparse matrix from a zarr store lazily, with custom chunk size and default:
>>> g = zarr.open(zarr_path)
>>> adata.X = ad.experimental.read_elem_as_dask(g["X"])
>>> adata.X
dask.array<make_dask_chunk, shape=(2700, 32738), dtype=float32, chunksize=(1000, 32738), chunktype=scipy.csr_matrix>
>>> adata.X = ad.experimental.read_elem_as_dask(
... g["X"], chunks=(500, adata.shape[1])
... )
>>> adata.X
dask.array<make_dask_chunk, shape=(2700, 32738), dtype=float32, chunksize=(500, 32738), chunktype=scipy.csr_matrix>
Reading a dense matrix from a zarr store lazily:
>>> adata.layers["dense"] = ad.experimental.read_elem_as_dask(g["layers/dense"])
>>> adata.layers["dense"]
dask.array<from-zarr, shape=(2700, 32738), dtype=float32, chunksize=(169, 2047), chunktype=numpy.ndarray>
Making a new anndata object from on-disk, with custom chunks:
>>> adata = ad.AnnData(
... obs=ad.io.read_elem(g["obs"]),
... var=ad.io.read_elem(g["var"]),
... uns=ad.io.read_elem(g["uns"]),
... obsm=ad.io.read_elem(g["obsm"]),
... varm=ad.io.read_elem(g["varm"]),
... )
>>> adata.X = ad.experimental.read_elem_as_dask(
... g["X"], chunks=(500, adata.shape[1])
... )
>>> adata.layers["dense"] = ad.experimental.read_elem_as_dask(g["layers/dense"])
"""
return DaskReader(_LAZY_REGISTRY).read_elem(elem, chunks=chunks)

Expand Down

0 comments on commit 17cfd6b

Please sign in to comment.