Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ValueError: cannot reshape array of size 4028826 into shape (512,512,16,1) #632

Open
bang2003er opened this issue Sep 10, 2024 · 1 comment

Comments

@bang2003er
Copy link

I am trying to download EM data from this website.

import numpy as np
import tifffile
from cloudvolume import CloudVolume
vol = CloudVolume("s3://open-neurodata/bumbarger/bumbarger13/image", mip=0, use_https=True)
print(vol.shape)
for i in tqdm(range(vol.shape[2])):
    img = vol[:,:,i]
    print(img.shape)

But I encountered the following error

(24576, 20480, 2762, 1)
  0%|                                                                                              | 0/2762 [00:00<?, ?it/s]File Read Error: 4028826 bytes, Bbox([11264, 11264, 0],[11776, 11776, 16], dtype=np.int64, unit='vx'), 3.9_3.9_50/11264-11776_11264-11776_0-16, errors: cannot reshape array of size 4028826 into shape (512,512,16,1)
  0%|                                                                                              | 0/2762 [03:51<?, ?it/s]
Traceback (most recent call last):
  File "test.py", line 15, in <module>
    img = vol[:,:,i]
  File "/home/zhangzb/anaconda3/envs/torch/lib/python3.8/site-packages/cloudvolume/frontends/precomputed.py", line 578, in __getitem__
    img = self.download(requested_bbox, self.mip)
  File "/home/zhangzb/anaconda3/envs/torch/lib/python3.8/site-packages/cloudvolume/frontends/precomputed.py", line 777, in download
    tup = self.image.download(
  File "/home/zhangzb/anaconda3/envs/torch/lib/python3.8/site-packages/cloudvolume/datasource/precomputed/image/__init__.py", line 211, in download
    return rx.download(
  File "/home/zhangzb/anaconda3/envs/torch/lib/python3.8/site-packages/cloudvolume/datasource/precomputed/image/rx.py", line 310, in download
    download_chunks_threaded(
  File "/home/zhangzb/anaconda3/envs/torch/lib/python3.8/site-packages/cloudvolume/datasource/precomputed/image/rx.py", line 618, in download_chunks_threaded
    schedule_jobs(
  File "/home/zhangzb/anaconda3/envs/torch/lib/python3.8/site-packages/cloudvolume/scheduler.py", line 150, in schedule_jobs
    return schedule_threaded_jobs(fns, concurrency, progress, total)
  File "/home/zhangzb/anaconda3/envs/torch/lib/python3.8/site-packages/cloudvolume/scheduler.py", line 39, in schedule_threaded_jobs
    tq.put(updatefn(fn))
  File "/home/zhangzb/anaconda3/envs/torch/lib/python3.8/site-packages/cloudvolume/threaded_queue.py", line 257, in __exit__
    self.wait(progress=self.with_progress)
  File "/home/zhangzb/anaconda3/envs/torch/lib/python3.8/site-packages/cloudvolume/threaded_queue.py", line 234, in wait
    self._check_errors()
  File "/home/zhangzb/anaconda3/envs/torch/lib/python3.8/site-packages/cloudvolume/threaded_queue.py", line 191, in _check_errors
    raise err
  File "/home/zhangzb/anaconda3/envs/torch/lib/python3.8/site-packages/cloudvolume/threaded_queue.py", line 153, in _consume_queue
    self._consume_queue_execution(fn)
  File "/home/zhangzb/anaconda3/envs/torch/lib/python3.8/site-packages/cloudvolume/threaded_queue.py", line 180, in _consume_queue_execution
    fn()
  File "/home/zhangzb/anaconda3/envs/torch/lib/python3.8/site-packages/cloudvolume/scheduler.py", line 32, in realupdatefn
    res = fn()
  File "/home/zhangzb/anaconda3/envs/torch/lib/python3.8/site-packages/cloudvolume/datasource/precomputed/image/rx.py", line 573, in process
    labels, bbox = download_chunk(
  File "/home/zhangzb/anaconda3/envs/torch/lib/python3.8/site-packages/cloudvolume/datasource/precomputed/image/rx.py", line 558, in download_chunk
    img3d = decode_fn(meta, filename, content, fill_missing, mip, 
  File "/home/zhangzb/anaconda3/envs/torch/lib/python3.8/site-packages/cloudvolume/datasource/precomputed/image/rx.py", line 642, in decode
    return _decode_helper(  
  File "/home/zhangzb/anaconda3/envs/torch/lib/python3.8/site-packages/cloudvolume/datasource/precomputed/image/rx.py", line 758, in _decode_helper
    return fn(
  File "/home/zhangzb/anaconda3/envs/torch/lib/python3.8/site-packages/cloudvolume/chunks.py", line 147, in decode
    return decode_raw(filedata, shape=shape, dtype=dtype)
  File "/home/zhangzb/anaconda3/envs/torch/lib/python3.8/site-packages/cloudvolume/chunks.py", line 328, in decode_raw
    return np.frombuffer(bytestring, dtype=dtype).reshape(shape, order='F')
ValueError: cannot reshape array of size 4028826 into shape (512,512,16,1)
@william-silversmith
Copy link
Contributor

Hi, I ran this code for a little bit and was not able to reproduce the error. Do you have the latest version of CloudVolume? What value of i do you encounter the error at?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants