Issue 1

[vincentsarago]

👋 rio-tiler-fs is an experimental plugin to create a custom COGReader that uses fsspec to open dataset instead of relying on GDAL.

This is experimental because file-like VSI plugin is not yet publish in a stable rasterio version

How To

Within the custom COGReader we are opening the input file (url) using fsspec and then passing it to rasterio.

# rio_tiler.io.cogeo.COGReader
with rasterio.open(src_path) as src:
    data = src.read()

# rio_tiler_fs.reader.COGReader
with fsspec.open(src_path) as fsobj:
    with rasterio.open(fsobj) as src:
        data = src.read()

Why

We have seen numerous reports of issue when using S3 files (mostly when using threads) where for unknown reason (to me) GDAL/Rasterio will loose the ability to read a file.

• Occasional "not recognized as a supported file format" errors when reading from S3 rasterio/rasterio#1686
• https://rasterio.groups.io/g/main/topic/31734579
• S3: intermittent OpenFailed error (GetFileSize?) OSGeo/gdal#1031
• CPLReleaseMutex: Error = 1 (Operation not permitted) rasterio/rasterio#2117

A second reason will be to enable custom cache 🤷
ref: gjoseph92/stackstac#90

[RichardScottOZ]

Yes - had this happen this week - using a dask Fargate cluster and occasionally otherwise something that reads fine then stops - and possibly when reading chunks of same file it craps out.

e.g. use a 1 machine cluster, ok for sometime - use 100 reading lots of parts, instant failure.