Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

optimize file based sources #12

Open
smnorris opened this issue Aug 6, 2024 · 1 comment
Open

optimize file based sources #12

smnorris opened this issue Aug 6, 2024 · 1 comment

Comments

@smnorris
Copy link
Collaborator

smnorris commented Aug 6, 2024

For files manually compiled, consider downloading from source where available (vs manually compiling and storing on s3).

Where manual compilation is still required:

  • include just the layer of interest in the file
  • standardize format
@smnorris smnorris changed the title file sources optimize file based sources Aug 6, 2024
@smnorris
Copy link
Collaborator Author

smnorris commented Aug 7, 2024

Looking at the first candidate (migratory bird sancuaries / national wildlife area), it doesn't seem like ogr can reach the .gdb via vsicurl - curl needs the -L flag to follow redirects and download successfully:

curl -v -L -O "https://data-donnees.az.ec.gc.ca/api/file?path=%2Fspecies%2Fprotectrestore%2Fcanadian-protected-conserved-areas-database%2FDatabases%2FProtectedConservedArea_2022.gdb.zip"

Edit - nope.

vsicurl follows the redirects - but gdal requests the file size and the server responds with a 404. Use this config option as a workaround CPL_VSIL_CURL_USE_HEAD=NO

This source should work:
/vsicurl?use_head=no&url=https://data-donnees.az.ec.gc.ca/api/file?path=%2Fspecies%2Fprotectrestore%2Fcanadian-protected-conserved-areas-database%2FDatabases%2FProtectedConservedArea_2022.gdb.zip

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant