Skip to content

v0.1.1
Compare
Choose a tag to compare
@leepc12 leepc12 released this 30 Mar 21:45
· 146 commits to master since this release
v0.1.1
35482e9
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

Important notice

  • To allow direct transfer between S3 and GCS, Autouri uses gsutil CLI. For this, GCSURI.USE_GSUTIL_FOR_S3 flag must be on. Otherwise Autouri will stream file transfer through a local machine. Old gsutil had a py3 compatibility issue. Users need to update gsutil to >= 4.47.

Added circleci (integrated testing)

Added CLI

  • See autouri --help for details

Improved locking to prevent race condition

  • Tested up to 10 (50 for local file) threads competing to write on the same file.

Others

  • Soft-linking for local paths (AbsPath)
  • Converting relative path into abs path (AbsPath.get_abspath_if_exists())
    • AbsPath only allows absolute path as its name says. so added a helper function for such conversion so that apps using Autouri can use it.
  • Detailed logging (INFO, DEBUG)
  • Can hide flag for AutoURI.cp() and AutoURI.localize() (return_flag=False).
  • Can get URL from public GCS/S3 bucket (without presigning)
    • For GCSURI, S3URI only. Formatting for a public URL looks like the following: 
       _GCS_PUBLIC_URL_FORMAT = 'http://storage.googleapis.com/{bucket}/{path}'
_S3_PUBLIC_URL_FORMAT = 'http://{bucket}.s3.amazonaws.com/{path}'
    • This is different from a presigned URL. It's for a file on a bucket with public access. If users use this function on a private bucket file then this function will still return a URL but public access to this URL will be limited (403).
Assets 2
Loading