PutObjectTagging required for S3 upload

[scottyhq]

The bug

We were trying to upload outputs from hyp3-isce2 (0.9.2) running locally to S3 by using --bucket and --bucket-prefix workflow arguments. It appears the latest S3 Express One Zone buckets (cheaper & faster supposedly but not redundant) don't support object tags, leading to the following traceback:

2024-01-12 18:29:14,299 - root - INFO - Uploading s3://fufits--usw2-az1--x-s3/testrun/S1_136231_IW2_20200604_20200616_VV_INT80_1AE2.zip
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/opt/conda/envs/hyp3-isce2/lib/python3.11/site-packages/hyp3_isce2/__main__.py", line 51, in <module>
    main()
  File "/opt/conda/envs/hyp3-isce2/lib/python3.11/site-packages/hyp3_isce2/__main__.py", line 47, in main
    sys.exit(process_entry_point.load()())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/hyp3-isce2/lib/python3.11/site-packages/hyp3_isce2/insar_tops_burst.py", line 544, in main
    upload_file_to_s3(Path(output_zip), args.bucket, args.bucket_prefix)
  File "/opt/conda/envs/hyp3-isce2/lib/python3.11/site-packages/hyp3lib/aws.py", line 52, in upload_file_to_s3
    S3_CLIENT.put_object_tagging(Bucket=bucket, Key=key, Tagging=tag_set)
  File "/opt/conda/envs/hyp3-isce2/lib/python3.11/site-packages/botocore/client.py", line 553, in _api_call
    return self._make_api_call(operation_name, kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/hyp3-isce2/lib/python3.11/site-packages/botocore/client.py", line 1009, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (NotImplemented) when calling the PutObjectTagging operation: This bucket does not support Object Tags.

Workaround options

I can see the usefulness of tags, but it would be nice to enable this bucket option, maybe with a conditional statement? Not sure how to check for this new bucket type, but could look into it if that seems like the best path forward?

Additional context

ASFHyP3/hyp3-isce2#175 (comment)

[asjohnston-asf]

Good point. The S3 tags are required for plugins being run by HyP3; HyP3 uses them to distinguish which files uploaded by the plugin are data files, browse images, and log files. The S3 tags could be made optional when running a plugin outside of HyP3 as described in ASFHyP3/hyp3-isce2#175 . It's worth mulling, and we'd be open to a PR, but I doubt it's a change we'll invest the time to make ourselves.

Separately, Express One Zone buckets are specifically optimized for workloads with high request volumes and low latency requirements. Per-request costs are half the cost of S3 Standard, but storage costs are seven times more expensive. I'd recommend against Express One Zone in favor of S3 Intelligent Tiering for most HyP3-adjacent workloads.

https://aws.amazon.com/s3/pricing/