Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] rgw/sfs: default retention mode not applied to objects on multipart_upload_v2::complete #761

Closed
giubacc opened this issue Oct 17, 2023 · 9 comments · Fixed by aquarist-labs/ceph#254
Closed

[BUG] rgw/sfs: default retention mode not applied to objects on multipart_upload_v2::complete #761

giubacc opened this issue Oct 17, 2023 · 9 comments · Fixed by aquarist-labs/ceph#254
Assignees
@giubacc
Labels
area/rgw-sfs RGW & SFS related kind/bug Something isn't working priority/1 Should be fixed for next release
Milestone
v0.24.0

Comments

@giubacc
Copy link

giubacc commented Oct 17, 2023 •
edited
Loading

[UPDATE] this is a backend issue.

SFSMultipartUploadV2::complete() does not perform the same updates on attrs made by SFSAtomicWriter::complete.
Specifically: the attribute RGW_ATTR_OBJECT_RETENTION is not updated with the bucket's default retention mode.

Describe the bug A clear and concise description of what the bug is.

After having created a bucket with a default retention mode:

image

Objects put in the bucket don't have the expected retention mode set:

image
@github-project-automation github-project-automation bot moved this to Backlog in S3GW Oct 17, 2023
@github-actions github-actions bot added the triage/waiting Waiting for triage label Oct 17, 2023
@giubacc giubacc added kind/bug Something isn't working area/ui User Interface triage/waiting Waiting for triage and removed triage/waiting Waiting for triage labels Oct 17, 2023
@votdev
Copy link
Contributor

votdev commented Oct 17, 2023 •
edited
Loading

Can you confirm that the retention mode settings are NOT set on the bucket? If they are set correctly, then this is a rgw/sfs issue.

@giubacc
Copy link
Author

giubacc commented Oct 17, 2023

I will double check; it can be a backed issue but it would be unlikely given that I'm using the same backend version used last week when I'm almost sure this was working.

@votdev
Copy link
Contributor

votdev commented Oct 17, 2023

I will double check; it can be a backed issue but it would be unlikely given that I'm using the same backend version used last week when I'm almost sure this was working.

There were no relevant changes to the UI + REST API in the last weeks. So it must be a rgw/sfs issue.

@giubacc
Copy link
Author

giubacc commented Oct 17, 2023

ok, I will do a complete analysis

@giubacc giubacc added area/rgw-sfs RGW & SFS related and removed area/ui User Interface labels Oct 17, 2023
@giubacc giubacc changed the title [BUG] UI: default retention mode not applied to objects [BUG] rgw/sfs: default retention mode not applied to objects on multipart_writer_v2::complete() Oct 17, 2023
@giubacc giubacc changed the title [BUG] rgw/sfs: default retention mode not applied to objects on multipart_writer_v2::complete() [BUG] rgw/sfs: default retention mode not applied to objects on multipart_upload_v2::complete Oct 17, 2023
@giubacc
Copy link
Author

giubacc commented Oct 17, 2023

@votdev confirmed rgw/sfs issue; last week I was using the PutObject API, the UI is using the multipart upload API that fails to update the object lock attributes on the uploaded object.

@jecluis
Copy link
Contributor

jecluis commented Oct 19, 2023

@giubacc do I understand you correctly: this is an sfs bug, not on the UI?

@giubacc
Copy link
Author

giubacc commented Oct 19, 2023

@giubacc do I understand you correctly: this is an sfs bug, not on the UI?

Exactly, my guess is that we should intervene on SFSMultipartUploadV2::complete() in the backend.
At the beginning I thought this was an UI issue, but then I did a complete analysis and the problem should be localized in the backend.

@jecluis jecluis added priority/1 Should be fixed for next release and removed triage/waiting Waiting for triage labels Oct 19, 2023
@jecluis jecluis added this to the v0.23.0 milestone Oct 19, 2023
@giubacc giubacc mentioned this issue Oct 23, 2023
Closed
@jecluis
Copy link
Contributor

jecluis commented Oct 25, 2023

do you have a reproducer script for this? something we can validate against?

@giubacc
Copy link
Author

giubacc commented Oct 25, 2023

do you have a reproducer script for this? something we can validate against?

This can be verified issuing an object multipart upload using the S3 APIs:

CreateMultipartUpload
UploadPart
CompleteMultipartUpload

On the contrary, issuing a simple object upload with the S3 API:

PutObject

does not produces the anomaly.

giubacc referenced this issue in giubacc/ceph Nov 22, 2023
rgw/sfs: set default obj-lock retention mode for multipart objects
97bab74 
When an object is uploaded with multipart, for object-lock enabled buckets, the default rentention mode must be set in the object's attributes (when a retention mode is not explicitely set by the user for the object).
Therefore, RGW_ATTR_OBJECT_RETENTION must be set in the attrs of each part being uploaded in the SFSMultipartUploadV2::complete() function.

Fixes: https://github.com/aquarist-labs/s3gw/issues/761
Signed-off-by: Giuseppe Baccini <[email protected]>
giubacc referenced this issue in giubacc/ceph Nov 22, 2023
rgw/sfs: set default obj-lock retention mode for multipart objects
10f1362 
When an object is uploaded with multipart, for object-lock enabled buckets,
the default rentention mode must be set in the object's attributes
(when a retention mode is not explicitely set by the user for the object).
Therefore, RGW_ATTR_OBJECT_RETENTION must be set in the attrs of each part
being uploaded in the SFSMultipartUploadV2::complete() function.

Fixes: https://github.com/aquarist-labs/s3gw/issues/761
Signed-off-by: Giuseppe Baccini <[email protected]>
@giubacc giubacc mentioned this issue Nov 22, 2023
Merged
11 tasks
@giubacc giubacc modified the milestones: v0.23.0, v0.24.0 Nov 23, 2023
@giubacc giubacc moved this from Backlog to In Review 👀 in S3GW Nov 23, 2023
giubacc referenced this issue in giubacc/ceph Nov 23, 2023
rgw/sfs: set default obj-lock retention mode for multipart objects
6bc54b5 
When an object is uploaded with multipart, for object-lock enabled buckets,
the default rentention mode must be set in the object's attributes
(when a retention mode is not explicitely set by the user for the object).
Therefore, RGW_ATTR_OBJECT_RETENTION must be set in the attrs of each part
being uploaded in the SFSMultipartUploadV2::complete() function.

Fixes: https://github.com/aquarist-labs/s3gw/issues/761
Signed-off-by: Giuseppe Baccini <[email protected]>
@github-project-automation github-project-automation bot moved this from In Review 👀 to Done in S3GW Dec 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@giubacc giubacc

Labels
area/rgw-sfs RGW & SFS related kind/bug Something isn't working priority/1 Should be fixed for next release
Projects
None yet
Milestone
v0.24.0
Development

Successfully merging a pull request may close this issue.

Fix default retention mode with multipart uploaded objects giubacc/ceph
3 participants
@jecluis @votdev @giubacc