-
Notifications
You must be signed in to change notification settings - Fork 101
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Bucket Lock
immutable objects support for AWS S3
#818
base: master
Are you sure you want to change the base?
Conversation
@ishan16696 You need rebase this pull request with latest master branch. Please check. |
d392c48
to
7921d0f
Compare
7921d0f
to
38c36a7
Compare
38c36a7
to
d3fe9b5
Compare
d3fe9b5
to
74a6c2d
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ishan16696 thanks for the well-written PR and the docs. I have a few nits. PTAL, thanks.
|
||
if *objectConfig.ObjectLockConfiguration.ObjectLockEnabled == "Enabled" { | ||
// assumption: retention period of bucket will always be in days, not years. | ||
return true, objectConfig.ObjectLockConfiguration.Rule.DefaultRetention.Days, nil |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Each of the fields in objectConfig.ObjectLockConfiguration.Rule.DefaultRetention.Days
is of pointer type, so again possibility of nil pointer dereference.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
here, it won't be an issue as caller for this func is checking the nil pointer reference.
if bucketImmutableExpiryTimeInDays != nil {
....
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
bucketImmutableExpiryTimeInDays
is deduced from GetBucketImmutabilityTime()
, which in turn returns objectConfig.ObjectLockConfiguration.Rule.DefaultRetention.Days
. Simply checking bucketImmutableExpiryTimeInDays
will not help, since you could have Rule
an non-nil, but DefaultRetention
could be nil, and throw an exception. That's the reason the entire path should be checked here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok, got your point. I will make the change.
snap.ImmutabilityExpiryTime = snap.CreatedOn.Add(time.Duration(*bucketImmutableExpiryTimeInDays) * 24 * time.Hour) | ||
} else { | ||
// retry to get bucketImmutableExpiryTimeInDays | ||
_, bucketImmutableExpiryTimeInDays, err = GetBucketImmutabilityTime(s) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How is bucketImmutableExpiryTimeInDays
being used here? The snapshot gets added to the snapList
without setting snap.ImmutabilityExpiryTime
, which doesn't seem right. Can you please confirm?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, you are correct. Actually I added this to handle the case when previous api call failed to get bucketImmutableExpiryTimeInDays
.
Another way to handle this is to add a retry mechanism in first api call, and remove this else handling part , wdyt ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, retry mechanism seems more intuitive here.
pkg/snapstore/s3_snapstore_test.go
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you please add some tests specific to handling immutable backups for S3 buckets? Like garbage collection for instance.
…l make snapshots immutable. Adjusted the Restoration and GC functionality to handle immutable snapshots for S3 object store.
defec1e
to
42fdeb2
Compare
I have rebased the PR on master. |
What this PR does / why we need it:
This PR enables backup-restore to handle AWS S3's object lock mechanism which will make snapshots taken by backup-restore immutable. It also adjusted the Restoration and garbage collector functionality to handle immutable snapshots for S3 object store.
Which issue(s) this PR fixes:
Fixes #777
Special notes for your reviewer:
These case I have tested:
For Restoration functionality
For Garbage collection of snapshots:
Release note: