You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
As a curator
I want to know the files differences between Wasabi and S3 Glacier
So that I know when there is a discrepancy between the two and can take appropriate action
Acceptance criteria
Given I have backed up files belonging to a dataset into AWS S3 Glacier bucket after they have been uploaded to Wasabi
When I continue to work on files
And I run a check tool
I am informed when a file has not been backed up yet
Additional Info
Add a boolean has_backup column to the file table.
when the transfer command is run with the --backup option (either manually or as part of automated backup), it will check for each file whether the transfer was successful and if so update the has_backup column for the corresponding file (using filesMetaDb new functionality below)
pli888
changed the title
Enable curator to list contents of directories in S3 Glacier backup bucket
Enable curator to list contents of directories in AWS S3 Glacier backup bucket
Nov 7, 2024
rija
changed the title
Enable curator to list contents of directories in AWS S3 Glacier backup bucket
Provide curator with tooling to check whether there are files not backup to S3 glacier
Nov 19, 2024
The DOI.md5 and DOI.filesizes files are not added to the database so what will happen with those?
Do we need to start adding them to the database?
What happens to other random files that make it into the Wasabi server somehow that are not listed in GigaDB, do they just get ignored? These should be flagged for investigation as to why they are there!
User story
Acceptance criteria
Additional Info
has_backup
column to thefile
table.transfer
command is run with the--backup
option (either manually or as part of automated backup), it will check for each file whether the transfer was successful and if so update thehas_backup
column for the corresponding file (usingfilesMetaDb
new functionality below)filesMetaToDb --doi 102484 --mark-as-backed-up <filepath>
filesMetaToDb --check
that will check for files that are not backed upThe text was updated successfully, but these errors were encountered: