Minor Revisions to Uploads before Reupload SOP is Released

[wa6gz]

Multiple sites raised a question at today's DA office hours about the ability to upload in pieces and to make corrections. After talking with @jshoughtaling and Heidi, it was unclear whether files should be uploaded on top of the existing upload to supplement or replace files. The two main use cases would be 1) uploading by modality as it's available (for which files would not conflict) and 2) to make corrections to known issues (for which files would be replaced). Technically, the Microsoft team confirmed that multiple uploads are possible to the same directory, replacing conflicting files if the modification time is newer. I had thought we certainly weren't allowing 2), and we also weren't recommending 1), but I wanted to open it up for comments before prescribing one method or another.

@clermontg, @del42, adding for your thoughts. Please add anyone else who has an interest in the structure of uploads.

[del42]

Hey @wa6gz very timely question! Tony Pan is assigned to lead this cloud update strategy effort and he should be starting the workgroup meetings next week. More nuances has arisen when we talked about opening the data to downstream use cases as it is being updated in same location.
Right now, everything will overwrite even if is not new file with same name.

[clermontg]

Will, Del, Tony This is a very healthy discussion and addressing real-world issues. Too bad I was sitting at this study section. Thank you Will for laying out two common use cases. Are there others? Simply means we really need to move quickly on this. Maybe write down what we learned from Alex Ruiz and make sure Microsoft reviews the part of the SOP that addresses whether files are overwritten, versioned, etc. I have identified project management help to draft a revised SOP if we can provide the pieces that such and SOP will include. I do not believe this should be a broad consultative process because of the pressing need. I will also post something on github, but do not have access now Gilles From: wa6gz ***@***.***> Sent: Friday, March 15, 2024 4:05 PM To: chorus-ai/data_acq_SOP ***@***.***> Cc: Clermont, Gilles ***@***.***>; Mention ***@***.***> Subject: [chorus-ai/data_acq_SOP] Minor Revisions to Uploads before Reupload SOP is Released (Discussion #19) Multiple sites raised a question at today's DA office hours about the ability to upload in pieces and to make corrections. After talking with @jshoughtaling<https://github.com/jshoughtaling> and Heidi, it was unclear whether files should be uploaded on top of the existing upload to supplement or replace files. The two main use cases would be 1) uploading by modality as it's available (for which files would not conflict) and 2) to make corrections to known issues (for which files would be replaced). Technically, the Microsoft team confirmed that multiple uploads are possible to the same directory, replacing conflicting files if the modification time is newer. I had thought we certainly weren't allowing 2), and we also weren't recommending 1), but I wanted to open it up for comments before prescribing one method or another. @clermontg<https://github.com/clermontg>, @del42<https://github.com/del42>, adding for your thoughts. Please add anyone else who has an interest in the structure of uploads. - Reply to this email directly, view it on GitHub<#19>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AL33RKVXHTQ3522WVYKLI3TYYNH7XAVCNFSM6AAAAABEYSW32WVHI2DSMVQWIX3LMV43ERDJONRXK43TNFXW4OZWGM3TSNZTG4>. You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>>