Should vcp to arc resume interrupted file transfers?

[SharonGoliath]

When transferring larger files (e.g. 9GB) to arc, if the transfer fails, the file just gets deleted. Should the client be capable of resuming an interrupted transfer?

[pdowler]

yes, we do need to support a resumable download in arc (cavern code). There are two options:

1. implement the put-transaction API from minoc -- mainly enable cavern to use most of the existing code on the server side and enable vcp to use the client side code (which will be necessary as we move vault to use SI as the storage layer)
2. don't delete the file at the end of a failed PUT and define a simpler mechanism (simpler than 1) to allow the client to append to rather than overwrite a file -- at a minimum, not deleting would be good for other things like streaming output to arc and being able to see where it got to before failing.

Option 2 obviously is going to be simple and lack a lot of integrity checking and a resume of that upload would have to assume that any bytes there (as reported in the http content-length header) are correct and a byte offset is all that's needed to resume.

The best solution probably has aspects of both: don't always delete after a fail, allow for optional simple append, and provide high integrity option 1. Other transport protocols could also improve user experience in the face of these kinds of failures (the options here are rather http-specific).