File upload job reset on 401

[jrchudy]

The following conversation occurred in slack on Tuesday, August 13, 2019:

cris williams [1:27 PM]
@rob this file size (almost 8GB) is still a lot for hatrac to chew on in the browser. Been uploading for a while and looks like it's stuck at 82% - if I cancel and try again, it would pick up where it left off, right?

karlcz [1:28 PM]
@cris no I think you'll lose your progress if you abort
it looks to me like the facebase web server may be in a stuck state again. let me try to kick it to see if your upload will internally retry and complete after that

cris williams [1:29 PM]
ok, thanks!

karlcz [1:31 PM]
service is restarted. but your upload may be stalled for a while since it goes into a retry loop with increasing delay times after each successive failure...

cris williams [1:32 PM]
it did get booted out - had to re-login. but i'll just do it again : )

karlcz [1:32 PM]
oh well

cris williams [1:32 PM]
it's how it goes! but thanks for trying : )

Rob Schuler [1:52 PM]
@cris did it succeed the next time around? if not we can try the hatrac CLI.

cris williams [1:55 PM]
it's still chugging (at 15% now) but if it fails then yeah, we'll go with cli

Rob Schuler [2:02 PM]
ok. 8GB is probably a bit of a stretch for the usual browser upload use case.

karlcz [2:04 PM]
I actually think chaise should perform better than the CLI! it does more concurrent work than the deriva-py client code.
@cris what kind of internet connection do you have?
it looks to me like your transfer may be running along at only about 10 Mb/s and I wonder if that is just the kind of network you have or if it indicates some bigger problem...

Carl Kesselman [2:12 PM]
I really think we are being silly trying to move around multi-gigabyte files using a web browser.

karlcz [2:22 PM]
I think it's mostly the slow WAN link that is making this impractical... We've conveniently used the chaise upload on pretty large ad hoc files with other projects, via university workstations on fast networks. But the CLI would be certainly better for this scenario, to restart after an interrupted transfer...

Mike D'Arcy [2:26 PM]
unfortunately, the hatrac-cli can't do restarts, and while deriva-upload-cli can, it doesn't have a simple "put this single file to hatrac and that's it" operating mode
i.e., you can just drive a single file upload from the command line - it still needs a local or server originated config file, and a proper directory layout for input (edited)

karlcz [2:46 PM]
I also just confirmed with Hongsuda that recordedit is supposed to support restart in this case. So, if Cris encounters another error she will try to exercise that feature. I don't know if there is a bug or just a UX/discoverability issue. (The restart is supported w/in the recordedit app instance by having a dismissable error dialog that re-arms the "submit" button to attempt to continue without disposing of the in-progress app state.)

hongsuda [2:59 PM]
This restart is gone if the page is refreshed..

cris williams [3:35 PM]
it completed this time (through the browser) : )

Investigate Recordedit behavior when a 401 occurs because of a http freeze and the httpd service is restarted. Recordedit should not force the page to reload. It should allow the user to continue where they left off.

[jrchudy]

issue #2379 further improves this functionality to resume where a file left off. If 2379 can be done, that would fix the issue of upload resetting but that doesn't address the other problem this issue aims to resolve, not reloading the page when the same user logs back in.