Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

don't reprocess duplicate jobs #28

Open
johnsmorgan opened this issue May 28, 2019 · 0 comments
Open

don't reprocess duplicate jobs #28

johnsmorgan opened this issue May 28, 2019 · 0 comments

Comments

@johnsmorgan
Copy link

For ease of pipelining, it would be good if when a duplicate job is resubmitted, if an identical job already exists, mwa_client were to connect to the existing job rather than create a new one.

Use case: I start a set of jobs processing/downloading with mwa_client -c csvfile -d destdir . When those jobs are partway through processing my computer crashes. After weeding out those jobs which are completely downloaded on my machine, I would like to pick up where I left off simply by running mwa_client -c csvfile -d destdir again.

This could be default behaviour or activated by a command line flag.

It would be nice if this action were to reset the clock on the one-week expiry of the job.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant