Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add @atomic_output decorator for outputs that get written to in blocks #157

Merged
merged 3 commits into from
Nov 7, 2023

Conversation

scottstanie
Copy link
Collaborator

@scottstanie scottstanie commented Nov 2, 2023

Used for long running parts where, if a process is interrupted, currently there would be a half-empty file with the expected output name. This replaces the final output name with a tempfile, then swaps it only after the process is complete.

closes #110

@scottstanie scottstanie added the work in progress WIP: Don't merge yet label Nov 2, 2023
@scottstanie scottstanie changed the title add decorator to swap a tempfile which gets written to in batches Add @atomic_output decorator for outputs that get written to in block Nov 2, 2023
@scottstanie scottstanie changed the title Add @atomic_output decorator for outputs that get written to in block Add @atomic_output decorator for outputs that get written to in blocks Nov 2, 2023
@scottstanie scottstanie removed the work in progress WIP: Don't merge yet label Nov 6, 2023
@scottstanie scottstanie merged commit 39a657a into isce-framework:main Nov 7, 2023
4 checks passed
@scottstanie scottstanie deleted the tempfile-decorator branch November 7, 2023 00:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Set up intermediate files with a tempdir, only transfer to file name after completion
1 participant