Ensure run results artifact get written during "after run hooks" #10941
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
5 tasks
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #10941 +/- ##
==========================================
- Coverage 89.19% 89.12% -0.08%
==========================================
Files 183 183
Lines 23496 23508 +12
==========================================
- Hits 20958 20951 -7
- Misses 2538 2557 +19
Flags with carried forward coverage won't be shown. Click here to find out more.
|
Previously in `get_execution_status` if a non `DbtRuntimeError` exception was raised, the finally would be entered, but the `status`/`message` would not be set, and thus a `status not defined` exception would get raised on attempting to return. Tangentially, there is another issue where somehow the `node_status` is becoming `None`. In all my playing with `get_execution_status` I found that trying to return an undefined variable in the `finally` caused an undefined variable exception. However, if in some python version, it instead just handed back `None`, then this fix should also solve that.
QMalcolm
force-pushed
the
qmalcolm--ensure-run-results-artifact-during-after-run-hooks
branch
from
0801f6e to
2929b89
Compare
MichelleArk
reviewed
Oct 30, 2024
| @@ -94,11 +94,16 @@ def get_execution_status(sql: str, adapter: BaseAdapter) -> Tuple[RunStatus, str | |||
| response, _ = adapter.execute(sql, auto_begin=False, fetch=False) | |||
| status = RunStatus.Success | |||
| message = response._message | |||
| except (KeyboardInterrupt, SystemExit): | |||
| raise | |||
There was a problem hiding this comment.
checking my own understanding -- these are eventually handled further along, here: https://github.com/dbt-labs/dbt-core/pull/10941/files#diff-299afe4cac0de7af89561129dd7c55a07bd53c06baafff397b51b460bc631bceR804?
MichelleArk
approved these changes
Oct 30, 2024
QMalcolm
deleted the
qmalcolm--ensure-run-results-artifact-during-after-run-hooks
branch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolves #10934
Problem
Occasionally there would be an unhandled exception during the execution of on run end hooks. Because the exception wasn't being handled, the
run_results.jsonartifact was not being written.Solution
Handle more exception cases during the execution of on run end hooks.
Demo
Fixing no run_results.json
Checklist