Scrub secret vars #9733
Merged
Scrub secret vars #9733
+122
−5
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #9733 +/- ##
==========================================
+ Coverage 88.09% 88.11% +0.01%
==========================================
Files 180 180
Lines 22546 22557 +11
==========================================
+ Hits 19863 19877 +14
+ Misses 2683 2680 -3
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
nielspardon
force-pushed
the
par-scrub-args
branch
2 times, most recently
from
bea03f4 to
638fbdb
Compare
nielspardon
force-pushed
the
par-scrub-args
branch
from
638fbdb to
a12e752
Compare
|
@dbeatty10 any reviewers available for this PR? |
dbeatty10
added
the
ready_for_review
dbeatty10
reviewed
Apr 11, 2024
There was a problem hiding this comment.
@nielspardon I looked at this along with @ChenyuLInx
The main question we have is regarding the test coverage. Could you take a look?
dbeatty10
reviewed
Apr 12, 2024
nielspardon
force-pushed
the
par-scrub-args
branch
3 times, most recently
from
e3fb9c2 to
d3b3336
Compare
nielspardon
commented
Apr 12, 2024
| @@ -108,7 +108,7 @@ def _new_file(self, searched, name, match): | |||
| class TestPartialParse(unittest.TestCase): | |||
| def setUp(self) -> None: | |||
| mock_project = MagicMock(RuntimeConfig) | |||
| mock_project.cli_vars = "" | |||
| mock_project.cli_vars = {} | |||
There was a problem hiding this comment.
@ChenyuLInx looks like you accidentally initialized this to a str but this should be a dict. I fixed it since it broke the tests with my changes.
nielspardon
force-pushed
the
par-scrub-args
branch
2 times, most recently
from
63cb9c0 to
e687342
Compare
- Scrub secret vars in RequiredVarNotFoundError - Scrub secret vars in StateCheckVarsHash event - Scrub secret vars in run results
nielspardon
force-pushed
the
par-scrub-args
branch
from
e687342 to
457fd68
Compare
ChenyuLInx
approved these changes
Apr 16, 2024
| } | ||
|
|
||
| def test__run_results_scrubbing(self, project): | ||
| results = run_dbt(["run", "--vars", "{DBT_ENV_SECRET_simple: abc, unused: def}"]) |
There was a problem hiding this comment.
Why we do both run and test at the same time?
ChenyuLInx
added
the
artifact_minor_upgrade
2 tasks
4 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
resolves #7247
Problem
Currently, dbt only supports secret env variables and only scrubs the values of those secret env variables from log messages. Secret variables being provided via the
--varsflag are not being scrubbed and the plaintext values appear in a couple of places.Solution
With this PR secret variables being provided via the
--varsflag are also scrubbed just like env variables reusing the same prefix as for env variables.Checklist