-
Notifications
You must be signed in to change notification settings - Fork 1.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Handle unknown type_code
for model contracts
#8887
Merged
Merged
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
7b8aa2d
Handle unknown `type_code` for model contracts
dbeatty10 2d6d923
Changelog entry
dbeatty10 a205d4a
Fix changelog entry
dbeatty10 dc70ff3
Functional test for a `type_code` that is not recognized by psycopg2
dbeatty10 4c8a3b0
Functional tests for data type mismatches
dbeatty10 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
kind: Fixes | ||
body: Handle unknown `type_code` for model contracts | ||
time: 2023-10-24T11:01:51.980781-06:00 | ||
custom: | ||
Author: dbeatty10 | ||
Issue: 8877 8353 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,76 @@ | ||
import pytest | ||
from dbt.tests.util import run_dbt, run_dbt_and_capture | ||
|
||
|
||
my_numeric_model_sql = """ | ||
select | ||
12.34 as price | ||
""" | ||
|
||
my_money_model_sql = """ | ||
select | ||
cast('12.34' as money) as price | ||
""" | ||
|
||
model_schema_money_yml = """ | ||
models: | ||
- name: my_model | ||
config: | ||
contract: | ||
enforced: true | ||
columns: | ||
- name: price | ||
data_type: money | ||
""" | ||
|
||
model_schema_numeric_yml = """ | ||
models: | ||
- name: my_model | ||
config: | ||
contract: | ||
enforced: true | ||
columns: | ||
- name: price | ||
data_type: numeric | ||
""" | ||
|
||
|
||
class TestModelContractUnrecognizedTypeCode1: | ||
@pytest.fixture(scope="class") | ||
def models(self): | ||
return { | ||
"my_model.sql": my_money_model_sql, | ||
"schema.yml": model_schema_money_yml, | ||
} | ||
|
||
def test_nonstandard_data_type(self, project): | ||
run_dbt(["run"], expect_pass=True) | ||
|
||
|
||
class TestModelContractUnrecognizedTypeCodeActualMismatch: | ||
@pytest.fixture(scope="class") | ||
def models(self): | ||
return { | ||
"my_model.sql": my_money_model_sql, | ||
"schema.yml": model_schema_numeric_yml, | ||
} | ||
|
||
def test_nonstandard_data_type(self, project): | ||
expected_msg = "unknown type_code 790 | DECIMAL | data type mismatch" | ||
_, logs = run_dbt_and_capture(["run"], expect_pass=False) | ||
assert expected_msg in logs | ||
|
||
|
||
class TestModelContractUnrecognizedTypeCodeExpectedMismatch: | ||
@pytest.fixture(scope="class") | ||
def models(self): | ||
return { | ||
"my_model.sql": my_numeric_model_sql, | ||
"schema.yml": model_schema_money_yml, | ||
} | ||
|
||
def test_nonstandard_data_type(self, project): | ||
expected_msg = "DECIMAL | unknown type_code 790 | data type mismatch" | ||
_, logs = run_dbt_and_capture(["run"], expect_pass=False) | ||
print(logs) | ||
assert expected_msg in logs |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@rlh1994 brought up the idea of raising a debug-level
warninglog message in this case.Seems like a valuable idea to explore. Maybe something like this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@MichelleArk how would you feel about adding this ☝️ debug output when the
type_code
isn't found?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It may be a little redundant given
unknown type_code
will appear in the error message either way. So I'd either omit it or write a more informative message like"type_code not found in adapter-provided lookup, defaulting to 'unknown type_code'"And we may actually need to use fire_event to fire a debuglevel event as opposed not logger.debug
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍 Splitting this into its own issue:
That way we can ship this fix while still giving that debug message enough attention.