Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better checks to ensure test data files exist for tables referenced in SQL integration tests #5184

Open
scholtzan opened this issue Mar 7, 2024 · 1 comment

Comments

@scholtzan
Copy link
Collaborator

scholtzan commented Mar 7, 2024

For tests defined in https://github.com/mozilla/bigquery-etl/tree/main/tests/sql referenced tables need a schema.json file which will get deployed to the integration test project. Test data is then inserted into these tables and the queries are run against these test tables.

The schema.json files have to exactly match the qualifiers in the SQL query code, otherwise these queries try to query tables in the production environment. This is however currently only caught on main since referenced tables in PRs get deployed to the very same staging environment. So the tests will still succeed.

We should add some additional checks to make sure that any table that is referenced in the query has a matching schema.json file defined.

┆Issue is synchronized with this Jira Task

@scholtzan
Copy link
Collaborator Author

cc @kwindau

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant