Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement relation filtering on get_catalog macro #1063

Merged
merged 6 commits into from
Feb 14, 2024
Merged

Implement relation filtering on get_catalog macro #1063

merged 6 commits into from
Feb 14, 2024

Conversation

mikealfare
Copy link
Contributor

@mikealfare mikealfare commented Dec 20, 2023 •
edited
Loading

resolves #950

Problem

When get_catalog runs, it needs to return all relations within a schema, hence it cannot be parallelized or scale. It also means that all relations will be returned, not just those managed by dbt.

Solution

Allow for a set of relations to be passed in to limit the query against the database.

Checklist

  • I have read the contributing guide and understand what's expected of me
  • I have run this code in development and it appears to resolve the stated issue
  • This PR includes tests, or tests are not required/relevant for this PR
  • This PR has no interface changes (e.g. macros, cli, logs, json artifacts, config files, adapter interface, etc) or this PR has already received feedback and approval from Product or DX

@mikealfare mikealfare self-assigned this Dec 20, 2023
@cla-bot cla-bot bot added the cla:yes label Dec 20, 2023
@mikealfare mikealfare marked this pull request as ready for review December 20, 2023 05:04
@mikealfare mikealfare requested a review from a team as a code owner December 20, 2023 05:04
@mikealfare mikealfare mentioned this pull request Jan 17, 2024
Merged
4 tasks
@mikealfare
Copy link
Contributor Author

Removing backport 1.7.latest as this has actually been merged into 1.7.latest already.

@mikealfare
Copy link
Contributor Author

This PR reflects the changes that were merged directly into 1.7.latest via #1073.

@mikealfare
update from main
f3d5e87
@mikealfare
Merge branch 'main' into applied-state/get-catalog-by-object
b49d323 
# Conflicts:
#	dbt/adapters/bigquery/impl.py
#	dbt/adapters/bigquery/relation_configs/_partition.py
@mikealfare mikealfare merged commit 62a8f80 into main Feb 14, 2024
15 checks passed
@mikealfare mikealfare deleted the applied-state/get-catalog-by-object branch February 14, 2024 20:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@colin-rogers-dbt colin-rogers-dbt colin-rogers-dbt approved these changes

Assignees

@mikealfare mikealfare

Labels
cla:yes
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[ADAP-928] [Feature] Implement relation filtering on get_catalog macro
2 participants
@mikealfare @colin-rogers-dbt