Skip to content

Commit

Permalink
Attempt to skip saved query processing when no semantic manifest changes
Browse files Browse the repository at this point in the history
  • Loading branch information
gshank committed Sep 26, 2024
1 parent ac66f91 commit ff76596
Show file tree
Hide file tree
Showing 2 changed files with 24 additions and 0 deletions.
6 changes: 6 additions & 0 deletions .changes/unreleased/Fixes-20240926-101220.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
kind: Fixes
body: Attempt to skip saved query processing when no semantic manifest changes
time: 2024-09-26T10:12:20.193453-04:00
custom:
Author: gshank
Issue: "10563"
18 changes: 18 additions & 0 deletions core/dbt/parser/manifest.py
Original file line number Diff line number Diff line change
Expand Up @@ -1141,6 +1141,24 @@ def process_metrics(self, config: RuntimeConfig):

def process_saved_queries(self, config: RuntimeConfig):
"""Processes SavedQuery nodes to populate their `depends_on`."""
# Note: This will also capture various nodes which have been re-parsed
# because they refer to some other changed node, so there will be
# false positives. Ideally we would compare actual changes.
semantic_manifest_changed = False
SemanticManifestNode = Union[SavedQuery, SemanticModel, Metric]
semantic_manifest_nodes: chain[SemanticManifestNode] = chain(
self.manifest.saved_queries.values(),
self.manifest.semantic_models.values(),
self.manifest.metrics.values(),
)
for node in semantic_manifest_nodes:
# Check if this node has been modified in this parsing run
if node.created_at > self.started_at:
semantic_manifest_changed = True
break # as soon as we run into one changed node we can stop

Check warning on line 1158 in core/dbt/parser/manifest.py

View check run for this annotation

Codecov / codecov/patch

core/dbt/parser/manifest.py#L1156-L1158

Added lines #L1156 - L1158 were not covered by tests
if semantic_manifest_changed is False:
return

current_project = config.project_name
for saved_query in self.manifest.saved_queries.values():
# TODO:
Expand Down

0 comments on commit ff76596

Please sign in to comment.