-
Notifications
You must be signed in to change notification settings - Fork 91
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Added a required 'name' argument to 'collect_metrics'. #736
Added a required 'name' argument to 'collect_metrics'. #736
Conversation
👋 @elongl |
e2e4310
to
8572db5
Compare
55f0585
to
4757a6c
Compare
6bcd126
to
0449107
Compare
macros/edr/data_monitoring/monitors_query/table_monitoring_query.sql
Outdated
Show resolved
Hide resolved
macros/edr/data_monitoring/monitors_query/table_monitoring_query.sql
Outdated
Show resolved
Hide resolved
{% for metric in column_metrics %} | ||
{% do metric_types.append(metric.type) %} | ||
{% do metric_name_to_type.update({metric.name: metric.type}) %} | ||
{% endfor %} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure this is the right place, but can we also save metric_type
in data_monitoring_metrics
?
(not sure what's the right place for this comment)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So my intuition was to do that as well,
and Ofek convinced me that it shouldn't matter how a metric is calculated for the storage of the metric. Regarding which algorithm to use in the cloud, we can view the type in dbt_tests
at the configuration. WDYT?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm the only case where I think it might matter is when the metric definition is changed - e.g. the name remains the same but the metric type is changed.
But that makes me think that it should actually be added to metric_properties
. WDYT?
This achieves two things:
- The metric will be backfilled if the type changes (same as any other configuration change like
timestamp_column
) - In the cloud I implemented logic that only pulls metrics with the most recent metric properties (to ensure we only rely on metrics collected with the most recent configuration - a bit like what happens in
get_anomaly_scores_query
).
0603fc8
to
2fa0bfa
Compare
2fa0bfa
to
c8ae4ad
Compare
12a7cc9
to
8dc050e
Compare
6a232f5
to
e997fc3
Compare
e997fc3
to
f9a2f8f
Compare
No description provided.