[APR-278] Add config refresher to update API key #351

rayz · 2024-11-25T17:55:07Z

Context

We would like to receive configuration updates via the datadog-agent config/v1 endpoint.

Solution

This pr creates a new object RefreshableConfiguration that has a task that sends a request to the endpoint every 15 seconds. This can be configured with refresh_interval_seconds.

Notes

The ResolvedEndpoint will have an optional RefreshableConfiguration that they can use to get the api key.

lib/saluki-config/src/refresher.rs

rayz · 2024-11-25T17:58:13Z

lib/saluki-config/src/refresher.rs

+    #[allow(unused)]
+    pub async fn query_agent(&self) -> Result<(), GenericError> {
+        let url = format!("https://localhost:5004/config/v1/");
+        let client = reqwest::ClientBuilder::new()


Using reqwest here since I needed .danger_accept_invalid_certs for the request to work.

The curl command i was trying to imitate was
curl -H "authorization: Bearer $(cat /etc/datadog-agent/auth/auth_token)" -k https://localhost:5004/config/v1/

pr-commenter · 2024-11-25T18:09:54Z

Regression Detector (DogStatsD)

Regression Detector Results

Run ID: b430d5be-2203-49f9-be23-1309b4d7cdf2

Baseline: 7.59.0
Comparison: 7.59.0

Optimization Goals: ✅ No significant changes detected

Fine details of change detection per experiment

perf	experiment	goal	Δ mean %	Δ mean % CI	trials
➖	dsd_uds_512kb_3k_contexts	ingress throughput	+0.00	[-0.01, +0.01]	1
➖	dsd_uds_1mb_50k_contexts_memlimit	ingress throughput	+0.00	[-0.00, +0.00]	1
➖	dsd_uds_100mb_250k_contexts	ingress throughput	+0.00	[-0.00, +0.00]	1
➖	dsd_uds_500mb_3k_contexts	ingress throughput	+0.00	[-0.01, +0.01]	1
➖	dsd_uds_1mb_50k_contexts	ingress throughput	+0.00	[-0.00, +0.00]	1
➖	dsd_uds_10mb_3k_contexts	ingress throughput	-0.00	[-0.00, +0.00]	1
➖	dsd_uds_1mb_3k_contexts	ingress throughput	-0.00	[-0.00, +0.00]	1
➖	dsd_uds_1mb_3k_contexts_dualship	ingress throughput	-0.00	[-0.00, +0.00]	1
➖	dsd_uds_100mb_3k_contexts	ingress throughput	-0.00	[-0.04, +0.04]	1
➖	dsd_uds_100mb_3k_contexts_distributions_only	memory utilization	-0.55	[-0.76, -0.35]	1
➖	quality_gates_idle_rss	memory utilization	-2.48	[-2.60, -2.36]	1

Bounds Checks: ❌ Failed

perf	experiment	bounds_check_name	replicates_passed	links
❌	quality_gates_idle_rss	memory_usage	0/10

Explanation

Confidence level: 90.00%
Effect size tolerance: |Δ mean %| ≥ 5.00%

Performance changes are noted in the perf column of each table:

✅ = significantly better comparison variant performance
❌ = significantly worse comparison variant performance
➖ = no significant change in performance

A regression test is an A/B test of target performance in a repeatable rig, where "performance" is measured as "comparison variant minus baseline variant" for an optimization goal (e.g., ingress throughput). Due to intrinsic variability in measuring that goal, we can only estimate its mean value for each experiment; we report uncertainty in that value as a 90.00% confidence interval denoted "Δ mean % CI".

For each experiment, we decide whether a change in performance is a "regression" -- a change worth investigating further -- if all of the following criteria are true:

Its estimated |Δ mean %| ≥ 5.00%, indicating the change is big enough to merit a closer look.
Its 90.00% confidence interval "Δ mean % CI" does not contain zero, indicating that if our statistical model is accurate, there is at least a 90.00% chance there is a difference in performance between baseline and comparison variants.
Its configuration does not mark it "erratic".

lib/saluki-config/src/refresher.rs

tobz

Left a bunch of comments.

bin/agent-data-plane/Cargo.toml

lib/saluki-components/Cargo.toml

bin/agent-data-plane/src/main.rs

lib/saluki-components/src/destinations/datadog_metrics/mod.rs

lib/saluki-config/src/refresher.rs

lib/saluki-components/src/destinations/datadog_metrics/mod.rs

lib/saluki-components/src/destinations/datadog_metrics/request_builder.rs

lib/saluki-config/src/refresher.rs

lib/saluki-components/src/destinations/datadog_metrics/mod.rs

pr-commenter · 2024-12-03T22:46:38Z

Regression Detector Links

Experiment Result Links

experiment	link(s)
dsd_uds_100mb_250k_contexts	[Profiling (ADP)] [Profiling (DSD)] [SMP Dashboard]
dsd_uds_100mb_3k_contexts	[Profiling (ADP)] [Profiling (DSD)] [SMP Dashboard]
dsd_uds_100mb_3k_contexts_distributions_only	[Profiling (ADP)] [Profiling (DSD)] [SMP Dashboard]
dsd_uds_10mb_3k_contexts	[Profiling (ADP)] [Profiling (DSD)] [SMP Dashboard]
dsd_uds_1mb_3k_contexts	[Profiling (ADP)] [Profiling (DSD)] [SMP Dashboard]
dsd_uds_1mb_3k_contexts_dualship	[Profiling (ADP)] [Profiling (DSD)] [SMP Dashboard]
dsd_uds_1mb_50k_contexts	[Profiling (ADP)] [Profiling (DSD)] [SMP Dashboard]
dsd_uds_1mb_50k_contexts_memlimit	[Profiling (ADP)] [Profiling (DSD)] [SMP Dashboard]
dsd_uds_500mb_3k_contexts	[Profiling (ADP)] [Profiling (DSD)] [SMP Dashboard]
dsd_uds_512kb_3k_contexts	[Profiling (ADP)] [Profiling (DSD)] [SMP Dashboard]
quality_gates_idle_rss	[Profiling (ADP)] [Profiling (DSD)] [SMP Dashboard]
dsd_uds_50mb_10k_contexts_no_inlining (ADP only)	[Profiling (ADP)] [SMP Dashboard]
dsd_uds_50mb_10k_contexts_no_inlining_no_allocs (ADP only)	[Profiling (ADP)] [SMP Dashboard]

lib/saluki-config/Cargo.toml

bin/agent-data-plane/src/main.rs

lib/saluki-components/src/destinations/datadog_metrics/endpoint/endpoints.rs

lib/saluki-components/src/destinations/datadog_metrics/mod.rs

lib/saluki-config/src/refresher.rs

tobz · 2024-12-04T17:07:01Z

lib/saluki-config/src/refresher.rs

+}
+
+fn default_agent_ipc_port() -> u64 {
+    0


Zero still doesn't make sense as a value for something we're connecting to, so we shouldn't specify a default port at all. This would let us bubble up an error when calling RefresherConfiguration::from_configuration if agent_ipc_port isn't set.

Probably have the code building the refreshable config in bin/agent-data-plane/src/main.rs catch the error and report an INFO-level log that dynamic configuration refreshing will be unavailable due to not being configured, etc etc.

tobz · 2024-12-04T17:12:01Z

lib/saluki-config/src/refresher.rs

+    ///
+    /// Defaults to 15 seconds.
+    #[serde(
+        rename = "agent_config_refresh_internal_seconds",


Sorry for the flip-flopping here, but now that I see how we're aligning more with the agent_ipc stuff in the Agent, I think we would actually want this to be agent_ipc_config_refresh_interval, which would align with how it's used by the comp/core/configsync component.

WIP

3559635

github-actions bot added area/config Configuration. area/components Sources, transforms, and destinations. destination/datadog-metrics Datadog Metrics destination. labels Nov 25, 2024

rayz commented Nov 25, 2024

View reviewed changes

lib/saluki-config/src/refresher.rs Outdated Show resolved Hide resolved

rayz commented Nov 25, 2024

View reviewed changes

use const for endpoint

f73b58e

tobz reviewed Nov 25, 2024

View reviewed changes

rayz added 3 commits November 26, 2024 15:32

feedback

2263ed0

update naming

d0ede29

more renaming

7f80b3a

tobz requested changes Nov 26, 2024

View reviewed changes

rayz added 7 commits November 26, 2024 16:27

partial feedback 2

aff7f99

remove query_agent

01a7b0f

wrap Arc around ArcSwap value

9cee999

update doc

3cdaf4d

add derive clone

62558cb

Merge branch 'main' into rayz/config-refresh

f1a1bfd

merge main

9f21c09

tobz requested changes Dec 3, 2024

View reviewed changes

rayz added 3 commits December 3, 2024 13:46

feedback

8fcaa38

Use Optional refresher config on primary endpoint

3bf6a42

attempt fix pipeline

e90bd3f

rayz marked this pull request as ready for review December 4, 2024 15:36

rayz requested review from a team as code owners December 4, 2024 15:36

tobz requested changes Dec 4, 2024

View reviewed changes

nits

98aa1dc

doc

ffffbfa

tobz requested changes Dec 4, 2024

View reviewed changes

rayz added 2 commits December 4, 2024 12:26

remove default agent ipc port

2ddd0a8

spelling

f2694a3

tobz approved these changes Dec 4, 2024

View reviewed changes

rayz merged commit 5ffdc5e into main Dec 4, 2024
17 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[APR-278] Add config refresher to update API key #351

[APR-278] Add config refresher to update API key #351

rayz commented Nov 25, 2024 •

edited

Loading

rayz Nov 25, 2024

pr-commenter bot commented Nov 25, 2024 •

edited

Loading

Fine details of change detection per experiment

Explanation

tobz left a comment

pr-commenter bot commented Dec 3, 2024 •

edited

Loading

tobz Dec 4, 2024

tobz Dec 4, 2024

[APR-278] Add config refresher to update API key #351

[APR-278] Add config refresher to update API key #351

Conversation

rayz commented Nov 25, 2024 • edited Loading

Context

Solution

Notes

rayz Nov 25, 2024

Choose a reason for hiding this comment

pr-commenter bot commented Nov 25, 2024 • edited Loading

Regression Detector (DogStatsD)

Regression Detector Results

Optimization Goals: ✅ No significant changes detected

Fine details of change detection per experiment

Bounds Checks: ❌ Failed

Explanation

tobz left a comment

Choose a reason for hiding this comment

pr-commenter bot commented Dec 3, 2024 • edited Loading

Regression Detector Links

Experiment Result Links

tobz Dec 4, 2024

Choose a reason for hiding this comment

tobz Dec 4, 2024

Choose a reason for hiding this comment

rayz commented Nov 25, 2024 •

edited

Loading

pr-commenter bot commented Nov 25, 2024 •

edited

Loading

pr-commenter bot commented Dec 3, 2024 •

edited

Loading