Convert DogStatsD payload to use string pool #676
Conversation
This commit begins the process of converting the dogstatsd payload generation to use the new string pool, introduced in #675. I have only partially converted the event sub-payload -- note that tagsets have not been switched yet -- and this shows a 6% improvement to `dogstatsd_setup` and a 5% - 74% improvement to `dogstatsd_all`. It shows there's some promise to the technique that improves as we scale up the total-bytes emitted. Of note I will need to eventually convert the `Generator` trait to emit a type with a lifetime. Since I can't do that incrementally the `generate` function in select areas temporarily does not come from the trait. I'll resolve this as a part of the work here. Signed-off-by: Brian L. Troutwine <[email protected]>
Regression Detector ResultsRun ID: 426b95a2-f8ad-486d-afe9-41d04da1a59b ExplanationA regression test is an integrated performance test for Because a target's optimization goal performance in each experiment will vary somewhat each time it is run, we can only estimate mean differences in optimization goal relative to the baseline target. We express these differences as a percentage change relative to the baseline target, denoted "Δ mean %". These estimates are made to a precision that balances accuracy and cost control. We represent this precision as a 90.00% confidence interval denoted "Δ mean % CI": there is a 90.00% chance that the true value of "Δ mean %" is in that interval. We decide whether a change in performance is a "regression" -- a change worth investigating further -- if both of the following two criteria are true:
The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values of "Δ mean %" mean that baseline is faster, whereas positive values of "Δ mean %" mean that comparison is faster. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed. No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%. Fine details of change detection per experiment.
|
Signed-off-by: Brian L. Troutwine <[email protected]>
Regression Detector ResultsRun ID: 2a312524-b510-4918-a5d9-5a88bb2f35cb ExplanationA regression test is an integrated performance test for Because a target's optimization goal performance in each experiment will vary somewhat each time it is run, we can only estimate mean differences in optimization goal relative to the baseline target. We express these differences as a percentage change relative to the baseline target, denoted "Δ mean %". These estimates are made to a precision that balances accuracy and cost control. We represent this precision as a 90.00% confidence interval denoted "Δ mean % CI": there is a 90.00% chance that the true value of "Δ mean %" is in that interval. We decide whether a change in performance is a "regression" -- a change worth investigating further -- if both of the following two criteria are true:
The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values of "Δ mean %" mean that baseline is faster, whereas positive values of "Δ mean %" mean that comparison is faster. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed. No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%. Fine details of change detection per experiment.
|
This commit improves the string pool spread in dogstatsd. I still have yet to adjust the metric generation to avoid cloning -- I'll do that in the next commit -- but setup is improved by 34% and _all from 52% to 5%. I suspect that if I can get the generation side to note clone we'll improve on the high-end. Signed-off-by: Brian L. Troutwine <[email protected]>
Regression Detector ResultsRun ID: d3d83036-05c6-46b2-9536-c74cc6b80051 ExplanationA regression test is an integrated performance test for Because a target's optimization goal performance in each experiment will vary somewhat each time it is run, we can only estimate mean differences in optimization goal relative to the baseline target. We express these differences as a percentage change relative to the baseline target, denoted "Δ mean %". These estimates are made to a precision that balances accuracy and cost control. We represent this precision as a 90.00% confidence interval denoted "Δ mean % CI": there is a 90.00% chance that the true value of "Δ mean %" is in that interval. We decide whether a change in performance is a "regression" -- a change worth investigating further -- if both of the following two criteria are true:
The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values of "Δ mean %" mean that baseline is faster, whereas positive values of "Δ mean %" mean that comparison is faster. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed. No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%. Fine details of change detection per experiment.
|
This commit makes an explicit `Template` that is used to generate a full `Metric<'a>`. Note the new lifetime, we avoid cloning so much now. At the top end this hits 1Gb/s in the _all benchmark. Signed-off-by: Brian L. Troutwine <[email protected]>
After instrumentation it appears that lading_rev spends 85% of its time in `fmt::write`, implying that if we want to go faster we'll need to make coercion into strings cheaper. Excellent result. Signed-off-by: Brian L. Troutwine <[email protected]>
Regression Detector ResultsRun ID: c28cf8d5-c672-4a52-ad53-4138aa5ec9e9 ExplanationA regression test is an integrated performance test for Because a target's optimization goal performance in each experiment will vary somewhat each time it is run, we can only estimate mean differences in optimization goal relative to the baseline target. We express these differences as a percentage change relative to the baseline target, denoted "Δ mean %". These estimates are made to a precision that balances accuracy and cost control. We represent this precision as a 90.00% confidence interval denoted "Δ mean % CI": there is a 90.00% chance that the true value of "Δ mean %" is in that interval. We decide whether a change in performance is a "regression" -- a change worth investigating further -- if both of the following two criteria are true:
The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values of "Δ mean %" mean that baseline is faster, whereas positive values of "Δ mean %" mean that comparison is faster. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed. No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%. Fine details of change detection per experiment.
|
What does this PR do?
This commit begins the process of converting the dogstatsd payload generation to use the new string pool, introduced in #675. I have only partially converted the event sub-payload -- note that tagsets have not been switched yet -- and this shows a 6% improvement to
dogstatsd_setupand a 5% - 74% improvement todogstatsd_all. It shows there's some promise to the technique that improves as we scale up the total-bytes emitted.EDIT: By the final commit we're capping _all out at 1Gb/s, up from ~300Mb/s.
Of note I will need to eventually convert the
Generatortrait to emit a type with a lifetime. Since I can't do that incrementally thegeneratefunction in select areas temporarily does not come from the trait. This will have to be resolved by converting all the payloads to non-copying implementations, which will take some time and is outside of the scope of this PR.Related issues
REF SMP-664