Block cache with an infinite stream option #687
Conversation
This commit makes two major changes to the block cache. The most important is that the block cache's major CPU consumption -- the construction of `Block` instances -- is now done in a separate OS thread from the tokio runtime. This allows us to introduce the second more important change: infinite streams of `Blocks`. It is now possible for users to construct an unending stream of `Block` instances that do not loop. We maintain a cache of constructed `Block`s up to the maximum total bytes allow to minimize any potential latency impact. Configuration is changed but not in a backward incompatible way. REF SMP-664 Signed-off-by: Brian L. Troutwine <[email protected]>
Regression Detector ResultsRun ID: 14811a98-a780-4047-b96f-8758c61625ec ExplanationA regression test is an integrated performance test for Because a target's optimization goal performance in each experiment will vary somewhat each time it is run, we can only estimate mean differences in optimization goal relative to the baseline target. We express these differences as a percentage change relative to the baseline target, denoted "Δ mean %". These estimates are made to a precision that balances accuracy and cost control. We represent this precision as a 90.00% confidence interval denoted "Δ mean % CI": there is a 90.00% chance that the true value of "Δ mean %" is in that interval. We decide whether a change in performance is a "regression" -- a change worth investigating further -- if both of the following two criteria are true:
The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values of "Δ mean %" mean that baseline is faster, whereas positive values of "Δ mean %" mean that comparison is faster. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed. No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%. Fine details of change detection per experiment.
|
I suspect the period of the PRNG used would affect how long it would take before Blocks begin to repeat, though for this application, it seems unlikely we'd exhaust that period unless it were quite small (e.g., for a period of 2^64 * 4 bytes, generating 10^9 bytes/second, around 334 years; I'm guessing that the 1,024 channels aren't generating 10^9 bytes/second each). |
Ah yeah, maybe better language would be something like "do not repeat from a finite pool of |
| payload::Config::TraceAgent(enc) => { | ||
| let ta = match enc { | ||
| payload::Encoding::Json => payload::TraceAgent::json(&mut rng), | ||
| payload::Encoding::MsgPack => payload::TraceAgent::msg_pack(&mut rng), | ||
| }; |
There was a problem hiding this comment.
Could this be refactored so the configs are handled inside the relevant payload implementation? (Is there a future benefit to having access to the payload configs here?)
There was a problem hiding this comment.
I think it could be and I started in on that but I felt like it was too big of a change for this PR. I find it awkward, on the subject, that the generator creates the Cache at all but I do find it desirable to maintain backward compatibility with our configs.
| /// Whether to use a fixed or streaming block cache | ||
| #[serde(default = "crate::block::default_cache_method")] | ||
| pub block_cache_method: block::CacheMethod, |
There was a problem hiding this comment.
This kind of seems like it should be set automatically by the payload, would that be possible?
There was a problem hiding this comment.
I think it could be but I hadn't considered that. Right now it's the Cache that maintains the difference and the payload is really just a mechanism -- to the Cache -- of how to get a Block. My sense was that we would drive to where streaming was on by default and leave fixed as an option for setups that are especially CPU conscious, so users would rarely see this setting.
I'm not sure we have enough information to say whether it's appropriate to stream or be fixed automatically, based on payload settings. Maybe I'm missing something you're seeing?
There was a problem hiding this comment.
Hmm, I see. I buy that. We'll have some mismatch if we introduce any streaming-only payloads. We can handle that when we need to though.
There was a problem hiding this comment.
To my mind the payloads should be unaware of how the generate chooses to use them. That's been our approach so far. I don't see that you can make a streaming only payload, or at least I cannot image one.
Signed-off-by: Brian L. Troutwine <[email protected]>
Regression Detector ResultsRun ID: 2e4c86a8-2f60-4380-a95e-897bbcf3967a ExplanationA regression test is an integrated performance test for Because a target's optimization goal performance in each experiment will vary somewhat each time it is run, we can only estimate mean differences in optimization goal relative to the baseline target. We express these differences as a percentage change relative to the baseline target, denoted "Δ mean %". These estimates are made to a precision that balances accuracy and cost control. We represent this precision as a 90.00% confidence interval denoted "Δ mean % CI": there is a 90.00% chance that the true value of "Δ mean %" is in that interval. We decide whether a change in performance is a "regression" -- a change worth investigating further -- if both of the following two criteria are true:
The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values of "Δ mean %" mean that baseline is faster, whereas positive values of "Δ mean %" mean that comparison is faster. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed. No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%. Fine details of change detection per experiment.
|
What does this PR do?
This commit makes two major changes to the block cache. The most important is that the block cache's major CPU consumption -- the construction of
Blockinstances -- is now done in a separate OS thread from the tokio runtime. This allows us to introduce the second more important change: infinite streams ofBlocks. It is now possible for users to construct an unending stream ofBlockinstances that do not loop. We maintain a cache of constructedBlocks up to the maximum total bytes allow to minimize any potential latency impact.Additional Notes
Configuration is changed but not in a backward incompatible way.
Related issues
REF SMP-664