Display Standard Deviation Estimate Along with Confidence Intervals in Terminal Output?

[sjsonucool]

First, thank you for your incredible work on Criterion.rs. It has become an indispensable tool in the Rust community for benchmarking and performance analysis. I appreciate the statistical rigor that Criterion.rs brings to the table, particularly with its detailed confidence intervals for metrics like mean and standard deviation.

When I run benchmarks using cargo bench --features benchmark -- --verbose, it provides detailed statistics on metrics like time, mean, median, standard deviation, and median absolute deviation, as shown below:

Benchmarking Polynomial Interpolation Benchmarking Polynomial Interpolation: Warming up for 3.0000 s Benchmarking Polynomial Interpolation: Collecting 100 samples in estimated 5.2450 s (1200 iterations) Benchmarking Polynomial Interpolation: Analyzing Polynomial Interpolation time: [4.4096 ms 4.4284 ms 4.4494 ms] Found 11 outliers among 100 measurements (11.00%) 6 (6.00%) high mild 5 (5.00%) high severe mean [4.4096 ms 4.4494 ms] std. dev. [72.660 µs 129.11 µs] median [4.3797 ms 4.4075 ms] med. abs. dev. [32.076 µs 66.471 µs]

However, the stats shown for mean median, std. dev., and med.abs.dev. only consists of confidence intervals and not the estimates. I am aware that the estimate values for additional stats can be found in the reports generated inside /target/criterion. However, my use case involves running multiple benchmarks of the same routine with different setups and scenarios. To avoid confusion and ensure accurate comparisons, I often clean the previously generated results before starting a new run.

Given this workflow, I would find it very helpful if the command line output could include the point estimates alongside their confidence intervals. This would allow me to directly collect all estimates from the terminal output using cargo bench --features benchmark -- --verbose > out_scenario_setup_1.txt, without needing to extract them from the /target/criterion directory before cleaning it.

Is there currently a way to achieve this, or could this feature be considered for a future release? It would greatly enhance the convenience and efficiency of using Criterion.rs for complex benchmarking tasks.

Thank you for considering this request.