From 69faa43f3b6cf19ee43cb29187089ecb099dffdb Mon Sep 17 00:00:00 2001
From: Pierre Le Marre <dev@wismill.eu>
Date: Fri, 8 Jul 2022 11:09:29 +0200
Subject: [PATCH] Fix benchmark instructions: differentiate tasty-bench & gauge
 options. [skip ci]

---
 benchmark/README.md | 57 +++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 55 insertions(+), 2 deletions(-)

diff --git a/benchmark/README.md b/benchmark/README.md
index b49486a..f1df4d9 100644
--- a/benchmark/README.md
+++ b/benchmark/README.md
@@ -1,13 +1,55 @@
+# Benchmark library choice
+
+You may choose between two libraries to perform the benchmarks:
+
+- `tasty-bench`: default
+- `gauge`: alternative, requires cabal flag `use-gauge`.
+
 # Comparing benchmarks for regression
 
 The following commands are tested with `cabal` version 3.0.
 
+## `tasty-bench`
+
+Run the benchmarks for the baseline code i.e. without the changes:
+
+```
+# Remove any old benchmark results file first
+$ rm results.csv
+$ cabal run bench -- --csv=old.csv
+```
+
+It will collect the benchmark results in `old.csv` file.
+
+If you want more accurate benchmark results you can add `--stdev 1`
+(default: 5).
+
+You may generate a SVG chart by adding `--svg chart.svg`.
+
+Modify the code and then run:
+
+```
+$ cabal run bench -- --csv=new.csv --baseline=old.csv
+```
+
+It will collect the new benchmark results in `new.csv` file.
+
+To generate a benchmark comparison in CSV between old and new changes from the
+benchmark results:
+
+```
+# Source: see “Comparison against baseline” in tasty-bench documentation.
+awk 'BEGIN{FS=",";OFS=",";print "Name,Old,New,Ratio"}FNR==1{next}FNR==NR{a[$1]=$2;next}{print $1,a[$1],$2,$2/a[$1];gs+=log($2/a[$1]);gc++}END{print "Geometric mean,,",exp(gs/gc)}' old.csv new.csv
+```
+
+## `gauge`
+
 Run the benchmarks for the baseline code i.e. without the changes:
 
 ```
 # Remove any old benchmark results file first
 $ rm results.csv
-$ cabal run bench -- --csvraw=results.csv --quick
+$ cabal run bench --flag use-gauge -- --csvraw=results.csv --quick
 ```
 
 It will collect the benchmark results in `results.csv` file.
@@ -51,11 +93,22 @@ $ export LIBRARY_PATH=/usr/lib/:/opt/local/lib
 $ cabal bench --extra-lib-dirs=/usr/local/opt/icu4c/lib --extra-include-dirs=/usr/local/opt/icu4c/include
 ```
 
+## `tasty-bench`
+
+Remove any old `results.csv` and run benchmarks with `has-icu` flag enabled:
+
+```
+$ rm results.csv
+$ cabal run bench --flag has-icu -- --csv=results.csv
+```
+
+## `gauge`
+
 Remove any old `results.csv` and run benchmarks with `has-icu` flag enabled:
 
 ```
 $ rm results.csv
-$ cabal run bench --flag has-icu -- --csvraw=results.csv --quick
+$ cabal run bench -f use-gauge -f has-icu -- --csvraw=results.csv --quick
 ```
 
 The following command will now show the comparison between `text-icu` and