Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow '/' in test name (rust) #26

Merged
merged 2 commits into from
Apr 27, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions .github/workflows/criterion-rs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
name: Criterion.rs Example
on:
push:
branches:
- master

jobs:
benchmark:
name: Run Criterion.rs benchmark example
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- run: rustup toolchain update
- name: Run benchmark
run: cd examples/criterion-rs && cargo bench -- --output-format bencher | tee output.txt
- name: Store benchmark result
uses: rhysd/github-action-benchmark@v1
with:
name: Rust Benchmark
tool: 'cargo'
output-file-path: examples/criterion-rs/output.txt
# Use personal access token instead of GITHUB_TOKEN due to https://github.community/t5/GitHub-Actions/Github-action-not-triggering-gh-pages-upon-push/td-p/26869/highlight/false
github-token: ${{ secrets.PERSONAL_GITHUB_TOKEN }}
auto-push: true
# Show alert with commit comment on detecting possible performance regression
alert-threshold: '200%'
comment-on-alert: true
fail-on-alert: true
alert-comment-cc-users: '@rhysd'
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
/src/*.js.map
/examples/rust/Cargo.lock
/examples/rust/target
/examples/criterion-rs/Cargo.lock
/examples/criterion-rs/target
/test/*.js
/test/*.js.map
/scripts/*.js
Expand Down
17 changes: 17 additions & 0 deletions examples/criterion-rs/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
[package]
name = "criterion_example"
version = "0.1.0"
authors = ["rhysd <https://rhysd.github.io>"]
edition = "2018"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
criterion = "*"

[lib]
bench = false

[[bench]]
name = "bench"
harness = false
37 changes: 37 additions & 0 deletions examples/criterion-rs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Rust Criterion example for benchmarking with `cargo bench`

- [Workflow for this example](../../.github/workflows/criterion-rs.yml)
- [Benchmark results on GitHub pages](https://rhysd.github.io/github-action-benchmark/dev/bench/)

This directory shows how to use [`github-action-benchmark`](https://github.com/rhysd/github-action-benchmark)
with [`criterion`](https://github.com/bheisler/criterion.rs).

## Run benchmarks

Official documentation for usage of `cargo bench` with Criterion:

https://github.com/bheisler/criterion.rs

e.g.

```yaml
- name: Run benchmark
run: cargo bench -- --output-format bencher | tee output.txt
```

Note that you should run the benchmarks using the bencher output format.


## Process benchmark results

Store the benchmark results with step using the action. Please set `cargo` to `tool` input.

```yaml
- name: Store benchmark result
uses: rhysd/github-action-benchmark@v1
with:
tool: 'cargo'
output-file-path: output.txt
```

Please read ['How to use' section](https://github.com/rhysd/github-action-benchmark#how-to-use) for common usage.
35 changes: 35 additions & 0 deletions examples/criterion-rs/benches/bench.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
#[macro_use]
extern crate criterion;
use criterion::{black_box, Criterion, BenchmarkId};
use criterion_example::{fib, fast_fib};

fn bench_fib_10(c: &mut Criterion) {
c.bench_function("BenchFib10", move |b| {
b.iter(|| {
let _ = fib(black_box(10));
});
});
}

fn bench_fib_20(c: &mut Criterion) {
c.bench_function("BenchFib20", move |b| {
Copy link
Contributor Author

@morenol morenol Apr 26, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that we can add whitespaces and other characters to the bench title that would make the title doesn't match with the extract regex.

I didn't add it because I am not sure if you want to add whitespaces and possibly other characters to the regex and because this can be avoided by the owner of the repo that is running this action by just renaming the bench title.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok. When someone complains about that, let's consider to support spaces by making output parser smarter.

b.iter(|| {
let _ = fib(20);
});
});
}

fn bench_fibs(c: &mut Criterion) {
let mut group = c.benchmark_group("Fibonacci");
for i in [20, 21].iter() {
group.bench_with_input(BenchmarkId::new("Recursive", i), i,
|b, i| b.iter(|| fib(*i)));
group.bench_with_input(BenchmarkId::new("Iterative", i), i,
|b, i| b.iter(|| fast_fib(*i)));
}
group.finish();
}


criterion_group!(benches, bench_fib_10, bench_fib_20, bench_fibs);
criterion_main!(benches);
25 changes: 25 additions & 0 deletions examples/criterion-rs/src/lib.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
pub fn fib(u: u32) -> u32 {
if u <= 1 {
1
} else {
fib(u - 2) + fib(u - 1)
}
}

pub fn fast_fib(n: u32) -> u32 {
let mut a = 0;
let mut b = 1;

match n {
0 => b,
_ => {
for _ in 0..n {
let c = a + b;
a = b;
b = c;
}
b
}
}
}

2 changes: 1 addition & 1 deletion src/extract.ts
Original file line number Diff line number Diff line change
Expand Up @@ -178,7 +178,7 @@ function extractCargoResult(output: string): BenchmarkResult[] {
const ret = [];
// Example:
// test bench_fib_20 ... bench: 37,174 ns/iter (+/- 7,527)
const reExtract = /^test (\w+)\s+\.\.\. bench:\s+([0-9,]+) ns\/iter \(\+\/- ([0-9,]+)\)$/;
const reExtract = /^test ([\w/]+)\s+\.\.\. bench:\s+([0-9,]+) ns\/iter \(\+\/- ([0-9,]+)\)$/;
const reComma = /,/g;

for (const line of lines) {
Expand Down