Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Include an overall summary number in compare output #204

Open
itamaro opened this issue May 19, 2022 · 2 comments
Open

Include an overall summary number in compare output #204

itamaro opened this issue May 19, 2022 · 2 comments

Comments

@itamaro
Copy link
Contributor

itamaro commented May 19, 2022

People often quote a single perf number when comparing versions (e.g. "Python 3.11 is 25% faster than Python 3.10")

The pyperformance compare command prints out the "change factor" per benchmark. It's not clear how to go from this list of numbers to a single overall number. Having the compare command compute and print it out would also help with consistency when quoting this single perf number (is it mean? geometric mean? only including significant results or all of them?)

@itamaro
Copy link
Contributor Author

itamaro commented May 19, 2022

ok, just learned this can be done with pyperf

pyperf compare_to baseline.json change.json

@itamaro itamaro closed this as completed May 19, 2022
@ericsnowcurrently
Copy link
Member

It would be worth updating pyperformance though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants