This release contains the nnbench
CLI as a headline feature for running benchmarking workloads as part of scripts and CML pipelines.
It also contains a large refactor of core abstractions such as the benchmark runners and reporters. In particular, the nnbench.FileReporter
got support for writing files with fsspec, making it possible to stream records to all major cloud storage providers among others.
What's Changed
- (fix) Change default == value to is by @maxmynter in #139
- Add nnbench video to readme by @maxmynter in #144
- Safe setting of CPU Info by @maxmynter in #146
- Bump oldest supported Python to 3.10 by @nicholasjng in #147
- Switch project to uv lockfile by @nicholasjng in #148
- Remove requirement files, strict version pins from examples by @nicholasjng in #149
- Remove unused io module stub by @nicholasjng in #150
- Add CLI entrypoints for module and program invocation by @nicholasjng in #152
- Run pre-commit autoupdate by @nicholasjng in #153
- Add Python 3.13 tests to CI by @nicholasjng in #155
- refactor: Remove context class, back to raw dictionaries by @nicholasjng in #154
- Remove setuptools_scm in favor of explicit version management by @nicholasjng in #156
- Remove transform facility and examples by @nicholasjng in #161
- Remove context processing options by @nicholasjng in #160
- Add
Benchmark.to_list()
andto_json()
to improve record IO by @nicholasjng in #162 - Remove compression, support file reporting and context passing on the command line by @nicholasjng in #163
- Add
nnbench compare
subcommand, comparison function by @nicholasjng in #164 - Refactor console reporter by @nicholasjng in #165
- Add --version switch to print version by @nicholasjng in #166
- Inline boolean option pairs, change subcommand section by @nicholasjng in #167
- Add run name to benchmark record by @nicholasjng in #171
- Simplify JSON representation computation facility by @nicholasjng in #172
- Add a fixture facility to nnbench by @nicholasjng in #173
- Remove typechecks from benchmark runner by @nicholasjng in #174
- Add TOML config parsing stub by @nicholasjng in #175
- Add artifact outputs to gitignore by @nicholasjng in #176
- Add API docs for all direct
nnbench
submodules by @nicholasjng in #178 - Add nnbench CLI guide by @nicholasjng in #179
- Make arguments optional on
ContextProviderDef
by @nicholasjng in #181 - Add fixture usage guide document by @nicholasjng in #182
- Fix README lightning demo example by @nicholasjng in #183
- Update code examples, fix parameter hydration bug by @nicholasjng in #184
- Refactor jsonifier to be taken as an argument to
BenchmarkRunner.run()
by @nicholasjng in #185
Full Changelog: v0.3.0...v0.4.0