debug_reprs taking up 41% of test runtime #303
Comments
|
I'm not sure off the top of my head and this isn't high on my priority list. I would encourage you to investigate. |
|
I think I have a solution in #304 |
dead-claudia
added a commit
to dead-claudia/journald-exporter
that referenced
this issue
Jul 6, 2024
...and switch the 32-bit integer parser to just exhaustive checking. (More on that later.) Why move away from QuickCheck? 1. The maintainer appears to have little interest in actually maintaining it. BurntSushi/quickcheck#315 2. Its API is incredibly inefficient, especially on failure, and it's far too rigid for my needs. For one, I need something looser than `Arbitrary: Clone` so things like `std::io::Error` can be generated more easily. Also, with larger structures, efficiency will directly correlate to faster test runs. Also, I've run into the limitations of not being able to access the underlying random number generator far too many times to count, as I frequently need to generate random values within ranges, among other things. - BurntSushi/quickcheck#279 - BurntSushi/quickcheck#312 - BurntSushi/quickcheck#320 - BurntSushi/quickcheck#267 3. It correctly limits generated `Vec` and `String` length, but it doesn't similarly enforce limits on test length. 4. There's numerous open issues in it that I've addressed, in some cases by better core design. To name a few particularly bad ones: - Misuse of runtime bounds in `Duration` generation, `SystemTime` generation able to panic for unrelated reasons: BurntSushi/quickcheck#321 - Incorrect generation of `SystemTime`: BurntSushi/quickcheck#321 - Unbounded float shrinkers: BurntSushi/quickcheck#295 - Avoiding pointless debug string building: BurntSushi/quickcheck#303 - Signed shrinker shrinks to the most negative value, leading to occasional internal panics: BurntSushi/quickcheck#301 There's still some room for improvement, like switching away from a recursive loop: BurntSushi/quickcheck#285. But, this is good enough for my use cases right now. And this code base is structured such that such a change is *much* easier to do. (It's also considerably simpler.) As for the integer parser change, I found a way to re-structure it so I could perform true exhaustive testing on it. Every code path has every combination of inputs tested, except for memory space as a whole. This gives me enough confidence that I can ditch the randomized property checking for it.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
It seems like we're formatting the test case arguments even on passing test runs, e.g. here is the flamegraph of a test run where the test passed, but we see 41% of runtime spent building up
debug_reprs.Does this make sense to you? do you see a way of modifying the test runner to only render the debug representation on failure?
The text was updated successfully, but these errors were encountered: