Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Durations's Arbitrary instance is dependant on Gen's size #321

Open
avandecreme opened this issue Dec 11, 2023 · 1 comment · May be fixed by #322
Open

Durations's Arbitrary instance is dependant on Gen's size #321

avandecreme opened this issue Dec 11, 2023 · 1 comment · May be fixed by #322

Comments

@avandecreme
Copy link

According to Gen::new doc:

The size parameter controls the size of random values generated. For example, it specifies the maximum length of a randomly generated vector, but is and should not be used to control the range of a randomly generated number. (Unless that number is used to control the size of a data structure.)

However Duration's Arbitrary is defined like this:

quickcheck/src/arbitrary.rs

Lines 1068 to 1072 in aa968a9

fn arbitrary(gen: &mut Gen) -> Self {
let seconds = gen.gen_range(0..gen.size() as u64);
let nanoseconds = gen.gen_range(0..1_000_000);
Duration::new(seconds, nanoseconds)
}

The way seconds is generated seems in violation of the above rule to me and has the consequence of generating very short duration only (with the default size of 100).

According to https://doc.rust-lang.org/nightly/core/time/struct.Duration.html#method.new it seems that we should be able to use the full u64 range without issue for seconds since we are already capping the nano seconds to one billion.

However from my testing this break SystemTime's instance because we can get overflows in the following implementation:

quickcheck/src/arbitrary.rs

Lines 1104 to 1112 in aa968a9

fn arbitrary(gen: &mut Gen) -> Self {
let after_epoch = bool::arbitrary(gen);
let duration = Duration::arbitrary(gen);
if after_epoch {
UNIX_EPOCH + duration
} else {
UNIX_EPOCH - duration
}
}

Any idea how we should fix this?

@neithernut
Copy link

We could use SystemTime::checked_add and checked_sub, and divide the duration by two if that fails. Maybe just do that in a loop until we end up with a valid SystemTime.

@neithernut neithernut linked a pull request Dec 12, 2023 that will close this issue
dead-claudia added a commit to dead-claudia/journald-exporter that referenced this issue Jul 6, 2024
...and switch the 32-bit integer parser to just exhaustive checking.
(More on that later.)

Why move away from QuickCheck?

1. The maintainer appears to have little interest in actually
   maintaining it. BurntSushi/quickcheck#315

2. Its API is incredibly inefficient, especially on failure, and it's
   far too rigid for my needs. For one, I need something looser than
   `Arbitrary: Clone` so things like `std::io::Error` can be generated
   more easily. Also, with larger structures, efficiency will directly
   correlate to faster test runs. Also, I've run into the limitations
   of not being able to access the underlying random number generator
   far too many times to count, as I frequently need to generate random
   values within ranges, among other things.
   - BurntSushi/quickcheck#279
   - BurntSushi/quickcheck#312
   - BurntSushi/quickcheck#320
   - BurntSushi/quickcheck#267

3. It correctly limits generated `Vec` and `String` length, but it
   doesn't similarly enforce limits on test length.

4. There's numerous open issues in it that I've addressed, in some
   cases by better core design. To name a few particularly bad ones:
   - Misuse of runtime bounds in `Duration` generation, `SystemTime`
     generation able to panic for unrelated reasons:
     BurntSushi/quickcheck#321
   - Incorrect generation of `SystemTime`:
     BurntSushi/quickcheck#321
   - Unbounded float shrinkers:
     BurntSushi/quickcheck#295
   - Avoiding pointless debug string building:
     BurntSushi/quickcheck#303
   - Signed shrinker shrinks to the most negative value, leading to
     occasional internal panics:
     BurntSushi/quickcheck#301

There's still some room for improvement, like switching away from a
recursive loop: BurntSushi/quickcheck#285.
But, this is good enough for my use cases right now. And this code
base is structured such that such a change is *much* easier to do.
(It's also considerably simpler.)

As for the integer parser change, I found a way to re-structure it so
I could perform true exhaustive testing on it. Every code path has
every combination of inputs tested, except for memory space as a whole.
This gives me enough confidence that I can ditch the randomized
property checking for it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants