Question/Suggestion: Keep an expected test duration log and guide CI users in using it

[nagisa]

It is not uncommon to have test suites where most of the tests run very quickly, and then there are a couple of heavy-weights as well. It is not uncommon that I see my test suite run to go for longer than it needs to because these heavy-weight tests ended up being started after many of the smaller tests have been completed – this is effectively a prime example of a long tail.

I wonder if it would make sense for nextest to keep information about how long tests run for and order the test execution accordingly – starting from the longest-running tests. Bonus points if there are mechanisms (similar to the test archive) to save this information between CI runs so that CI utilization can be optimized.

Perhaps nextest already has such functionality? If it does, please point me at something I can learn to set something like this up! I read through the documentation and experimented a little with nextest invocations to see if it does anything like this already, but I wasn't able to make any conclusions here.

[sunshowers]

Thanks -- yes, that would be really cool. This is similar to #905 except it's the opposite.

The big challenge is actually maintaining a log of test execution times, which is not something that nextest does at the moment. There are subtle details that need to be addressed to make that happen, and it's just work that needs to be done.

I'll close this in favor of #905.