Tests don't get properly added to the totals when ASSUME is outside of GIVEN

[AsparagusEduardo]

This might just be a thing of improperly made tests.

[mrgriffin]

This is possibly due to the first pass over the test which counts how many PARAMETRIZEs there are. In hindsight we could probably remove this pass and reallocate a bigger results array before calling into the test if needed.

EDIT: Actually my suggestion isn't great, because it would make it difficult to compute the cost of our test cases. But on the other hand, with the structured RNG having landed the costs are much more uniform now.

[mrgriffin]

This is possibly (hopefully! 🤞) improved by the estimateCost changes in #2955.

[hedara90]

Is this issue still relevant or could it be restated?
It still seems to be a problem with ASSUME usage outside of ASSUMPTIONS and GIVEN, but in a different way.

When ASSUME is placed outside of GIVEN in a single test it generates one "ASSUME failed" per thread.
Would it be possible to make it so that ASSUMES places outside of GIVEN/ASSUMPTIONS fail to compile?

[AlexOn1ine]

@AsparagusEduardo this should be fixed by now, right?

[AsparagusEduardo]

@AsparagusEduardo this should be fixed by now, right?

it is as @hedara90 said, still not fixed.

Would it be possible to make it so that ASSUMES places outside of GIVEN/ASSUMPTIONS fail to compile?

Either that or mark them as INVALID like other tests syntaxes are made

[AsparagusEduardo]

Would it be possible to make it so that ASSUMES places outside of GIVEN/ASSUMPTIONS fail to compile?
Either that or mark them as INVALID like other tests syntaxes are made

I don't know how to do this though. I can make a PR that makes all ASSUMES be inside of GIVEN and we start looking for it in reviews?
EDIT: In the documentation, we already have ASSUMEs inside GIVEN

[hedara90]

From what I saw when I looked through the code there was only 1 ASSUME that was outside ASSUMPTIONs or GIVEN blocks (in test/random.c).

[AsparagusEduardo]

From what I saw when I looked through the code there was only 1 ASSUME that was outside ASSUMPTIONs or GIVEN blocks (in test/random.c).

I have the brach ready, there are a lot more (over 30).

[hedara90]

Ah, I kinda meant "that have to be outside"

[AsparagusEduardo]

#5308