Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Relaxing float comparison on fuzz testing #6342

Merged

Conversation

alanprot
Copy link
Member

@alanprot alanprot commented Nov 15, 2024

What this PR does:
Relaxing a bit the float comparison on the fuzz test.

Currently, we allow floats to differ an absolute margin (1e-6). This PR is also allowing the floats to differ by a relative fraction (1.e-10 or 0.00000001% - this is less than 1 second if the result is a epoch now) .

This helps prevent fuzz test failing when the result number is relatively big, ex:

  query_fuzz_test.go:397: case 0 results mismatch.
        range query: avg(exp({__name__="test_series_2"}))
        res1: {} =>
        485165195.4097903 @[1731610569.504]
        1318815734.4832146 @[1731610599.504]
        3584912846.131592 @[1731610629.504]
        9744803446.248903 @[1731610659.504]
        26489122129.84347 @[1731610689.504]
        72004899337.38588 @[1731610719.504]
        195729609428.83878 @[1731610749.504]
        532048240601.79865 @[1731610779.504]
        1446257064291.475 @[1731610809.504]
        3931334297144.042 @[1731610839.504]
        1965909731169.7258 @[1731610869.504]
        1966326556439.2627 @[1731610899.504]
        1967459604995.087 @[1731610929.504]
        1970539550295.1455 @[1731610959.504]
        1978911709636.9426 @[1731610989.504]
        2001669598240.7139 @[1731611019.504]
        2063531953286.4404 @[1731611049.504]
        2231691268872.9204 @[1731611079.504]
        2688795680717.759 @[1731611109.504]
        3931334297144.042 @[1731611139.504]
        1965909731169.7258 @[1731611169.504]
        1966326556439.2627 @[1731611199.504]
        1967459604995.087 @[1731611229.504]
        1970539550295.1455 @[1731611259.504]
        1978911709636.9426 @[1731611289.504]
        2001669598240.7139 @[1731611319.504]
        2063531953286.4404 @[1731611349.504]
        2231691268872.9204 @[1731611379.504]
        2688795680717.759 @[1731611409.504]
        3931334297144.042 @[1731611439.504]
        1965909731169.726 @[1731611469.504]
        1966326556439.2625 @[1731611499.504]
        1967459604995.0867 @[1731611529.504]
        1970539550295.1455 @[1731611559.504]
        1978911709636.9429 @[1731611589.504]
        2001669598240.714 @[1731611619.504]
        2063531953286.4404 @[1731611649.504]
        2231691268872.9204 @[1731611679.504]
        2688795680717.759 @[1731611709.504]
        3931334297144.042 @[1731611739.504]
        1965909731169.7258 @[1731611769.504]
        1966326556439.2627 @[1731611799.504]
        1967459604995.087 @[1731611829.504]
        1970539550295.1455 @[1731611859.504]
        1978911709636.9426 @[1731611889.504]
        2001669598240.7139 @[1731611919.504]
        2063531953286.4404 @[1731611949.504]
        2231691268872.9204 @[1731611979.504]
        2688795680717.759 @[1731612009.504]
        3931334297144.042 @[1731612039.504]
        3931334297144.042 @[1731612069.504]
        3931334297144.042 @[1731612099.504]
        3931334297144.042 @[1731612129.504]
        3931334297144.042 @[1731612159.504]
        3931334297144.042 @[1731612189.504]
        3931334297144.042 @[1731612219.504]
        3931334297144.042 @[1731612249.504]
        3931334297144.042 @[1731612279.504]
        3931334297144.042 @[1731612309.504]
        res2: {} =>
        485165195.4097903 @[1731610569.504]
        1318815734.4832146 @[1731610599.504]
        3584912846.131592 @[1731610629.504]
        9744803446.248903 @[1731610659.504]
        26489122129.84347 @[1731610689.504]
        72004899337.38588 @[1731610719.504]
        195729609428.83878 @[1731610749.504]
        532048240601.79865 @[1731610779.504]
        1446257064291.475 @[1731610809.504]
        3931334297144.042 @[1731610839.504]
        1965909731169.7258 @[1731610869.504]
        1966326556439.2627 @[1731610899.504]
        1967459604995.087 @[1731610929.504]
        1970539550295.1455 @[1731610959.504]
        1978911709636.9426 @[1731610989.504]
        2001669598240.7139 @[1731611019.504]
        2063531953286.4404 @[1731611049.504]
        2231691268872.9204 @[1731611079.504]
        2688795680717.759 @[1731611109.504]
        3931334297144.042 @[1731611139.504]
        1965909731169.7258 @[1731611169.504]
        1966326556439.2627 @[1731611199.504]
        1967459604995.087 @[1731611229.504]
        1970539550295.1455 @[1731611259.504]
        1978911709636.9426 @[1731611289.504]
        2001669598240.7139 @[1731611319.504]
        2063531953286.4404 @[1731611349.504]
        2231691268872.9204 @[1731611379.504]
        2688795680717.759 @[1731611409.504]
        3931334297144.042 @[1731611439.504]
        1965909731169.7258 @[1731611469.504]
        1966326556439.2627 @[1731611499.504]
        1967459604995.087 @[1731611529.504]
        1970539550295.1455 @[1731611559.504]
        1978911709636.9426 @[1731611589.504]
        2001669598240.7139 @[1731611619.504]
        2063531953286.4404 @[1731611649.504]
        2231691268872.9204 @[1731611679.504]
        2688795680717.759 @[1731611709.504]
        3931334297144.042 @[1731611739.504]
        1965909731169.7258 @[1731611769.504]
        1966326556439.2627 @[1731611799.504]
        1967459604995.087 @[1731611829.504]
        1970539550295.1455 @[1731611859.504]
        1978911709636.9426 @[1731611889.504]
        2001669598240.7139 @[1731611919.504]
        2063531953286.4404 @[1731611949.504]
        2231691268872.9204 @[1731611979.504]
        2688795680717.759 @[1731612009.504]
        3931334297144.042 @[1731612039.504]
        3931334297144.042 @[1731612069.504]
        3931334297144.042 @[1731612099.504]
        3931334297144.042 @[1731612129.504]
        3931334297144.042 @[1731612159.504]
        3931334297144.042 @[1731612189.504]
        3931334297144.042 @[1731612219.504]
        3931334297144.042 @[1731612249.504]
        3931334297144.042 @[1731612279.504]
        3931334297144.042 @[1731612309.504]

Which issue(s) this PR fixes:
Fixes #

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

Copy link
Contributor

@yeya24 yeya24 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make sense. thanks!

@yeya24 yeya24 merged commit 0eab788 into cortexproject:master Nov 16, 2024
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants