Query Frontend splitting metrics check is flaky

[pawarpranav83]

• I added CHANGELOG entry for this change.
• Change is not relevant to the end user.

Changes

Fixes #6951, by using receiver instead of prometheus instance.

Verification

[pawarpranav83]

@yeya24 , for the store tests, what is the directory string that I should specify?

thanos/test/e2e/query_frontend_test.go

Line 601 in d388b74

_, err = e2eutil.CreateBlock(ctx, prom.Dir(), timeSeries, 20, timestamp.FromTime(startTime), timestamp.FromTime(endTime), nil, 0, metadata.NoneFunc)

[yeya24]

@pawarpranav83 You can just go ahead and change this code to use remote write to ingest data instead. It should be the same as creating the block directly

[pawarpranav83]

ok, so only one query is failing -

{
        name:           "binary expression",
        qryFunc:        func() string { return `http_requests_total{pod="1"} / http_requests_total` },
        expectedSeries: 2,
},

https://github.com/pawarpranav83/thanos/blob/8889222e610f032889bcb555385c7a60c1b38577/test/e2e/query_frontend_test.go#L872-L876

How do we solve this?

[pawarpranav83]

For the Timestamp of samples, should I have different timestamps from start to end, increment with step size (its implemented like that in CreateBlock function), it worked for the range query but not for instant query in the other test func, so should I add that?

[MichaHoffmann]

I think it doesnt matter for instant query, i might be wrong though!

[MichaHoffmann]

ok, so only one query is failing -

{
        name:           "binary expression",
        qryFunc:        func() string { return `http_requests_total{pod="1"} / http_requests_total` },
        expectedSeries: 2,
},

https://github.com/pawarpranav83/thanos/blob/8889222e610f032889bcb555385c7a60c1b38577/test/e2e/query_frontend_test.go#L872-L876

How do we solve this?

just a guess, but can you try sorting the results before comparing? Can you print the results too please, im curious what is different!

[pawarpranav83]

It was the sample value since it was 0, it was giving NaN, when changed to 1, it worked :)

[MichaHoffmann]

It was the sample value since it was 0, it was giving NaN, when changed to 1, it worked :)

Good catch!

[pawarpranav83]

re-run test?
And are there any more changes to be made?

[yeya24]

Thanks!