query: distributed engine - allow querying overlapping intervals #7985
Conversation
| @@ -340,7 +340,11 @@ test-e2e: docker-e2e $(GOTESPLIT) | |||
| # NOTE(GiedriusS): | |||
| # * If you want to limit CPU time available in e2e tests then pass E2E_DOCKER_CPUS environment variable. For example, E2E_DOCKER_CPUS=0.05 limits CPU time available | |||
| # to spawned Docker containers to 0.05 cores. | |||
| @$(GOTESPLIT) -total ${GH_PARALLEL} -index ${GH_INDEX} ./test/e2e/... -- ${GOTEST_OPTS} | |||
| @if [ -n "$(SINGLE_E2E_TEST)" ]; then \ | |||
| $(GOTESPLIT) -total ${GH_PARALLEL} -index ${GH_INDEX} ./test/e2e -- -run $(SINGLE_E2E_TEST) ${GOTEST_OPTS}; \ | |||
There was a problem hiding this comment.
is make really needed for that? i just run them with "go test"
There was a problem hiding this comment.
not necessarily. Though I had all sorts of issues with my go-env, this was the fastest solution that worked.
I left it in because it could help others new to go as well.
I can ofc remove if you think I'm an outlier:)
| @@ -2376,93 +2376,3 @@ func TestDistributedEngineWithExtendedFunctions(t *testing.T) { | |||
| }, time.Now, promclient.QueryOptions{}, 1) | |||
| testutil.Equals(t, model.SampleValue(0), result[0].Value) | |||
| } | |||
|
|
|||
| func TestDistributedEngineWithDisjointTSDBs(t *testing.T) { | |||
| t.Skip("skipping test as this replicates a bug") | |||
There was a problem hiding this comment.
This got converted to a passing test right? Perhaps we could have one case where the test passes and another where it fails?
There was a problem hiding this comment.
Yes, it got converted.
Perhaps we could have one case where the test passes and another where it fails?
Sure, let me do that
There was a problem hiding this comment.
@fpetkovski added a failing test, which is flaky.
Happy flow with flag enabled is successful in ~10 seconds.
Unhappy flow with the disabled flag can be successful in around 40~ seconds. I haven't validated it yet, but I suspect its because an endpoint is not taken into account and we don't converge on high MinT
Changes
When distributed query mode is enabled, we query endpoints based on the highest MinT they share. If you mix endpoints, e.g storage gateways and receivers and your SGW has a lower MinT than your receivers, only shared intervals will be returned.
This introduces a hidden flag
query.distributed-with-overlapping-intervalthat joins them into the largest overlap instead of shared intervals.Provides a temporary fix for: #7757
Verification
The change is behind a feature flag and is disabled by default. There is also a test reproducing failure scenario.