Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactoring: introduce Distributor.sendWriteRequestToBackends() #7877

Merged

Conversation

pracucci
Copy link
Collaborator

@pracucci pracucci commented Apr 11, 2024

What this PR does

I'm working to add support to tee writes both to ingesters and partitions, as part of the migration procedure to the new experimental ingest storage. The full implementation is in this PR: #7871

In this PR I'm proposing a preliminary refactoring to make #7871 easier (and possibly safer) to review. My n. 1 goal is to not introduce any regression in the path writing to ingesters.

If you review only this PR, the design may look weird. I suggest to review the design in #7871 (and leave any comment about the design there), and treat this PR just as a preliminary PR towards that design.

Which issue(s) this PR fixes or relates to

N/A

Checklist

  • Tests updated.
  • Documentation added.
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX].
  • about-versioning.md updated with experimental features.

@pracucci pracucci marked this pull request as ready for review April 11, 2024 11:03
@pracucci pracucci requested a review from a team as a code owner April 11, 2024 11:03
Copy link
Member

@pstibrany pstibrany left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Copy link
Contributor

@colega colega left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, left some nits.

pkg/distributor/distributor.go Outdated Show resolved Hide resolved
pkg/distributor/distributor.go Outdated Show resolved Hide resolved
Comment on lines +1363 to +1365
remoteRequestContext := func() context.Context {
ctx, _ := remoteRequestContextAndCancel()
return ctx
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Opinionated: I'm not sure this is really adding value.

In batchCleanup below you want the cancel, and you just call _, cancel := remoteRequestContextAndCancel() which is fine. What's the reason for not doing localCtx, _ := remoteRequestContextAndCancel() in the push function? Is it to make sure that nobody adds the cancellation call there?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pstibrany I agree with Oleg, but what are your thoughts? You added this function in #7871.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was to simplify the code and remove comment saying that we don't cancel here, only use the context. It makes more sense in "full" PR #7871, I'd keep it since we know more changes are coming.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was to simplify the code and remove comment saying that we don't cancel here, only use the context.

Specifically this comment:

			// Do not cancel the remoteRequestContext in this callback:
 			// there are more callbacks using it at the same time.

which is still in this PR :)

Copy link
Contributor

@colega colega Apr 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 (I imagined the reason, but I found it strange that the other usage of remoteRequestContextAndCancel does just use the underscore. Anyway, I'm also good with keeping it)

@pracucci pracucci force-pushed the refactoring-before-add-write-tee-support-in-distributor branch from d588529 to d646932 Compare April 11, 2024 16:37
@pracucci pracucci merged commit b53ab29 into main Apr 11, 2024
29 checks passed
@pracucci pracucci deleted the refactoring-before-add-write-tee-support-in-distributor branch April 11, 2024 16:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants