Propagate annotations of CombinePerKey transforms to resulting component transforms #28489
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue:
If somehow you have an annotation(s) on a
CombinePerKey
transform (e.g. because you subclass it and add annotations or a runner adds annotations), the annotations end up disappearing after getting translated bylifted_stages
.Considerations:
I don't think it's super obvious what the correct behavior is here. It's a bit surprising the annotation disappears but it may also be surprising for someone who meant to signal some behavior for one transform to suddenly have it for the four resulting component transforms. I put up this PR in case it does make sense to propagate the annotations since for our use case it simplifies things a lot
Context:
We're writing a runner and we'd like to be able to save the top level user labels that users assign to their transforms so we can nicely log which user transform is being run at every phase. The most obvious way to me to do this was to create some custom annotations and then search for them at phase execution time. I was surprised when the CPK annotations were missing. Open to other ways to do this though
GitHub Actions Tests Status (on master branch)
See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.