Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't use side inputs with Combine.PerKeyWithHotKeyFanout #20637

Open
damccorm opened this issue Jun 4, 2022 · 3 comments
Open

Can't use side inputs with Combine.PerKeyWithHotKeyFanout #20637

damccorm opened this issue Jun 4, 2022 · 3 comments

Comments

@damccorm
Copy link
Contributor

damccorm commented Jun 4, 2022

PerKeyWithHotKeyFanout doesn't allow a user to specify a side input and access it in the CombineFn. When the function withHotKeyFanout is called, any sideinputs stored in the Combine.PerKey are dropped. (https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Combine.java#L1571): 


    public PerKeyWithHotKeyFanout<K, InputT, OutputT> withHotKeyFanout(
        SerializableFunction<?
super K, Integer> hotKeyFanout) {
      return new PerKeyWithHotKeyFanout<>(fn, fnDisplayData, hotKeyFanout);

   }

We would like to be able to use side inputs and hot key fanout.

Imported from Jira BEAM-11005. Original Jira may contain additional context.
Reported by: miagilepner.

@blazingbhavneek
Copy link

Hey there! 👋 I'm new to this repository and eager to contribute! 🌟 Could you kindly suggest some entry point or files to look into?

@damccorm
Copy link
Contributor Author

Hey, saw you added this comment several places. I'd recommend focusing on a single issue at first (I answered the underlying question here - #20298 (comment))

@marc7806
Copy link
Contributor

marc7806 commented Oct 6, 2023

.take-issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants