Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Task]: [flink] EncodedValueComparator can use serialzed bytes for deterministic Coders #30139

Closed
1 of 16 tasks
je-ik opened this issue Jan 27, 2024 · 3 comments
Closed
1 of 16 tasks

Comments

@je-ik
Copy link
Contributor

je-ik commented Jan 27, 2024

What needs to happen?

EncodedValueComparator enforces deserialization of serialized bytes for key comparison. This can be avoided for deterministic Coders.

Issue Priority

Priority: 3 (nice-to-have improvement)

Issue Components

  • Component: Python SDK
  • Component: Java SDK
  • Component: Go SDK
  • Component: Typescript SDK
  • Component: IO connector
  • Component: Beam YAML
  • Component: Beam examples
  • Component: Beam playground
  • Component: Beam katas
  • Component: Website
  • Component: Spark Runner
  • Component: Flink Runner
  • Component: Samza Runner
  • Component: Twister2 Runner
  • Component: Hazelcast Jet Runner
  • Component: Google Cloud Dataflow Runner
@shourya5
Copy link

.take-issue

@je-ik je-ik changed the title [Task]: [flink] EncodedValueComparator can use deserialzed bytes for deterministic Coders [Task]: [flink] EncodedValueComparator can use serialzed bytes for deterministic Coders Jan 29, 2024
@kirupha2000
Copy link
Contributor

@je-ik I'd like to contribute to this issue if it's still open. I went over the issue description but I wasn't able to understand where exactly the de-serialization is enforced and also how to determine if the bytes were encoded by a deterministic coder. Can you please guide me?

@shourya5 shourya5 removed their assignment Sep 27, 2024
@je-ik
Copy link
Contributor Author

je-ik commented Sep 30, 2024

Hi @kirupha2000 , the current implementation of mentioned EncodedValueComparator seems correct to me now. Closing this.

@je-ik je-ik closed this as completed Sep 30, 2024
@github-actions github-actions bot added this to the 2.60.0 Release milestone Sep 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants