Solace Read connector #31476
Merged
Solace Read connector #31476
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… fields serializable.
|
Run Java_Amazon-Web-Services_IO_Direct PreCommit |
|
Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment |
Abacn
pushed a commit
that referenced
this pull request
Jun 13, 2024
* Splitting the #31476 - Leaving only PTransform AutoValue configurations
…-0-watermark # Conflicts: # sdks/java/io/solace/build.gradle # sdks/java/io/solace/src/main/java/org/apache/beam/sdk/io/solace/SolaceIO.java
…ce-connector # Conflicts: # sdks/java/io/solace/build.gradle # sdks/java/io/solace/src/main/java/org/apache/beam/sdk/io/solace/SolaceIO.java # sdks/java/io/solace/src/main/java/org/apache/beam/sdk/io/solace/broker/SempClientFactory.java # sdks/java/io/solace/src/main/java/org/apache/beam/sdk/io/solace/broker/SessionServiceFactory.java # sdks/java/io/solace/src/main/java/org/apache/beam/sdk/io/solace/data/Solace.java
…lder as an argument
3 tasks
…nnector # Conflicts: # sdks/java/io/solace/build.gradle # sdks/java/io/solace/src/main/java/org/apache/beam/sdk/io/solace/SolaceIO.java # sdks/java/io/solace/src/main/java/org/apache/beam/sdk/io/solace/broker/SempClientFactory.java # sdks/java/io/solace/src/main/java/org/apache/beam/sdk/io/solace/broker/SessionServiceFactory.java # sdks/java/io/solace/src/main/java/org/apache/beam/sdk/io/solace/data/Solace.java # sdks/java/io/solace/src/main/java/org/apache/beam/sdk/io/solace/data/SolaceRecordCoder.java # sdks/java/io/solace/src/test/java/org/apache/beam/sdk/io/solace/MockSempClientFactory.java # sdks/java/io/solace/src/test/java/org/apache/beam/sdk/io/solace/MockSessionServiceFactory.java # sdks/java/io/solace/src/test/java/org/apache/beam/sdk/io/solace/SolaceIOTest.java # sdks/java/io/solace/src/test/java/org/apache/beam/sdk/io/solace/data/SolaceDataUtils.java # sdks/java/io/solace/src/test/java/org/apache/beam/sdk/io/solace/data/SolaceTest.java
…ace-connector # Conflicts: # sdks/java/io/solace/build.gradle # sdks/java/io/solace/src/main/java/org/apache/beam/sdk/io/solace/read/SolaceCheckpointMark.java
|
@bzablocki Could you click the re-request review, if this is ready for my review? |
# Conflicts: # sdks/java/io/solace/build.gradle # sdks/java/io/solace/src/main/java/org/apache/beam/sdk/io/solace/broker/SessionServiceFactory.java
# Conflicts: # sdks/java/io/solace/src/main/java/org/apache/beam/sdk/io/solace/data/Solace.java
bzablocki
force-pushed
the
solace-connector
branch
from
a36ec57 to
da26ab5
Compare
damondouglas
approved these changes
Jun 21, 2024
reeba212
pushed a commit
to reeba212/beam
that referenced
this pull request
Dec 4, 2024
* wip solace connector * wip solace connector * some checker errors resolved * all checker errors resolved * improving unit tests * respond to pr commments * Documentation * Small refactor - move data classes out of the client * refactor * Add github action for integration test of Solace * testing github workflow * bump testcontainers to 1.19.7 - soalce semp was updated with an admin user access * Use FlowHandle to acknowledge messages to make SolaceCheckpointMark's fields serializable. * Handle StaleSessionException error * Add @internal annotation to mark the SolaceIO API beta and subject to change. * Improve documentation * Back to ack based on bytesxmlmessages. Deduplicate default to false. * update changes.md with Solace read connector * remove ack by id code * remove todo comment * Add licenses to package-info.java files * Restructure documentation * update aws test after upgrading testcontainers version. * Disable publishing docs until the first pass on the master branch * Remove files from this branch to split PR into smaller chunks * refactor tests for readability * revert upgrade of testcontainers - not needed in this PR chunk * revert upgrade of testcontainers - not needed in this PR chunk * spotless * remove IT tests from this pr * Tech Writer review * Add a field to Solace.Record mapped from BytesXMLMessage.getAttachmentByteBuffer() * Add and fix some documentation * Remove CheckpointMark's reference to the UnboundedSolaceReader - unnecessary. * Revert "Remove CheckpointMark's reference to the UnboundedSolaceReader - unnecessary." This reverts commit 2e1c10e. * Solace project init - github workflow file, gradle module * Splitting the apache#31476 - Leaving only PTransform AutoValue configurations * remove unnecessary dependencies * remove info from CHANGES.md * Add watermark-related code * Remove excessive @nullable annotations on Solace.Record class * Remove entry from CHANGES.md * Make Record builder package-private * Refactor SolaceIO - the constructor of Read takes a configuration builder as an argument * Change payload and attachments type to immutable ByteString * Downgrade Record builders access modifiers to package private * Add documentation * Add documentation to classes and methods in Solace.java * typo * Add SolaceCheckpointMark.java * Make SolaceCheckpointMark visible for testing * Remove SolaceRecordCoder and take advantage of @DefaultSchema
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hi all,
I've been recently working on creating a Read connector for the Solace message broker (addresses #31440). I also created a Design Doc describing all the details and design choices.
The PR includes the code for the Read connector, unit tests and integration test.
A PR for the Write connector will follow soon.
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, commentfixes #<ISSUE NUMBER>instead.CHANGES.mdwith noteworthy changes.See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)
See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.