You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
Given the level of integration with pubsub between the different GCP components (cloud run, eventarc, dataflow, bigquery, workflows, ...), having a pubsub sink would be highly beneficial.
I'm currently missing this to take advantage of the pubsub bigquery CDC feature as documented here: https://cloud.google.com/pubsub/docs/bigquery.
Describe the solution you'd like
I'd like a sink wose definition would look like:
CREATE SINK sink1 FROM mv1
WITH (
connector='google_pubsub',
topic='topic1'
)
FORMAT PLAIN ENCODE JSON; -- possibly AVRO [1]
I tried to use the native RW bigquery sink, but it's using the legacy tabledata.insertAll method, which is not compatible with the CDC message ingestion.
Is your feature request related to a problem? Please describe.
Given the level of integration with pubsub between the different GCP components (cloud run, eventarc, dataflow, bigquery, workflows, ...), having a pubsub sink would be highly beneficial.
I'm currently missing this to take advantage of the pubsub bigquery CDC feature as documented here: https://cloud.google.com/pubsub/docs/bigquery.
Describe the solution you'd like
I'd like a sink wose definition would look like:
[1] https://cloud.google.com/pubsub/docs/samples/pubsub-publish-avro-records
Describe alternatives you've considered
I tried to use the native RW bigquery sink, but it's using the legacy tabledata.insertAll method, which is not compatible with the CDC message ingestion.
Additional context
opening the issue here as discussed in https://github.com/orgs/risingwavelabs/discussions/15237
The text was updated successfully, but these errors were encountered: