Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(pg-cdc): fix support for partitioned table #18456

Merged
merged 13 commits into from
Sep 22, 2024

Conversation

KeXiangWang
Copy link
Contributor

@KeXiangWang KeXiangWang commented Sep 8, 2024

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

The bug is found by user.
The bug contains two symptom:

  1. If the specified publication is created without publish_via_partition_root = true, the source/table creation will throw an error with message ERROR: relation "original_table" is already member of publication "rw_publication".
  2. If the publication is created by RW, RW cannot get any data.

The root cause is the publish_via_partition_root = true. Only with this option, the wal of each parition will be published in the root table's publication.

The fix is inspired by https://groups.google.com/g/debezium/c/NroAVr2saxU/m/eSeS8hYXCwAJ

Checklist

  • I have written necessary rustdoc comments
  • I have added necessary unit tests and integration tests
  • I have added test labels as necessary. See details.
  • I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features Sqlsmith: Sql feature generation #7934).
  • My PR contains breaking changes. (If it deprecates some features, please create a tracking issue to remove them in the future).
  • All checks passed in ./risedev check (or alias, ./risedev c)
  • My PR changes performance-critical code. (Please run macro/micro-benchmarks and show the results.)
  • My PR contains critical fixes that are necessary to be merged into the latest release. (Please check out the details)

Documentation

  • My PR needs documentation updates. (Please use the Release note section below to summarize the impact on users)

Release note

https://github.com/risingwavelabs/risingwave-docs/pull/2628

Copy link

gitguardian bot commented Sep 8, 2024

️✅ There are no secrets present in this pull request anymore.

If these secrets were true positive and are still valid, we highly recommend you to revoke them.
Once a secret has been leaked into a git repository, you should consider it compromised, even if it was deleted immediately.
Find here more information about risks.


🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

@github-actions github-actions bot added the type/fix Bug fix label Sep 8, 2024
@KeXiangWang KeXiangWang force-pushed the wkx/fix-pg-cdc-partitioned-table branch 2 times, most recently from 689547f to f032130 Compare September 8, 2024 01:51
@KeXiangWang
Copy link
Contributor Author

KeXiangWang commented Sep 8, 2024

The fix in this PR works. However, due to a bug in sea-schema, CI fails. Their fix is here, which is included in 0.15.0.

The upgrading to 0.15.0 of sea-schema is non-trivial. Will create another PR to fix.

Copy link
Contributor

@StrikeW StrikeW left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@lmatz lmatz added the user-facing-changes Contains changes that are visible to users label Sep 9, 2024
@KeXiangWang KeXiangWang force-pushed the wkx/fix-pg-cdc-partitioned-table branch from 76f364a to 709df33 Compare September 19, 2024 02:54
@KeXiangWang KeXiangWang requested a review from a team as a code owner September 19, 2024 02:54
@KeXiangWang KeXiangWang force-pushed the wkx/fix-pg-cdc-partitioned-table branch from 07d9887 to 212d9df Compare September 20, 2024 22:13
@KeXiangWang KeXiangWang force-pushed the wkx/fix-pg-cdc-partitioned-table branch from 212d9df to 24b1f68 Compare September 20, 2024 22:41
@graphite-app graphite-app bot requested a review from a team September 21, 2024 03:06
@KeXiangWang KeXiangWang added this pull request to the merge queue Sep 22, 2024
Merged via the queue into main with commit bc65ffb Sep 22, 2024
37 of 38 checks passed
@KeXiangWang KeXiangWang deleted the wkx/fix-pg-cdc-partitioned-table branch September 22, 2024 00:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/fix Bug fix user-facing-changes Contains changes that are visible to users
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants