Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: change tracking stream failed #17072

Merged
merged 1 commit into from
Dec 19, 2024
Merged

Conversation

zhyass
Copy link
Member

@zhyass zhyass commented Dec 18, 2024

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

Use materialized cte for standard stream to fix the error.

The change query for stream:
PostgreSQL Dialect

WITH
  _change$6763a8db AS materialized (
    SELECT
      "a",
      "b",
      a_change$action,
      a_change$row_id,
      d_a,
      d_b,
      d_change$action,
      d_change$row_id
    FROM
      (
        SELECT
          "a",
          "b",
          _row_version,
          'INSERT' AS a_change$action,
          if(
            is_not_null (_origin_block_id),
            concat(
              to_uuid (_origin_block_id),
              lpad(hex(_origin_block_row_num), 6, '0')
            ),
            _change_insert$6763a8db._base_row_id
          ) AS a_change$row_id
        FROM
          DEFAULT.s AS _change_insert$6763a8db
      ) AS A 
      FULL OUTER JOIN (
        SELECT
          "a" AS d_a,
          "b" AS d_b,
          _row_version,
          'DELETE' AS d_change$action,
          if(
            is_not_null (_origin_block_id),
            concat(
              to_uuid (_origin_block_id),
              lpad(hex(_origin_block_row_num), 6, '0')
            ),
            _change_delete$6763a8db._base_row_id
          ) AS d_change$row_id
        FROM
          DEFAULT.s AS _change_delete$6763a8db
      ) AS D ON A.a_change$row_id = D.d_change$row_id
    WHERE
      A.a_change$row_id IS NULL
      OR D.d_change$row_id IS NULL
      OR A._row_version > D._row_version
  )
SELECT
  "a",
  "b",
  a_change$action AS change$action,
  a_change$row_id AS change$row_id,
  d_change$action IS NOT NULL AS change$is_update
FROM
  _change$6763a8db
WHERE
  a_change$action IS NOT NULL
UNION ALL
SELECT
  d_a,
  d_b,
  d_change$action,
  d_change$row_id,
  a_change$action IS NOT NULL
FROM
  _change$6763a8db
WHERE
  d_change$action IS NOT NULL

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

@zhyass zhyass changed the title fix: change tracking stream panic fix: change tracking stream failed Dec 18, 2024
@github-actions github-actions bot added the pr-bugfix this PR patches a bug in codebase label Dec 18, 2024
@zhyass zhyass marked this pull request as draft December 18, 2024 05:50
@zhyass zhyass marked this pull request as ready for review December 19, 2024 01:21
@zhyass zhyass force-pushed the feature_fix branch 2 times, most recently from 7c7b239 to fa92da6 Compare December 19, 2024 05:16
@Dousir9 Dousir9 enabled auto-merge December 19, 2024 05:46
@Dousir9 Dousir9 disabled auto-merge December 19, 2024 05:58
@Dousir9 Dousir9 added this pull request to the merge queue Dec 19, 2024
Merged via the queue into databendlabs:main with commit aa9a972 Dec 19, 2024
75 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr-bugfix this PR patches a bug in codebase
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Bug: Change tracking stream test case fails
2 participants