Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Table metadata fails to reflect if is_deleted column is set along with version in ReplacingMergeTree engine #330

Open
kxd8163 opened this issue Aug 19, 2024 · 0 comments

Comments

@kxd8163
Copy link

kxd8163 commented Aug 19, 2024

Describe the bug

For tables defined with version column and is_deleted:

ENGINE = ReplacingMergeTree(version_col, is_deleted)

metadata reflection fails with following:

metadata.reflect(bind=engine, schema=db_name)
*** sqlalchemy.exc.ConstraintColumnNotFoundError: Can't create TableCol on table 'myThirdReplacingMT
': no column named 'version_col, is_deleted' is present.`

The issue lays in insufficient parsing logic for ReplacingMergeTree([ver [, is_deleted]]) clause that assumes only version being set

To Reproduce

-- with ver and is_deleted
CREATE OR REPLACE TABLE myThirdReplacingMT
(

    key Int64,

    someCol String,

    eventTime DateTime,

    is_deleted UInt8

)

ENGINE = ReplacingMergeTree(eventTime, is_deleted)

ORDER BY key

SETTINGS allow_experimental_replacing_merge_with_cleanup = 1;

....
metadata.reflect(bind=engine, schema=url)

Fix

--- a/clickhouse_sqlalchemy/engines/mergetree.py
+++ b/clickhouse_sqlalchemy/engines/mergetree.py
@@ -218,6 +218,8 @@ class ReplacingMergeTree(MergeTree):
     def reflect(cls, table, engine_full, **kwargs):
         engine = parse_columns(engine_full, delimeter=' ')[0]
         version_col = engine[len(cls.__name__):].strip('()') or None
+        if version_col is not None:
+            version_col = version_col.split(',')[0]
 
         return cls(
             version=version_col,

Versions

  • Version of package with the problem: clickhouse-sqlalchemy==0.3.2
  • Python version: 3.12.3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant