-
Notifications
You must be signed in to change notification settings - Fork 752
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: fix non-null constraint #13939
fix: fix non-null constraint #13939
Conversation
Docker Image for PR
|
…fix_non_null_constraint
.../service/src/pipelines/processors/transforms/transform_resort_addon_without_source_schema.rs
Outdated
Show resolved
Hide resolved
when run script split_tx.send(Ok(Split {
info: split_info,
rx: data_rx,
}))
.await
.expect("fail to send split from stream"); // do expect here let me remove the bad test added by me. |
b7c458b
to
e3208fe
Compare
…fix_non_null_constraint
Docker Image for PR
|
.../service/src/pipelines/processors/transforms/transform_resort_addon_without_source_schema.rs
Outdated
Show resolved
Hide resolved
pass checksb in distributed mode and standalone mode without A12 and A13 |
for A12 and A13, we find out these in standalone and distributed mode.(snowflake will give an error in console, but in python script, it doesn't catch the error), these are expected. -- MERGE-INTO-A12: transactions
MERGE INTO transactions USING (
SELECT t.user_id, t.asset_type,
SUM(t.quantity) AS total_quantity
FROM transactions t
GROUP BY t.user_id, t.asset_type
UNION ALL
SELECT t.user_id, 'NEW_TRANSACTION' AS asset_type, -- New transaction type for insert
30 AS quantity -- Example quantity
FROM transactions t
WHERE t.user_id % 2 = 0 -- Condition to select subset for new data
GROUP BY t.user_id
) AS combined_transactions ON transactions.user_id = combined_transactions.user_id AND transactions.asset_type = combined_transactions.asset_type
WHEN MATCHED THEN
UPDATE SET transactions.quantity = combined_transactions.total_quantity
WHEN NOT MATCHED THEN
INSERT (user_id, transaction_type, asset_type, quantity, transaction_time)
VALUES (combined_transactions.user_id, 'trade', combined_transactions.asset_type, combined_transactions.total_quantity, '2023-01-01') -D mergeinto
bendsql command failed: warning: --database is ignored when --dsn is set
Error: APIError: ResponseError with 1006: null value in column `transaction_id` violates not-null constraint
Traceback (most recent call last):
File "/home/ubuntu/github/wizard2/checksb/checksb.py", line 66, in execute_sql
result = subprocess.run(command, text=True, capture_output=True, check=True)
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['bendsql', "--query=\n\n-- MERGE-INTO-A12: transactions\nMERGE INTO transactions USING (\n SELECT t.user_id, t.asset_type,\n SUM(t.quantity) AS total_quantity\n FROM transactions t\n GROUP BY t.user_id, t.asset_type\n UNION ALL\n SELECT t.user_id, 'NEW_TRANSACTION' AS asset_type, -- New transaction type for insert\n 30 AS quantity -- Example quantity\n FROM transactions t\n WHERE t.user_id % 2 = 0 -- Condition to select subset for new data\n GROUP BY t.user_id\n) AS combined_transactions ON transactions.user_id = combined_transactions.user_id AND transactions.asset_type = combined_transactions.asset_type\n WHEN MATCHED THEN\n UPDATE SET transactions.quantity = combined_transactions.total_quantity\n WHEN NOT MATCHED THEN\n INSERT (user_id, transaction_type, asset_type, quantity, transaction_time)\n VALUES (combined_transactions.user_id, 'trade', combined_transactions.asset_type, combined_transactions.total_quantity, '2023-01-01')", '-D', 'mergeinto']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ubuntu/github/wizard2/checksb/checksb.py", line 165, in <module>
main()
File "/home/ubuntu/github/wizard2/checksb/checksb.py", line 151, in main
execute_sql_scripts("bendsql", f"{base_sql_dir}/action.sql", database_name)
File "/home/ubuntu/github/wizard2/checksb/checksb.py", line 83, in execute_sql_scripts
execute_sql(query, sql_tool, database, warehouse)
File "/home/ubuntu/github/wizard2/checksb/checksb.py", line 73, in execute_sql
raise RuntimeError(error_message)
RuntimeError: bendsql command failed: warning: --database is ignored when --dsn is set
Error: APIError: ResponseError with 1006: null value in column `transaction_id` violates not-null constraint |
cc @BohuTANG merge into wizard test is passed, can we merge this? |
Worth to do, we can highlight this in the changelog and our doc, tell the user to add the default value for the not null column if they are not defined before. |
A12/A13 need be fixed to make them works for Databend and Snowflake, I will do it this weekend. @JackTan25 |
…5/databend into fix_non_null_constraint
Fixed: databendlabs/wizard@9cd4d96 |
I hereby agree to the terms of the CLA available at: https://databend.rs/dev/policies/cla/
Summary
Summary about this PR
bug: Not Null constraint is not consistent with postgres and snowflake #13932
This change is