QueryPlan Optimizer aims to run TPCH 100x faster #116
Conversation
* PR_SET_NAME workaround AWS Lambdas (and other virtualized platforms) lack of support for PR_SET_NAME causing a blocking exception. Pending an upstream PR or fix in ClickHouse, this patch allows this function to fail unharmed. The resulting executable has been tested on various platforms without drawbacks and discussed in clickhouse issue [29378](ClickHouse/ClickHouse#29378) $ sed -i '/Cannot set thread name/c\' /ClickHouse/src/Common/setThreadName.cpp * Disable AVX2 support
Although we lock llvm@15, different github action runner may have llvm 15.0.7 or 15.0.7_1. This will cause ccache miss a lot. Hope brew update will keep llvm latest
Although we lock llvm@15, different github action runner may have llvm 15.0.7 or 15.0.7_1. This will cause ccache miss a lot. Hope brew update will keep llvm latest
1. Add missing transforms
2. Disable IQueryPlanStep operator==
3. virtual void updateOutputStream() throw Exception
4. Remove ASTSerDerHelper
1. Throw exception on serialize or deserialize
2. Fix JoinStep and JoiningTransform
3. Drop PlanCache
4. Fix chdbVersion
|
wait are you going to build a new Query optimizer from scratch ? |
No, transplant optimizer from ByConity which has forked from ClickHouse v21.8 |
|
Is there any way to implement this in the main branch of Clickhouse as well? Would be huge |
Perhaps later, even though it has already taken me months, I am unsure if this will be successful.😅 |
|
@auxten you can do it !!! |
|
It makes sense to also send this PR to the main repository - interesting if it will pass the tests. Although I expect it is going to be difficult... |
1. Fix all ASOF::Inequality to ASOFJoinInequality
2. Remove Exchange and RemoteExchange stuff
3. Remove DynamicFilter
4. Remove ProjectionStep
|
One downside: if it is not tested by ClickHouse CI, it will most likely contain a ton of bugs, and having these bugs will lead to reputational risks for ClickHouse. It means - we have to send a PR to the main ClickHouse repository. |
For sure. This will be considered highly experimental and it won't have mainstream ambitions unless we all agree the results are overwhelmingly positive. On the other hand as a fork, we can afford taking some early risks the mainline project cannot 😉 |
|
Absolutely agree with @alexey-milovidov. As a data tool the quality is the most important thing. This branch is very very experimental. I have never expected it would draw so much attention. |
1. Remove FinishSortingStep
2. Fix FilterStep
3. Fix ProjectionMatchContext
4. Fix TableScanExecutor::TableScanExecutor
5. Just keep serialize/deserialize throw Exception
|
|
|
@auxten Rebase? |
I didn't working on this for a long time. It's still very far from general available. |
Still in a very early stage. This PR is only for tracking progress.