Improve compiling speed

[murphyatwork]

Motivation

As everyone already knows, StarRocks is a fast database, but it has a relatively slow compilation process.

The primary factor contributing to the long compile times is the use of C++ templates. To eliminate the overhead of virtual function calls and ensure that every function is type-specific, many functions are implemented using C++ templates. This allows the compiler to better inline and vectorize the code. However, the downside is that the code size increases significantly, resulting in slower compilation and larger binary sizes.

To address this issue, we have several strategies in mind:

1. Remove unnecessary header files: By moving includes from common headers into the corresponding .cpp files, we can reduce the compiler’s frontend time.
2. Limit over-templating: Avoid templating everything and focus only on cases where it brings substantial benefits.

Tasks

3. Refactoring the compiling model with c++ modules

Remove unnecessary header files

• Generate a quantitative analysis report of the whole compiling process, figure out the heavy headers
• Make the codebase work well with IWYU
• Add clang-tidy include-cleaner
• Use clang-include-fixer
  • update 20241024: tried this tool, but it cannot actually remove any unused includes. But with VSCode it also use this tool, looks like need more investigation
• Prefer forward-declaration than include header file

Limit over-templating(Reduce object file size)

• Use bloaty to get a precise compiled size within binary
  • 20241024: encountered a bug with bloaty, it cannot help: overflow in vm range google/bloaty#382
• Reduce top 20 files size

Top 20 object files, summarized as 588MB

-- find . -name "*.o" -type f -exec ls -lh {} + | sort -k5 -hr | head -n 20
 98M Oct 23 22:42 ./exprs/CMakeFiles/Exprs.dir/case_expr.cpp.o
 53M Oct 23 22:41 ./storage/CMakeFiles/Storage.dir/persistent_index.cpp.o
 50M Oct 23 22:39 ./exprs/CMakeFiles/Exprs.dir/agg/factory/aggregate_resolver_minmaxany.cpp.o
 47M Oct 23 22:39 ./exprs/CMakeFiles/Exprs.dir/cast_expr.cpp.o
 42M Oct 23 22:39 ./exprs/CMakeFiles/Exprs.dir/agg/factory/aggregate_resolver_sumcount.cpp.o
 40M Oct 23 22:39 ./exprs/CMakeFiles/Exprs.dir/agg/factory/aggregate_resolver_avg.cpp.o
 33M Oct 23 22:41 ./gen_cpp/CMakeFiles/StarRocksGen.dir/build/gen_cpp/opcode/builtin_functions.cpp.o
 27M Oct 23 22:39 ./exprs/CMakeFiles/Exprs.dir/agg/factory/aggregate_resolver_others.cpp.o
 25M Oct 23 22:38 ./exprs/CMakeFiles/Exprs.dir/agg/factory/aggregate_resolver_variance.cpp.o
 24M Oct 23 22:37 ./exec/CMakeFiles/Exec.dir/aggregator.cpp.o
 23M Oct 23 22:38 ./exprs/CMakeFiles/Exprs.dir/agg/factory/aggregate_resolver_approx.cpp.o
 20M Oct 23 22:38 ./exprs/CMakeFiles/Exprs.dir/agg/factory/aggregate_resolver_stream.cpp.o
 18M Oct 23 22:38 ./exprs/CMakeFiles/Exprs.dir/arithmetic_expr.cpp.o
 16M Oct 23 22:38 ./exprs/CMakeFiles/Exprs.dir/binary_predicate.cpp.o
 15M Oct 23 22:38 ./exprs/CMakeFiles/Exprs.dir/array_functions.cpp.o
 14M Oct 23 22:40 ./storage/CMakeFiles/Storage.dir/dictionary_cache_manager.cpp.o
 12M Oct 23 22:40 ./gen_cpp/CMakeFiles/StarRocksGen.dir/build/gen_cpp/FrontendService_types.cpp.o
 11M Oct 23 22:38 ./exprs/CMakeFiles/Exprs.dir/condition_expr.cpp.o
 11M Oct 23 22:36 ./exec/CMakeFiles/Exec.dir/aggregate/agg_hash_variant.cpp.o
9.9M Oct 23 22:37 ./exec/CMakeFiles/Exec.dir/sorting/sort_column.cpp.o

[sikehish]

I'd like to take this up.

[murphyatwork]

I'd like to take this up.

Thanks man. As there's actually a lot of cleanup work, i've not figured out a smart tool to automate these work). I think you can either start with some cleanup work(removing unnecessary includes or optimize large object files) or trying to leverage some tools. The first is much easier, if you can understand how does our build process work.

[murphyatwork]

update 20241024:

1. clang-include-fixer: tried this tool, but it cannot actually remove any unused includes. But with VSCode it also use this tool, looks like need more investigation
2. bloaty: encountered a bug with bloaty, it cannot help: overflow in vm range google/bloaty#382

[kevincai]

precompiled header (.pch) might also be worthy to take into consideration.