[Backport 9.5] Speed-up dramatically proj.db build time. #4284
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Current proj.db build time is typically 50 to 60 seconds (and up to 7.5 hours on arm64 cross-compilation with full emulation!). Most of it is due to running consistency checks. Those checks actually only need to run once each time when we update the content of the database. When skipping them, the build time is cut to 3 seconds or so. So in data/CMakeLists.txt, let keep track of an expected md5sum resulting from the concatenation of the data/sql/*.sql files. When building proj.db, we check if the got and expected md5sum match. If they do build proj.db by inserting the consistency check triggers after having inserted data record. If there's a mismatch, do a one time build with the triggers inserted before the data records, check that proj.db builds fine with that, and if so, emit a CMake error message indicating to the user that they must update the PROJ_DB_SQL_EXPECTED_MD5 variable in data/CMakeLists.txt with the provided value. Next runs will go through the fast build path, until content is updated again.
rouault
force-pushed
the
backport-4280-to-9.5
branch
from
d6a5285
to
1c3b1b2
Compare
rouault
added
the
funded through GSP
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport #4280
Authored by: @rouault