From 88f3dc8679fc1cd91b572da1220eb62216d21705 Mon Sep 17 00:00:00 2001 From: Ignacio del Valle Alles Date: Mon, 12 Feb 2024 16:21:14 +0100 Subject: [PATCH] Nacho/improve docs (#44) Improves the docs related to parallelization and mount caches --- rust/README.md | 21 +++++++++++++-------- 1 file changed, 13 insertions(+), 8 deletions(-) diff --git a/rust/README.md b/rust/README.md index 6fe6b3d..ceb2757 100644 --- a/rust/README.md +++ b/rust/README.md @@ -84,7 +84,7 @@ cross: RUN --mount=$EARTHLY_RUST_CARGO_HOME_CACHE --mount=$EARTHLY_RUST_TARGET_CACHE cross build --target $TARGET --release END ``` -## COPY_OUTPUT +## +COPY_OUTPUT This function copies files out of the target cache into the image layers. Use it function when you want to `SAVE ARTIFACT` from the target folder (mounted cache), always trying to minimize the total size of the copied fileset. @@ -182,15 +182,20 @@ all: ## Mount caches and parallelization -This library uses several mount caches per tuple of `{project, os_release}`: +As of today, local Cargo builds cannot run in parallel for a given project, given that the output artifact folder is globally locked for the whole build. +This library overcomes such limitation by using different mount caches for the target folder, one per Earthly target. +While multiple concurrent builds of the same Earthly target will still block, the user now has the choice of creating new Earthly targets instead of reusing the same to increase parallelization. + +Hence, this library uses several mount caches per tuple of `{project, os_release}`: - One cache mount for `$CARGO_HOME`, shared across all target builds without any locking involved. -- A family of locked cache mounts for `$CARGO_TARGET_DIR`. One per target. +- A family of locked cache mounts for `$CARGO_TARGET_DIR`. One per Earthly target. Notice that: -- the previous targets builds might belong to one or multiple Earthly builds. -- builds will only be blocked by concurrent ones of the same target +- the previous targets builds might belong to one or multiple Earthly builds, given that the caches involved are global. +- builds will only be blocked by concurrent ones of the same target. +- Earthly target builds are atomic in the sense that the artifacts returned are guaranteed to be the ones generated by that build. For example, running `earthly +all` in the previous example will: -- run all targets (`+lint,+build,+test,+fmt,+check-dependencies`) in parallel without any blocking involved -- use a common cache mount for `$CARGO_HOME` -- use one individual `$CARGO_TARGET_DIR` cache mount per target +- run all targets (`+lint,+build,+test,+fmt,+check-dependencies`) in parallel without any blocking involved. +- use a common cache mount for `$CARGO_HOME`. +- use one individual `$CARGO_TARGET_DIR` cache mount per Earthly target.