Per-configuration caching

[orenbenkiki]

Problem

cargo needlessly rebuilds all dependencies whenever some tool uses a different build configuration. For example, if one does cargo check ; cargo test ; cargo clippy ; cargo doc then all the dependencies (and the crate itself) would be rebuilt 4 times, while rebuilding them twice (once for check and clippy, and once for test and doc) would have sufficed.

Proposed Solution

Ideally, cargo would use a global cache outside the package for the results of compiling each dependency with each specific configuration. This would also make sccache mostly redundant. Since this would be so massively useful and still hasn't been done I can only assume there's a really good reason against it, so leave this one aside.

More simply, cargo would cache the results of compiling the current crate's dependencies (and for that matter, the crate itself) in a separate target sub-directory for each configuration (e.g., use a hash of all the configuration parameters, same one that is used today to decide whether the current target is a match for what is requested by each tool).

Notes

As a workaround, I invoke all my cargo commands through a Makefile, and prefix each one with ./with_configuration.sh NAME cargo ... where NAME identifies the configuration needed for that specific cargo command.

The with_configuration.sh file is a blunt instrument - one has to be careful to wrap all the relevant commands with it (though the script offers some protection against this), and specify the correct configuration name for each one (which the script does not enforce).

It is possible to make this script smarter (have it automatically pick the correct configuration name based on the name of the cargo command) but that would still be fragile (e.g., there are many tools, this will not consider the value of environment variables, etc.).

In general the right thing would be for cargo itself to directly allow for caching multiple versions of the built products according to their specific configuration.

#!/usr/bin/bash

# Execute a cargo command in a specific compilation configuration.
#
# Cargo has this delightful restriction that it only caches the built version of dependencies (and the current crate)
# for a specific compilation configurations. However, during development, one uses multiple configurations: one for
# `cargo check`, another for `cargo build` and `cargo test`, and yet another one for `cargo tarpaulin`. Each time one
# issues one of these commands, the results of compiling the dependencies and the crate itself using any other
# configuration is lost and the whole thing is rebuilt from scratch, which is needlessly slow.
#
# This script wraps around any cargo command and provides it with the cached copy of all the compiled results of the
# specific environment it wants. This still means one builds everything once per configuration but at least it is ONLY
# once.
#
# Typically you'd want at least a `base` configuration for `cargo build`, `cargo test` and `cargo doc`, a separate
# `check` configuration for `cargo check` and `cargo clippy`, and possibly a `tarpaulin` configuration for `cargo
# tarpaulin` coverage collection.
#
# This does mean that results will be stored in `.target.$NAME` instead of `target`, e.g. documentation will be
# generated into `.target.base/doc` instead of `target/doc`.
#
# This form of caching should really be a built-in feature of `cargo` itself.

NAME="$1"
shift

if [ -d target ]
then
    if [ -f target/NAME ]
    then
        OLD_NAME=`cat target/NAME`
        echo "Saving the abandoned target.$OLD_NAME" 1>&2
        mv target ".target.$OLD_NAME"
    else
        echo "Cowardly refusing to mess with an unnamed target directory" 1>&2
        exit 1
    fi
fi

if [ -f target ]
then
    rm -f target
fi

if [ -e ".target.$NAME" ]
then
    mv .target.$NAME target
else
    mkdir target
    echo "$NAME" > target/NAME
fi

function cleanup() {
    mv target .target.$NAME
    echo "Wrap all cargo commands with_configuration.sh!" > target
}

trap cleanup EXIT

"$@"

[ehuss]

check, clippy, and doc should not be rebuilding your dependencies. cargo test will, but that is because building and checking are two fundamentally different operations. Can you say more about your project or setup that causes check and clippy to rebuild dependencies?

[orenbenkiki]

Sorry, I was (partially) wrong - I tested more carefully and it does seem that check, build, test and doc do play nicely with each other. However...

They all play badly with tarpaulin. So if I do check, build, test, doc - great, no extra recompilation. If I now do tarpaulin to see coverage, I get an new compilation of everything, and following that, doing any of check, build, test or doc will again recompile everything, which means the next time I want to look at coverage, it will recompile everything again, which is a royal PITA (at least for me).

Also, if one specifies RUSTFLAGS to only one of the commands (e.g., only for test but not for check) then again this will trigger recompilation which will forget the previous variant - so one has to be careful to specify the same flags for all these commands - this is what originally threw me off. Which still doesn't help with tarpaulin... BTW, I tried to figure out what is the exact set of flags it specifies so it could reuse the same dependencies when using --skip-clean but never found the correct incantation - e.g. RUSTFLAGS="-C link-dead-code -C debuginfo=2" didn't work.

So my point still stands, even if it less painful than I originally made it out to be; different tools (e.g. coverage) do compile with different options and cargo only caches the last variant, discarding the rest and rebuilding everything whenever a new variant is asked for.

Using the with_configuration.sh script as a band-aid allows me to have two condifurations - base for most everything and tarpaulin for coverage, which means I only need to compile all the dependencies twice instead of each time I switch between the tools. But as I described, this is a fragile solution better addressed within cargo itself.

Another benefit of direct cargo support would be the ability, at least in principle, to run tools in parallel for the same environment, e.g. clippy / doc and tarpaulin at the same time - something whic today is impossble as they require different compiled versions of everything.

[ehuss]

Ah, yea, changing RUSTFLAGS will cause everything to be recompiled. There are more details in #8716 as to why that happens.

[orenbenkiki]

I guess this is a duplicate of #8716 then...