clean: add --with-downloads option

[jonhoo]

This patch adds the --with-downloads flag to cargo clean, which
allows the command to also remove related artifacts from CARGO_HOME
such as downloaded .crate files in ~/.cargo/cache/ and extracted
source directories in ~/.cargo/src/.

Note that this feature is not intended to replace the cargo-cache
command which does smart cache management. Instead, this command simply
blows away whatever it's asked to delete with no smarts whatsoever.

Fixes #3289.

/cc @matthiaskrgr since it seems likely you might have thoughts.

I realize the exact name of this argument is likely going to be subject to some bikeshedding. Alex suggested --global in #3289 (comment), but I didn't go with that since this specifically hits CARGO_HOME, which isn't necessarily "global". I initially used --include-cache, which seemed sufficiently unambiguous, but landed on --with-downloads following Matthias' suggestion.

This PR is also my first encounter with the Filesystem type, so observations about my (potential mis-)use of that would be helpful! Same thing with whether I'm dealing correctly with the package cache lock (specifically in the no -p case).

[rust-highfive]

r? @alexcrichton

(rust-highfive has picked a reviewer for you, use r? to override)

[matthiaskrgr]

Hmm I also have the feeling that the arg name could be improved.
If I am on a train (offline) learning rust and have no idea that there is a build artifacts cache and download cache, I may accidentally just blow away everything and no longer be able to build afterwards.

I think something like cargo clean --downloads or --with-downloads would be clearer, what do you think?

[jonhoo]

Oooh, yeah, I like --with-downloads (or --include-downloads?). It's not completely clear that it includes the extracted source too, but I think it's close enough. I'll make the adjustment on Monday unless other suggestions arise by then.

[bors]

☔ The latest upstream changes (presumably #10080) made this pull request unmergeable. Please resolve the merge conflicts.

[alexcrichton]

I'm going to switch this over to @ehuss who I believe has worked a lot more with paths/caching than I have

r? @ehuss

[bors]

☔ The latest upstream changes (presumably #10072) made this pull request unmergeable. Please resolve the merge conflicts.

[ehuss]

Can you say more about what your use case is? I would not expect anyone to ever need to run such a command.

BTW, I've been working on some experiments with automating cleaning of stale cache files. Related to that, I think we will likely want to separate out cache-related actions to a separate subcommand (probably as a subcommand to clean, though I'm not 100% certain on that).

[jonhoo]

For cargo clean --with-downloads (so no -p), I could envision myself running that occasionally as I've definitely accumulated a bunch of old versions of things up through the years. Sure, I could do something fancy based on time using cargo-cache, but I don't think I would actually save that much compared to just blowing it all away. And I could do it without installing anything :p

As for --with-downloads -p ..., that one is definitely more niche. The use-case for me is specifically around #10071 where I happen to know that whatever Cargo may have cached for a given crate version is now incorrect. Separate from the concerns around what the correct behavior is in that case, and whether it should even be legal, I would much rather have Cargo clean its own caches than have some external program go poking in Cargo's directories to remove things.

There's also the completeness argument: if we have --with-downloads for non--p, we should also support it combined with -p.

BTW, I've been working on some experiments with automating cleaning of stale cache files. Related to that, I think we will likely want to separate out cache-related actions to a separate subcommand (probably as a subcommand to clean, though I'm not 100% certain on that).

What does "stale" mean in this context? I definitely like the sound of that though!

[ehuss]

What does "stale" mean in this context? I definitely like the sound of that though!

The idea is to track the last time something was used, and remove unused things after some period of time. The trick is doing that efficiently, which so far has been difficult.

When I get a chance, I'll try to follow back with some ideas about the interface for manual cleaning.

[bors]

☔ The latest upstream changes (presumably #10168) made this pull request unmergeable. Please resolve the merge conflicts.

[abhizer]

This would be a useful feature, what is blocking this from getting merged?

[epage]

I'm going to go ahead and close this as this is fairly stale and there is more design work to be done before we get back to a PR.

[matthiaskrgr]

@abhizer you can look if https://crates.io/crates/cargo-cache suits your needs

[jonhoo]

I'm going to go ahead and close this as this is fairly stale and there is more design work to be done before we get back to a PR.

Closing this now as I assume you just hit the wrong button 😅

I still think a flag like this might be useful, but don't have a direct need for it myself any more.