cargo ./target fills with outdated artifacts as toolchains are updated/changed

[alexheretic]

cargo has to recompile dependencies once a toolchain has updated/changed, this leaves all previous artifacts intact. However, this means when you're updating toolchains the target dir will just grow and grow.

Currently you can cargo clean and rebuild, but it would be nice to be able to clean only the dependencies that are not compatible for the current toolchain. Perhaps cargo clean --incompatible ?

For example building a small crate

# cargo +nightly-2018-02-07 build; du -hd2 ./target
57M   ./target/debug/deps
4.0K  ./target/debug/examples
528K  ./target/debug/.fingerprint
4.0K  ./target/debug/native
18M   ./target/debug/incremental
7.6M  ./target/debug/build
82M   ./target/debug
82M   ./target

# cargo +nightly-2018-02-08 build; du -hd2 ./target
113M  ./target/debug/deps
4.0K  ./target/debug/examples
1.1M  ./target/debug/.fingerprint
4.0K  ./target/debug/native
35M   ./target/debug/incremental
16M   ./target/debug/build
163M  ./target/debug
163M  ./target

# cargo +nightly build; du -hd2 ./target
169M  ./target/debug/deps
4.0K  ./target/debug/examples
1.6M  ./target/debug/.fingerprint
4.0K  ./target/debug/native
52M   ./target/debug/incremental
23M   ./target/debug/build
245M  ./target/debug
245M  ./target

This can obviously be much more problematic on larger crates.

[alexcrichton]

This is currently a feature of Cargo that it aggressively caches, but we could always add a comment to delete otherwise stale artifacts!

[alexheretic]

Yes, I don't think the current behaviour is wrong. It would just be nice to able to say "get rid of the old cached stuff now, I'm not going back to rust 1.22 any time soon". Particularly for things like Rls where we can expect users to update the toolchain fairly often, but don't really inform them about cleaning ./target/rls/.

[seanmonstar]

Since I always develop on nightly, this is true every day. I'm kind of in the habit now of deleting ./target each morning.

However, it seems like this also affects the cache used by Travis CI, as noted in seanmonstar/reqwest#259

[shepmaster]

I've experienced this problem both for local development and in Travis.

For example, my Travis CI cache has 56 copies of every dependency; sometimes the dependency is updated and sometimes the compiler is updated. For one crate I looked at, each build was 250K. In total, my cache weighed in at 1.4 GB. This caused the cache to take about 7-9 minutes to download and upload, compared to a build time of ~2 minutes! Some of that cache is from other parts of the build, but after clearing the cache and rebuilding, it's only ~175MB. Roughly 1.2 GB of the Travis cache was composed of these build artifacts.

I also primarily locally develop using nightly, and often find my laptop's disk running out of space because of multiple outdated gigabytes worth of build artifacts.

I agree that cargo clean --outdated would be a good first step. Right now, I do some complicated shell-scriptery to find all the directories that contain a Cargo.toml and run cargo clean in them.

A potential second step would be to do a bit of inspection during a build to note that there are (many? / large?) artifacts lying around that were not used and warn the user about them. A future thing would be akin to git's garbage collection, where unused artifacts older than some date are just automatically removed.

[Manishearth]

Aside from CI, how common is switching compilers for a given project? I feel like it's not very common.

Can we have an on-by-default pref that specifies whether or not these things are deleted? Travis's image can turn the pref off.

We can also implement something where we record whenever an artifact was last used, and unnecessary artifacts that haven't been touched for a couple days get deleted. But I'd rather go the flat deletion route.

[shepmaster]

how common is switching compilers for a given project? I feel like it's not very common.

Ideally it happens every 6 weeks at a minimum (stable), or perhaps every few days (nightly).

[Manishearth]

I mean, switching back and forth between compilers.

If you update your stable or nightly you don't need the old artifacts anymore. It's only an issue if you're switching back and forth, which I feel is rare.

[dwijnand]

Switching between current stable and current nightly is at least fairly common.

[Manishearth]

Off of CI? I find that a bit hard to believe.

Either way, this would at least work as a preference, preferably on by default.