Cargo check does not check doctests

[jonhoo]

Problem
The command cargo check --all-targets does not compile doctests.

Steps

1. Add a doctest to a project with a syntax error in it.
2. Run cargo check --all-targets, which should "Check all targets" according to --help
3. Observe that the error in the doctest is not reported.

Notes
I suspect this has been known for some time (#4592 (comment)). Combined with #5242, this means that there is no way to check the code in a doctest without running it (and using cargo test). From memory, I think that's because doctests are very different from all other code and tests, but I think it's surprising enough (I keep re-discovering this even now!) that it should probably be fixed.

Output of cargo version: cargo 1.32.0-nightly (b3d0b2e54 2018-11-15)

[dwijnand]

This is blocked on the lack of upstream (rustdoc) support for checking doc examples. I had a go looking, but I couldn't find an upstream issue tracking this.

[jyn514]

This is blocked on the lack of upstream (rustdoc) support for checking doc examples. I had a go looking, but I couldn't find an upstream issue tracking this.

Rustdoc implemented --no-run a few months ago; rust-lang/rust#87022 is the tracking issue (since today).

[gilescope]

Anyone tempted to have a crack at integrating it?

[est31]

I would like to work on it, if nobody beats me to it. Can look into it next week or so.

[ehuss]

There is a PR open to add this (#8859). I don't think --no-run is quite the same as --check. You could maybe ask Guillaume if they mind if you pick it back up.

[jyn514]

@ehuss --check isn't the same either - that checks the crate itself, not any of its doctests.

[jyn514]

I wonder if it makes sense to add an option to forward options to rustc verbatim, so you could pass --emit=metadata and things like that instead of needing a dedicated option for everything.

[ehuss]

Ah, I misunderstood what --check does.

[ehuss]

It seems like --no-run has an unfortunate interaction with compile_fail in that it still performs linking, which means you cannot check compile_fail tests (in a sense, they always fail, because the dependencies won't be available). I'm not sure if that's reason enough to not use that flag, but it is a concern.

Being able to pass arbitrary flags is certainly an option, but it introduces a compatibility hazard when you want to change the way rustdoc behaves in the future. For example, something like rust-lang/rust#50784 or rust-lang/rust#75341 could require fundamentally changing the way doctests work, and could introduce difficulties if users are passing flags that may not be compatible. It's certainly an option, but it would be nice if rustdoc could not leak its details too much.

I would expect the combination of rustdoc --check --test to check tests in a way similar to how cargo check works, and then that could just be added to a cargo check run somehow.