Report multiple "failed to get dependency" errors at once

[QuarticCat]

Problem

We have many cargo packages spread across many repos. When someone enters a project, we need to grant access to dependency repos. And it's easy to miss several ones. So we often see that

1. Cargo failed to get a dependency.
2. The user reported that error.
3. We fixed the permission issue.
4. The user reran Cargo and got another "failed to get dependency" error. Back to 2.

If Cargo can report multiple "failed to get dependency" errors at once, we could save some RTTs in this scenario.

Related code:

cargo/src/cargo/core/resolver/dep_cache.rs

Lines 246 to 267 in f1e4470

	// Next, transform all dependencies into a list of possible candidates
	// which can satisfy that dependency.
	let mut all_ready = true;
	let mut deps = deps
	.into_iter()
	.filter_map(|(dep, features)| match self.query(&dep, first_version) {
	Poll::Ready(Ok(candidates)) => Some(Ok((dep, candidates, features))),
	Poll::Pending => {
	all_ready = false;
	// we can ignore Pending deps, resolve will be repeatedly called
	// until there are none to ignore
	None
	}
	Poll::Ready(Err(e)) => Some(Err(e).with_context(|| {
	format!(
	"failed to get `{}` as a dependency of {}",
	dep.package_name(),
	describe_path_in_context(cx, &candidate.package_id()),
	)
	})),
	})
	.collect::<CargoResult<Vec<DepInfo>>>()?;

Proposed Solution

No response

Notes

No response

[weihanglo]

Talked to the issue author offline. The case doesn't seem unreasonable or uncommon.

Because the "failed to get dependency" error is a fatal error, the program would exit early anyway, so a straightforward solution of it is collecting every error of that kind and returning them all at once.

However, the build_deps function gets called in a hot loop, and it may query remote registries by HTTP request, which is costly. That is a performance concern, and I assume that was why we did early returns for those fatal errors.

@Eh2406 I wonder if there is any good ideas to improve the situation on the resolver side.

[epage]

While I'm assuming these details were covered offline, @QuarticCat could you add details on what you are granting access that then fails, whether it be on the registry index, registry .crates, git repos, etc.

[QuarticCat]

@epage All of our private dependencies are git repos. We have no self-hosted registry yet. So it's like

# Cargo.toml

[dependencies]
dep1 = { git = "ssh://git@xxx.com/yyy/dep1", tag = "v0.1.0" }
dep2 = { git = "ssh://git@xxx.com/yyy/dep2", tag = "v0.1.0" }

If users have no access to both dep1 and dep2, they will only get

error: failed to get `dep1` as a dependency of package `the-project`

Caused by:
  failed to load source for dependency `dep1`

Caused by:
  Unable to update ssh://git@xxx.com/yyy/dep1?tag=v0.1.0

...

After we grant access to dep1, the user will then get

error: failed to get `dep2` as a dependency of package `the-project`

Caused by:
  failed to load source for dependency `dep2`

Caused by:
  Unable to update ssh://git@xxx.com/yyy/dep2?tag=v0.1.0

...

What can be improved here is that users should get all these error messages at once.