Allow specifying build dependencies `opt-level` with `[project.dev.build-override.package]`

[mkurnikov]

Problem

We have a compiler + VM monorepo with a build step, where we compile a standard library to include into the executable later.
The step itself depends on the part of the monorepo, and it's dependency tree is around 700 crates. But the actual compilation is mostly happens in one compiler crate.

I want to compile that compiler crate with opt-level = 1. This way, the compilation step reduced from 70s to 20s.
But I don't want to compile with opt-level = 1 the whole set of 700 crates of build dependencies.

Proposed Solution

There already a [profile.dev.build-override] which allow to specify opt-level for the whole set of [build-dependencies].
There's also [profile.dev.package] which allows specifying compiler.opt-level = 1.
I want to have

[profile.dev.build-override.package]
compiler.opt-level = 1

which combines those two together.

Notes

The idea has been mentioned before in #7940, but the discussion there seem to be focused on the library crates.
I want to control the opt-level myself.

I can contribute the feature if it deemed acceptable - building blocks are already in-place.

[epage]

But I don't want to compile with opt-level = 1 the whole set of 700 crates of build dependencies.

What is the reason for this?

[profile.dev.build-override.package]

Note that this would be

{
  "profile": {
    "dev": {
        "build-override": {
          "opt-level": 1,
          "package": {
            "compiler": {
              "opt-level": 1,
            }
          }
        }
    }
  }
}

Note that opt-level (a profile field) and package (a parent table) are at the same level. While technically there wouldn't be overlaps between profile fields and package and they are all cargo-defined, I'm not too thrilled about mixing fields like this.

[mkurnikov]

Thank you for the response.

What is the reason for this?

There are a number of very heavy dependencies for the dep-tree of the compiler crate. I hoped I'll be able to profile which crates are used in the compilation step, and only enable opt-level = 1 for them. Otherwise, my debug dep-tree grows by that 700 crates (to be fair, only like 5-7 are workspace-level and frequently changed). Plus those crates are opt-level = 1 so about twice as slow to compile than the usual debug.

Also that build.rs is invoked on cargo check too, so suddenly it also needs to compile the whole thing with opt-level = 1 (and cargo check is supposed to be fast).

I can just compile the whole debug with opt-level = 1 to deduplicate though, I just thought maybe there could be a better way to handle it.

Note

I was thinking of package table as "per-package configuration for the workspace", and build-override as "build-dependencies-only configuration for the workspace".
So build-override.package reads nicely as "per-package configuration for build-dependencies in the workspace", doesn't seem to be too contrived.

[epage]

Thanks for providing more context!

I'm going to explore a route but I'm thinking it won't work for your needs. In general, we've been trying to identify alternatives to build scripts (#14948). For code-gen, one idea is for tests to do the code-gen and then commit the result. This works well when the input has a relative low velocity. It also works best when the output is text which it sounds like yours is a binary blob. I would then consider a build orchestrator (e.g. just, make) to coordinate handling of assets but you are bundling the asset directly in your binary.

I can just compile the whole debug with opt-level = 1 to deduplicate though, I just thought maybe there could be a better way to handle it.

FYI we are wanting to find a way to split up "faster iteration time" from "debugging" profiles (#15931). If we can do that, one thing we'd want to explore is finding a good balance for optimization levels between build times and execution times that can make the overall operation faster, whether its faster build scripts and proc-macros or faster test runs.

I was thinking of package table as "per-package configuration for the workspace", and build-override as "build-dependencies-only configuration for the workspace".
So build-override.package reads nicely as "per-package configuration for build-dependencies in the workspace", doesn't seem to be too contrived.

Fully understand where you are coming from on this. I'm more concerned about the structure of the data model. Maybe its something we'd do but I personally am hesitant.