Sub-packages aren't well supported (julia 1.10 feature setting `manifest=` in Project.toml): precompiling more than needed

[NHDaly]

It seems like TestEnv doesn’t know about sub-packages, so if you edit a small package, and then restart the repl, the first time your run TestEnv.activate("SubPackage"), TestEnv will attempt to build the whole top-level project.

---

Our codebase is organized as a monorepo, with lots of subpackages in the same repository. Those subpackages all share the top-level Manifest, which is the recommended way to do subpackages starting in julia 1.10. To do that, they set this field in their Project.toml: manifest = "../../Manifest.toml".

For example (abbreviated):

name = "RelationalAIBase"
uuid = "88ef06d0-***"
manifest = "../../Manifest.toml"
version = "0.1.0"

[deps]
ExceptionUnwrapping = "460bff9d-24e4-43bc-9d9f-a8973cb893f4"
RAI_Metrics = "e3bce84f-ddf9-455c-86ea-d4978b856730"

[extras]
JET = "c3a54625-cd67-489e-a8e7-0a5a0ff4e31b"
ReTestItems = "817f1d60-ba6b-4fd5-9520-3cf149f6a823"
Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"

[targets]
test = ["JET", "ReTestItems", "Test"]

In this example, RAI_Metrics is also a subpackage.

---

The issue here is that TestEnv.activate("RelationalAIBase") ends up precompiling the whole top-level Project.toml and Manifest.toml, which can take 2-3 minutes.

This is particularly annoying if you are editing a leaf package that most of the codebase depends on, because such a package should have the best cycle-times (since it has no deps), yet you have to build all of the top-level project just to test it.

Note that Pkg already handles these correctly: If you do using SubPackage, it only precompiles that package and its dependencies.
Similarly, if you do Pkg.test("SubPackage") it only precompiles only:

• that package
• its dependencies
• its test-only dependencies

So we need to apply that same fix to TestEnv to replicate this.

Links:

• Added support for manifest field in Pkg: Respect manifest if provided in Project.toml JuliaLang/Pkg.jl#3303
• ReTestItems' use of TestEnv: https://github.com/JuliaTesting/ReTestItems.jl/blob/79cb313194a6eab07f7529bedc25e5b66403b141/src/ReTestItems.jl#L354-L355

[NHDaly]

A discovery I've made:

Even though Pkg.test("SubPackage") and using SubPackage both work correctly already, it looks like Pkg.instantiate() does not! 💡

julia> using Pkg

julia> Pkg.activate("packages/RAI_Metrics/")
  Activating project at `~/work/raicode2/packages/RAI_Metrics`

julia> using RAI_Metrics
Precompiling RAI_Metrics
  1 dependency successfully precompiled in 2 seconds. 46 already precompiled.

julia> Pkg.instantiate()
Precompiling project...
  Progress [============>                            ]  12/43

So i think the issue actually comes from this Pkg.instantiate(ctx) line:

TestEnv.jl/src/julia-1.9/common.jl

Lines 21 to 27 in 25d9afb

	function ctx_and_pkgspec(pkg::AbstractString)
	pkgspec = deepcopy(PackageSpec(pkg))
	ctx = Context()
	isinstalled!(ctx, pkgspec) || throw(TestEnvError("$pkg not installed 👻"))
	Pkg.instantiate(ctx)
	return ctx, pkgspec
	end

Removing it does make this go away, but then no precompilation happens.
I want it to precompile only the exact packages in the sandbox_ctx, so something still needs to change.

[NHDaly]

Actually, changing that line to

    Pkg.instantiate(ctx; allow_autoprecomp = false) # do precomp later within sandbox

seems like a good idea. Copied that from here:
https://github.com/JuliaLang/Pkg.jl/blob/27c1b1ee5cf15571eb5e54707e812d646ac1dde3/src/Operations.jl#L2217

But we still need to find what to change to precompile the sandbox_ctx

[NHDaly]

😊 Aha, it makes sense: i think we just need this line at the end:

    # Now that we have set up the sandbox environment, precompile all its packages:
    # (Reconnect the `io` back to the original context so the caller can see the
    # precompilation progress.)
    Pkg.precompile(temp_ctx; io=ctx.io)

I'll start a PR!