`Ensure that your project references 'Aspire.Hosting.AppHost'` testing builder error is misleading when package is a transitive dependency

[afscrome]

Is there an existing issue for this?

• I have searched the existing issues

Describe the bug

If you create a test builder via DistributedApplicationTestingBuilder.Create(); in a project that has an indirect transitive reference to Aspire.Hosting.AppHost, the test host fails with the error Ensure that your project references 'Aspire.Hosting.AppHost', even though the project does have a reference to that assembly.

Fix is to ensure you have a direct dependency, not a transitive one

Expected Behavior

The test host should either support the transitive dependency, or the error message should explicitly call out that you need a direct (non transitive) dependency)

Steps To Reproduce

dotnet new aspire-starter --test-framework nunit

Add the following test

    [Test]
    public async Task BuilderTest()
    {
        var builder = DistributedApplicationTestingBuilder.Create();
        await using var app = await builder.BuildAsync().WaitAsync(DefaultTimeout);
    }

run dotnet test

Exceptions (if any)

System.InvalidOperationException : No application host assembly was found. Ensure that your project references 'Aspire.Hosting.AppHost'.
  Stack Trace:
     at Aspire.Hosting.Testing.DistributedApplicationTestingBuilder.TestingBuilder.<CreateInnerBuilder>g__FindApplicationAssembly|3_2() in /_/src/Aspire.Hosting.Testing/DistributedApplicationTestingBuilder.cs:line 315
   at Aspire.Hosting.Testing.DistributedApplicationTestingBuilder.TestingBuilder.<>c__DisplayClass3_0.<CreateInnerBuilder>b__0(DistributedApplicationOptions applicationOptions, HostApplicationBuilderSettings hostBuilderOptions) in /_/src/Aspire.Hosting.Testing/DistributedApplicationTestingBuilder.cs:line 290
   at Aspire.Hosting.Testing.TestingBuilderFactory.System.IObserver<System.Collections.Generic.KeyValuePair<System.String,System.Object>>.OnNext(KeyValuePair`2 value) in /_/src/Aspire.Hosting.Testing/DistributedApplicationEntryPointInvoker.cs:line 282
   at System.Diagnostics.DiagnosticListener.Write(String name, Object value)
   at System.Diagnostics.DiagnosticSource.Write[T](String name, T value)
   at Aspire.Hosting.DistributedApplicationBuilder.LogBuilderConstructing(DistributedApplicationOptions appBuilderOptions, HostApplicationBuilderSettings hostBuilderOptions) in /_/src/Aspire.Hosting/DistributedApplicationBuilder.cs:line 484
   at Aspire.Hosting.DistributedApplicationBuilder..ctor(DistributedApplicationOptions options) in /_/src/Aspire.Hosting/DistributedApplicationBuilder.cs:line 134
   at Aspire.Hosting.DistributedApplicationBuilder..ctor(String[] args) in /_/src/Aspire.Hosting/DistributedApplicationBuilder.cs:line 94
   at Aspire.Hosting.Testing.TestingBuilderFactory.CreateBuilder(String[] args, Action`2 onConstructing) in /_/src/Aspire.Hosting.Testing/DistributedApplicationEntryPointInvoker.cs:line 230
   at Aspire.Hosting.Testing.DistributedApplicationTestingBuilder.TestingBuilder.CreateInnerBuilder(String[] args, Action`2 configureBuilder) in /_/src/Aspire.Hosting.Testing/DistributedApplicationTestingBuilder.cs:line 288
   at Aspire.Hosting.Testing.DistributedApplicationTestingBuilder.TestingBuilder..ctor(String[] args, Action`2 configureBuilder) in /_/src/Aspire.Hosting.Testing/DistributedApplicationTestingBuilder.cs:line 281
   at Aspire.Hosting.Testing.DistributedApplicationTestingBuilder.Create(String[] args, Action`2 configureBuilder) in /_/src/Aspire.Hosting.Testing/DistributedApplicationTestingBuilder.cs:line 135
   at Aspire.Hosting.Testing.DistributedApplicationTestingBuilder.Create(String[] args) in /_/src/Aspire.Hosting.Testing/DistributedApplicationTestingBuilder.cs:line 120
   at aspirequickcotnainer.Tests.WebTests.ShortLivedContainer() in S:\aspirequickcotnainer\aspirequickcotnainer.Tests\WebTests.cs:line 13
   at NUnit.Framework.Internal.TaskAwaitAdapter.GenericAdapter`1.GetResult()
   at NUnit.Framework.Internal.AsyncToSyncAdapter.Await[TResult](Func`1 invoke)
   at NUnit.Framework.Internal.AsyncToSyncAdapter.Await(Func`1 invoke)
   at NUnit.Framework.Internal.Commands.TestMethodCommand.RunTestMethod(TestExecutionContext context)
   at NUnit.Framework.Internal.Commands.TestMethodCommand.Execute(TestExecutionContext context)
   at NUnit.Framework.Internal.Execution.SimpleWorkItem.<>c__DisplayClass3_0.<PerformWork>b__0()
   at NUnit.Framework.Internal.ContextUtils.<>c__DisplayClass1_0`1.<DoIsolated>b__0(Object _)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
   at NUnit.Framework.Internal.ContextUtils.DoIsolated(ContextCallback callback, Object state)
   at NUnit.Framework.Internal.ContextUtils.DoIsolated[T](Func`1 func)
   at NUnit.Framework.Internal.Execution.SimpleWorkItem.PerformWork()

.NET Version info

No response

Anything else?

No response

[eerhardt]

I think DistributedApplicationTestingBuilder.Create() is a bit of a pit of failure. It means "use the current app as the AppHost", where the "current app" is the test app. It tries to find the app assembly by walking the current stack:

aspire/src/Aspire.Hosting.Testing/DistributedApplicationTestingBuilder.cs

Lines 301 to 313 in 25c6c40

	static Assembly FindApplicationAssembly()
	{
	// Walk the stack trace to find the first assembly that has the 'dcpclipath' metadata attribute.
	// This will be selected as the application host assembly. DCP is necessary to launch the application.
	var stackTrace = new StackTrace();
	foreach (var stackFrame in stackTrace.GetFrames())
	{
	var asm = stackFrame.GetMethod()?.DeclaringType?.Assembly;
	if (asm is not null && GetDcpCliPath(asm) is { Length: > 0 })
	{
	return asm;
	}
	}

It assumes that the current app is an AppHost, but that isn't the case by default in tests.

Maybe the error message here needs to be updated? Or the template changed so the tests projects directly reference Aspire.Hosting.AppHost.

Another weird thing here is that the DCP paths get generated into any project that directly references Aspire.Hosting.AppHost, even if it doesn't set IsAspireHost=true. I would have thought it would only get an assembly attribute when the property is set.

cc @ReubenBond @DamianEdwards

[DamianEdwards]

Or the template changed so the tests projects directly reference Aspire.Hosting.AppHost.

The templates don't include a test that uses DistributedApplicationTestingBuilder.Create() though. Honestly I'm not even sure what that scenario even is? Using the overloads of Create that accept an entry type/assembly are the only ones I've even used.

Updating the error message and reviewing the doc comments to make this clearer seems like goodness.

[ReubenBond]

Honestly I'm not even sure what that scenario even is?

The scenario is testing with ad-hoc app hosts. Eg, creating a subset of your infrastructure for testing purposes rather than needing to create a new AppHost project for various scenarios or using config switches to exclude unwanted resources. We do this in the eShop functional tests, but in those tests we explicitly reference Aspire.Hosting.AppHost & set <IsAspireHost>true</IsAspireHost> currently.

The functionality needs Aspire.Hosting.AppHost so that it can find DCP and to gain the Aspire capability for debugging. If it isn't working transitively, we may need to add a buildTransitive folder in the Aspire.Hosting.AppHost so that transitive packages references behave the same way as direct references.

[DamianEdwards]

@ReubenBond I'm not seeing where in the example you linked to, the DistributedApplicationTestingBuilder.Create() method is being called? I see the DistributedApplication.Create() method being used to create an ad-hoc app host in the fixture, but where is the testing builder created?

[ReubenBond]

@DamianEdwards I'm pointing out the scenario: creating a builder in test for ad-hoc app hosts. That eShop code predates v9.1. We use this Create API in our Hosting tests via a wrapper type, TestDistributedApplicationBuilder

[DamianEdwards]

@ReubenBond ok thanks. So this is a new API to support the scenario of using ad-hoc app hosts from tests that requires a direct dependency on Aspire.Hosting.AppHost, yes? If that's the case sems a doc comment and exception message improvement would address this feedback?

[ReubenBond]

Sounds good to me. Ideally the user wouldn't have to do anything, and it would Just Work. We could make it work by changing how Aspire.Hosting.AppHost is packaged so that transitive references behave identically to direct references. I'm not sure why transitive references should ever behave differently from direct references. Do you know, @DamianEdwards?

[DamianEdwards]

Yeah at the moment the build targets in Aspire.Hosting.AppHost don't apply when referenced transitively but that can be changed if it doesn't introduce any other issues.

[afscrome]

There's a bit more to this - adding a direct reference to Aspire.Hosting.AppHost works if you have a ProjectReference to an app host project. If you don't, then you also need to add <Sdk Name="Aspire.AppHost.Sdk" Version="9.1.0" /> to your test project for the above error to go away.