Skip to content

Handle assemblies loaded via a stream #1443

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

jeremyosterhoudt
Copy link
Contributor

Hello,

We are using a test framework called Gauge (https://gauge.org/) to do regression testing and wanted to roll performance tests in with those regression tests. In our case we want to use Gauge steps to control and kick off different BenchmarkDotNet benchmarks (so we have a common top level framework for all of our regression tests).

One issue we have run into is Gauge loads assemblies via streams, this causes the assembly location to be an empty string when running against net core/net standard. More information about the assembly location being empty in this case can be found in Microsoft's documentation (https://docs.microsoft.com/en-us/dotnet/api/system.reflection.assembly.location).

This PR provides a way to handle that scenario by providing a default location for the project build artifacts. Other non-CsProjGenerators seem to do this by default so we have only updated this generator.

It's possible this could be handled a different way, but we've found this solution is working well for us.

Any feedback is welcomed.

@dnfclas
Copy link

dnfclas commented Apr 29, 2020

CLA assistant check
All CLA requirements met.

Copy link
Member

@adamsitnik adamsitnik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @jeremyosterhoudt

Thank you for your PR, it looks good to me.

One thing that I am not sure of is the next part of generating the project.

To be able to build the auto-generated project it must reference the project that defines benchmarks.

We search for the project file using this method. If we can't find it, BDN throws.

After we find the path to .csproj we fill the project template:

<ProjectReference Include="$CSPROJPATH$" />

Before I merge your PR I would like to make sure that this part works for you scenario. Could you please verify that? One way to test it would be to edit your benchmarks and reference BenchmarkDotNet.csproj instead of the NuGet package. Other would be to run dotnet pack -c Release in BenchmarkDotNet\src\BenchmarkDotNet\, take the generated .nuspec file, move it to some folder and add this folder to the nuget.config file of your project and just install it from your IDE.

Thank you,
Adam

//Simulate loading an assembly from a stream
var benchmarkDotNetAssembly = typeof(MockFactory.MockBenchmarkClass).GetTypeInfo().Assembly;
var streamLoadedAssembly = Assembly.Load(File.ReadAllBytes(benchmarkDotNetAssembly.Location));
var assemblyType = streamLoadedAssembly.GetRunnableBenchmarks().Select(type => type).FirstOrDefault();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

smart way to mock it! 👍

@jeremyosterhoudt
Copy link
Contributor Author

@adamsitnik

Thanks for the feedback.

I followed the instructions to generate a nuget spec using dotnet pack -c Release and used that spec in my project. Everything is running as expected.

Screen Shot 2020-04-30 at 11 21 57 AM

Thanks,
Jeremy

Copy link
Member

@adamsitnik adamsitnik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jeremyosterhoudt thank you for the verification!

@adamsitnik adamsitnik merged commit 1efd5e1 into dotnet:master Apr 30, 2020
@adamsitnik adamsitnik added this to the v0.13.0 milestone May 3, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants