TypeLoadException is thrown when using third party exporter

[filzrev]

When using latest master branch code.
BenchmarkDotNet.Samples project throw following exception.

Unhandled exception. System.TypeLoadException: Method 'ExportAsync' in type 'VSDiagnosticsExporter' from assembly 'VSDiagnostics.BenchmarkDotNet, Version=18.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' does not have an implementation.
   at Microsoft.VSDiagnostics.VSDiagnosticsDiagnoser.get_Exporters()
   at BenchmarkDotNet.Configs.ImmutableConfigBuilder.GetExporters(IEnumerable`1 exporters, ImmutableHashSet`1 uniqueDiagnosers, IList`1 configAnalyse) in C:\Projects\GitHub\thirdparty\BenchmarkDotNet\src\BenchmarkDotNet\Configs\ImmutableConfigBuilder.cs:line 129
   at BenchmarkDotNet.Configs.ImmutableConfigBuilder.Create(IConfig source) in C:\Projects\GitHub\thirdparty\BenchmarkDotNet\src\BenchmarkDotNet\Configs\ImmutableConfigBuilder.cs:line 46
   at BenchmarkDotNet.Running.BenchmarkConverter.GetFullTypeConfig(Type type, IConfig config) in C:\Projects\GitHub\thirdparty\BenchmarkDotNet\src\BenchmarkDotNet\Running\BenchmarkConverter.cs:line 98
   at BenchmarkDotNet.Running.BenchmarkConverter.MethodsToBenchmarksWithFullConfig(Type type, MethodInfo[] benchmarkMethods, IConfig config) in C:\Projects\GitHub\thirdparty\BenchmarkDotNet\src\BenchmarkDotNet\Running\BenchmarkConverter.cs:line 44
   at BenchmarkDotNet.Running.BenchmarkConverter.TypeToBenchmarks(Type type, IConfig config) in C:\Projects\GitHub\thirdparty\BenchmarkDotNet\src\BenchmarkDotNet\Running\BenchmarkConverter.cs:line 26
   at BenchmarkDotNet.Running.TypeFilter.<>c__DisplayClass1_0.<Filter>b__0(Type type) in C:\Projects\GitHub\thirdparty\BenchmarkDotNet\src\BenchmarkDotNet\Running\TypeFilter.cs:line 65
   at System.Linq.Enumerable.SelectListIterator`2.MoveNext()
   at System.Linq.Enumerable.WhereEnumerableIterator`1.ToArray()
   at BenchmarkDotNet.Running.TypeFilter.Filter(IConfig effectiveConfig, IEnumerable`1 types) in C:\Projects\GitHub\thirdparty\BenchmarkDotNet\src\BenchmarkDotNet\Running\TypeFilter.cs:line 64
   at BenchmarkDotNet.Running.BenchmarkSwitcher.RunWithDirtyAssemblyResolveHelper(String[] args, IConfig config, Boolean askUserForInput, CancellationToken cancellationToken) in C:\Projects\GitHub\thirdparty\BenchmarkDotNet\src\BenchmarkDotNet\Running\BenchmarkSwitcher.cs:line 162
   at BenchmarkDotNet.Running.BenchmarkSwitcher.RunAsync(String[] args, IConfig config, CancellationToken cancellationToken) in C:\Projects\GitHub\thirdparty\BenchmarkDotNet\src\BenchmarkDotNet\Running\BenchmarkSwitcher.cs:line 111
   at BenchmarkDotNet.Engines.BenchmarkDotNetSynchronizationContext.ExecuteUntilComplete[T](ValueTask`1 valueTask) in C:\Projects\GitHub\thirdparty\BenchmarkDotNet\src\BenchmarkDotNet\Engines\BenchmarkSynchronizationContext.cs:line 101
   at BenchmarkDotNet.Engines.BenchmarkSynchronizationContext.ExecuteUntilComplete[T](ValueTask`1 valueTask) in C:\Projects\GitHub\thirdparty\BenchmarkDotNet\src\BenchmarkDotNet\Engines\BenchmarkSynchronizationContext.cs:line 29
   at BenchmarkDotNet.Running.BenchmarkSwitcher.Run(String[] args, IConfig config) in C:\Projects\GitHub\thirdparty\BenchmarkDotNet\src\BenchmarkDotNet\Running\BenchmarkSwitcher.cs:line 73
   at BenchmarkDotNet.Samples.Program.Main(String[] args) in C:\Projects\GitHub\thirdparty\BenchmarkDotNet\samples\BenchmarkDotNet.Samples\Program.cs:line 27

It seems be occurred because IExporter interface has breaking changes on recent commit.
Is it able to keep interface backward compatibility?

Otherwise, third party exporter dependency needs updated after BenchmarkDotNet 0.16.x is released.

[adamsitnik]

Great catch @filzrev !

Otherwise, third party exporter dependency needs updated after BenchmarkDotNet 0.16.x is released.

With such a breaking change we need to bump the version to at least to 0.17

[timcassell]

Yep, this is a known issue. Back-compat is not really possible after the async refactor.

Otherwise, third party exporter dependency needs updated after BenchmarkDotNet 0.16.x is released.

Correct.

With such a breaking change we need to bump the version to at least to 0.17

No we don't. 0.16 is still unreleased.

[adamsitnik]

No we don't. 0.16 is still unreleased.

Thanks, I was not aware of that.

[adamsitnik]

cc @karpinsn BDN is introducing breaking changes that VS profiler package will need to handle

[karpinsn]

Thanks for the heads up, we will work on getting a fix out. Do we have a timeline for the 0.16 release?

[karpinsn]

I have a fix, turned out to be pretty straightforward and works with the old and new version. Are there any more planned changes to the API for 0.16? If not I can push out a new package this week / next but I only want to do so assuming there won't be other breaking changes coming. Otherwise I can hang onto the fix until we have a release date and know there won't be breaking changes.

[timcassell]

There will be more breaking changes to toolchains, but no other changes are planned for diagnosers or exporters unless any issues are found.

[filzrev]

Is seems BenchmarkDotNet.Samples.Maui project also causing build error by breaking API change. (ExportToLog is removed, replaced to internal ExportToLogAsync)

I'm tryed to fix build errors.
But samples/BenchmarkDotNet.Maui.slnx failed to build due to some components/workload is missing on my environment.