#521 added support for async GlobalSetup, but it appears that InProcess toolchain does not wait for the async setup method to finish. Simple reproducer:
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
using System.Threading.Tasks;
namespace AsyncGlobalSetupRepro
{
[InProcess]
public class MyBenchmarkClass
{
private string _foo;
[GlobalSetup]
public async Task MyAsyncSetup()
{
await Task.Delay(1000);
_foo = "hello";
}
[Benchmark]
public int MyBenchmark() => _foo.Length; // NullReferenceException!
}
public class Program
{
public static void Main(string[] args)
{
BenchmarkRunner.Run(typeof(MyBenchmarkClass));
}
}
}It looks like MyAsyncSetup gets invoked as a regular method here, and then JITting starts immediately without waiting for it to finish:
|
engineParameters.GlobalSetupAction?.Invoke(); // whatever the settings are, we MUST call global setup here, the global cleanup is part of Engine's Dispose |
#521 added support for async GlobalSetup, but it appears that InProcess toolchain does not wait for the async setup method to finish. Simple reproducer:
It looks like MyAsyncSetup gets invoked as a regular method here, and then JITting starts immediately without waiting for it to finish:
BenchmarkDotNet/src/BenchmarkDotNet/Engines/EngineFactory.cs
Line 28 in e7ff4ce