loading experience

Architecture & quality

BenchmarkDotNet: measuring the performance of .NET code

Stop guessing: how to write reliable benchmarks and read the results without being fooled.

BenchmarkDotNet: measuring the performance of .NET code

Measuring time with a Stopwatch in a loop gives numbers distorted by the JIT, caches and the garbage collector. BenchmarkDotNet handles warm-up, repetitions and statistics for you.

Warm-up and repetitions remove the noise that skews hand-made measurements.
Warm-up and repetitions remove the noise that skews hand-made measurements.

A complete benchmark

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;

BenchmarkRunner.Run<ConcatBenchmark>();

[MemoryDiagnoser]
public class ConcatBenchmark
{
    [Params(10, 1000)]
    public int N;

    [Benchmark(Baseline = true)]
    public string WithPlus()
    {
        var s = "";
        for (var i = 0; i < N; i++) s += i;
        return s;
    }

    [Benchmark]
    public string WithStringBuilder()
    {
        var sb = new System.Text.StringBuilder();
        for (var i = 0; i < N; i++) sb.Append(i);
        return sb.ToString();
    }
}
dotnet run -c Release

Reading the results

  • Mean: average time per operation.
  • Error and StdDev: how stable the measurements are; if they are large, the result is unreliable.
  • Ratio: comparison with the method marked as baseline.
  • Allocated: memory allocated per operation, often more important than time.

Comparing .NET versions

The same benchmark can run on several runtimes, for example with the --runtimes net8.0 net10.0 command-line option: it is the most concrete way to evaluate an upgrade.

Golden rules

Always in Release configuration, never with a debugger attached, with realistic data and measuring what really matters to the user. Only optimise code that a profiler or a benchmark identifies as a bottleneck.

Comments (0)

No comments yet.

Leave a comment