Overhead compensation is a key advantage of NProfiler's modern profiling engine. It significantly improves the accuracy and reliability of the profiling results.
Overhead compensation removes the distortions caused by the profiler (known as profiler overhead) from the performance data, allowing you to see how your code would have performed without the profiler. If a profiler does not carefully remove its overhead, the performance data is often completely misleading, and you end up trying to optimize the wrong parts of your code.
Most .NET profilers ignore these distortions because estimating and removing them is quite complex. For example, some profilers we tested reported that a method took 70% of the time, when in reality it was closer to 20%. Unfortunately, it is not immediately apparent that the execution times displayed are incorrect. You simply trust the profiler and waste time optimizing the wrong methods.
Fortunately, with NProfiler, you are on the safe side. It has excellent overhead compensation and can estimate and subtract even massively distorting profiler overhead to provide highly realistic performance data.
Let's take a look at the following C# code example. It generates a list of 10,000,000 random numbers and sorts them. You can use it to test various profilers for correctness.
var random = new Random(0);
var list = new List<int>();
for (int i = 0; i < 10_000_000; i++)
{
list.Add(random.Next());
}
list.Sort();
If we run the code without (!) a profiler and use the Stopwatch class to determine the actual execution times, we find that filling the list takes about 20% of the time, while the sort call takes 80% of the time.
When we profile the code using the tracing/instrumentation modes of various commercial and free profilers, we often get completely wrong times. Even expensive profilers want us to believe that filling the list takes the most time (20% in reality), while sorting takes almost no time (80% in reality).
The size of the error depends on the profiling settings. If you choose a lighter/less detailed profiling mode, such as sampling, less overhead is added and the measurement errors are smaller. If you choose a more detailed profiling mode that uses instrumentation, the errors are serious with most profilers.
You can download trial versions of various profilers and test them with our code sample.
Code can be profiled in two different ways: either by sampling or by tracing/instrumentation. NProfiler supports both profiling modes.
Sampling means that the profiler pauses all threads several hundred times per second and takes snapshots of the call stacks. Based on these samples, the profiler then estimates the time spent on different methods and lines of code.
Sampling is very fast and causes only minor profiler overhead. Therefore, overhead compensation is less critical for sampling profilers, so we will not discuss it further here.
The disadvantage of sampling is that it is less detailed. For example, you cannot collect hit counts or performance data for JIT compilation, module loading, etc. Due to these limitations, profilers also support tracing/instrumentation, which is more detailed.
Tracing/instrumenting profilers inject instructions at the entry and exit points of methods and between lines of code. The inserted code performs time measurements and aggregation. This allows the profiler to collect “real” times instead of statistical estimates, as well as hit counts and event-based performance data for JIT compilation, module loading, etc.
The disadvantage of instrumentation is that the inserted code slows down the profiled application and causes high profiler overhead. Therefore, overhead compensation is critical for tracing/instrumenting profilers.
Let's assume that a profiler needs to insert ten lines of additional code for each profiled line of code in order to collect line-level data. As a result, the profiled application runs ten times slower, and the measured times are inflated by about 1,000% if the profiler does not subtract the overhead from the results.
It gets even worse. Depending on the settings chosen, profilers often instrument only the developer's code and not the .NET Framework methods, to improve performance. The profiled application would become extremely slow if all .NET Framework methods were instrumented, and we typically don't need line-level data for framework methods.
Unfortunately, this optimization leads to an uneven distribution of profiler overhead. Due to instrumentation, our custom code runs ten times slower, while non-instrumented .NET Framework methods (such as the sort call in the example above) run at full speed. The performance data displayed reflects these massive distortions if a profiler does not compensate for them—and most profilers do not. For this reason, .NET profilers display incorrect data.
Fortunately, NProfiler is quite good at estimating and subtracting the execution time of the injected code. On average, its performance data should be much more accurate than that of other .NET profilers.
However, in some cases, NProfiler's overhead compensation may not work correctly. Due to the superscalar architecture of modern CPUs, it is difficult to correctly compensate for the overhead of the profiler. There may be cases where NProfiler subtracts too little or too much overhead. Please let us know about such cases so that we can investigate them.
As a best practice, we recommend profiling only those namespaces that are of interest to you. This prevents massive slowdowns in the profiled application and reduces the overhead of the profiler, making overhead compensation more robust. To do this, set “Profiled Methods” to “Custom” when configuring the profiling session.