Write Microbenchmark Framework with Warmup and Statistics

Warmup, Iterations, and Statistics

Google Benchmark, LLVM's llvm-exegesis, and Julia's @benchmark all share the same skeleton: warmup, many timed iterations, then aggregate statistics. Without warmup, first-touch page faults and cold icache skew the first samples. Without variance reporting, a lucky 5% speedup from turbo noise looks like a win.

Framework Structure

Separate setup from the timed body. Loop enough iterations that timer overhead is below 1% of measured time. Track min, median, and standard deviation.

cCopy
Loading…

• Compile with -O2 or -O3 unless you explicitly test debug behavior
• Prevent dead-code elimination with volatile or blackhole sinks
• Reset state between iterations when the work mutates inputs
• Log environment: CPU model, governor, ASLR on/off

Working Through the Exercise

Keep the relevant documentation open while you implement. When your output disagrees with the reference, trace one failing case by hand before changing random lines.

• Trace first: walk one failing input step by step
• Read the manual: skim the official doc for the mechanism named in this task
• Compare baselines: save measurements from the prior step so speedups are honest

Why for this exercise

You will build a microbenchmark harness with configurable warmup and iteration counts plus mean/min output. This exercise requires honest statistics so you can tell signal from measurement noise.