Identify and Avoid Common Benchmarking Pitfalls

False Wins and Silent Lies

Dead-code elimination, constant folding, comparing debug vs release builds, and measuring only one iteration are the classic ways microbenchmarks lie. LLVM and GCC will delete loops whose results are unused. Turbo Boost makes the first timed run faster than the tenth unless you pin frequency or run long enough.

Pitfalls Checklist

Always consume results. Match compiler flags across A/B tests. Warm up branch predictors and caches. Run on idle hardware or pin cores.

cCopy
Loading…

• Do not compare -O0 against -O3 and call it an optimization
• Background cron jobs add variance; check uptime and load average
• Tiny inputs fit in L1 and hide memory effects visible at production size
• Print assembly with objdump -d when numbers look impossible

Working Through the Exercise

Keep the relevant documentation open while you implement. When your output disagrees with the reference, trace one failing case by hand before changing random lines.

• Trace first: walk one failing input step by step
• Read the manual: skim the official doc for the mechanism named in this task
• Compare baselines: save measurements from the prior step so speedups are honest

Why for this exercise

You will identify at least three benchmarking mistakes in provided snippets and fix them. This exercise asks you to demonstrate one pitfall (such as DCE) and show corrected timing.