GPU Threads vs CPU Threads: Warps and Wavefronts

CPU Threads vs GPU Warps

A CPU thread is an independent execution context with its own program counter, registers, and stack. Context switching is expensive because the OS must save and restore the full register file. Modern CPUs run tens of threads across a handful of cores.

A GPU thread is much lighter. On NVIDIA hardware, 32 threads form a warp; on AMD, 64 threads form a wavefront. All threads in a warp execute the same instruction on different data. This is Single Instruction, Multiple Thread (SIMT) execution.

Divergence and Memory Patterns

The critical difference is divergence. If threads in the same warp take different branches, the hardware serializes them:

cCopy
Loading…

For example, a warp where 16 threads take branch A and 16 take branch B runs both paths sequentially, halving throughput. Memory access matters too: if thread i reads address base + i * 4, the warp issues one coalesced transaction. If each thread reads base + i * 1024, the warp may issue up to 32 separate transactions.

• Warp (NVIDIA): 32 threads in lockstep
• Wavefront (AMD): 64 threads, same idea
• SM: Streaming Multiprocessor that schedules warps

For This Exercise

You will implement a warp simulation that shows branch divergence in two serialized phases and compares coalesced vs scattered memory access. This task requires you to count transactions and explain why GPU threads are not scheduled like CPU threads. Every later GPU architecture exercise in this subtrack assumes you think at warp granularity, not individual thread granularity.