Memory Coalescing: The Key to GPU Bandwidth

Coalescing Combines Warp Requests

A modern GPU can issue hundreds of billions of floating-point operations per second, but global memory bandwidth is typically 1-2 terabytes per second. With thousands of concurrent threads, each thread gets only a thin slice of that bandwidth unless accesses are merged efficiently.

Memory coalescing is the mechanism that combines requests from threads in the same warp into as few cache-line transactions as possible. Coalescing is about transaction count, not bytes transferred.

Three Access Patterns

Consider 32 threads in a warp, each reading a 4-byte integer with 128-byte cache lines:

cCopy
Loading…

For example, in pattern (a), threads 0-31 read addresses base+0 through base+124, all within one 128-byte line. In pattern (b), each thread touches a different line despite requesting the same total bytes.

• Coalesced: adjacent threads read adjacent addresses
• Strided: large stride (column-major matrix reads) destroys coalescing
• Misaligned: unaligned base splits one logical pattern across two lines
• Fix: read row-major, stage through shared memory to transpose

For This Exercise

You will implement a transaction counter for coalesced, strided, and misaligned patterns, then compare row-major vs column-major matrix reads. This task requires you to print a summary table with efficiency percentages. The analysis you build here is the same reasoning CUDA kernel authors apply when nvcc warns about uncoalesced global loads.