How GPUs hide memory latency by juggling 64 threads at once
In short: Processing massive data sets requires more than just raw speed; it requires a clever way to handle waiting times. AMD's GCN architecture achieves this by grouping threads into 'wavefronts,' allowing the hardware to swap tasks instantly when one group hits a memory bottleneck.
Unlike traditional CPUs that execute instructions sequentially, GPUs like those using AMD’s Graphics Core Next (GCN) architecture are designed for explicit data-parallelism. In this model, programs are organized into parallel threads where each thread handles a different element of an input dataset. To maximize efficiency, GCN groups these threads into units of 64 called 'wavefronts.' Within a wavefront, every thread shares a single program counter and executes in lock-step, allowing the hardware to issue 64 operations per instruction fetch while significantly reducing power and area requirements.
The core challenge for GPUs is memory latency—the time it takes to retrieve data from system or graphics memory. To solve this, GCN attributes multiple wavefronts (up to 10) to each SIMD Vector Unit (SIMD-VU). When one wavefront stalls while waiting for a memory result, the CU Scheduler quickly assigns a different wavefront to the unit. This 'hiding' of latency ensures that the hardware remains productive even when individual threads are blocked. While GCN links speed to wavefront-parallelism, AMD's older TeraScale framework required software to identify separate tasks inside one wavefront.
The physical layout of a GCN Compute Unit (CU) supports this complex orchestration. One CU combines 64 shader processors with four texture mapping units and four SIMD-VUs. Each SIMD-VU manages its own private memory for scalar registers (holding 4 bytes each) and vector registers (representing sets of 64 numbers). By distributing these registers across different wavefronts, the hardware can maintain a high occupancy of active tasks, ensuring that the GPU spends less time waiting and more time computing.
Opens the Intellect+ feed with this card first. Swipe up for the next one.
Sources
- Summary by Intellect · source: Graphics Core Next
To cite this card, link to https://intellectplus.co.uk/app/science-graphics-core-next.
How we source, check and correct cards: Editorial standards.
General information, not professional advice. Spotted an error? Flag the card or see corrections and complaints.
Intellect Plus is a visual educational discovery platform for exploring fascinating facts and ideas. Brain Food, Not Brain Rot. What is Intellect Plus?
Related
More in Found by OrchestratorBot · More in Science · Latest · A–Z · All topics
