Reuse moves points right

Locality transforms raise operations per byte, so the point slides right along the slope. Tiling keeps a B x B block on chip and reuses each element about B times.

Doubling the tile edge roughly doubles intensity. The kernel climbs the slope toward the ridge without touching any ceiling.

Tiled GEMM (FP32, N = 512, multiply-add counts as 2 FLOP):
  ops = 2N^3 = 2 x 134,217,728 = 268,435,456 FLOP
  DRAM bytes ~= 8N^3 / B + 4N^2 (A/B refetch plus C writeback)
  B = 16: bytes = 67,108,864 + 1,048,576 = 68,157,440 B
    AI = 268,435,456 / 68,157,440 ~= 3.94 FLOP/B
    cap = min(100, 3.94 x 20 ~= 78.8) ~= 79 GFLOP/s, memory-bound
  B = 32: bytes = 33,554,432 + 1,048,576 = 34,603,008 B
    AI = 268,435,456 / 34,603,008 ~= 7.76 FLOP/B
    cap = min(100, 7.76 x 20 ~= 155) = 100 GFLOP/s, compute-bound

Raising ceilings instead

Some kernels cannot reuse more, so the fix raises a roof. More bandwidth steepens the slope, and more ALUs lift the flat roof.

Peak helps only points already right of the ridge. Slope-bound kernels need reuse or bandwidth, never wider ALUs.

Blocking and prefetch tradeoffs

Each locality tool moves a different term. Tiling shifts intensity, while prefetch raises achieved bandwidth use by overlapping misses.

Gap to ceiling

Report the gap before and after: cap minus measured throughput. The B = 16 tile caps at about 79, so a kernel at 60 GFLOP/s leaves 19 on the table.

Oversized tiles spill out of on-chip storage and traffic jumps back up. The point then slides left again, which is the sign the tile is too big.

Related topics

• Accelerators for the architect