Systolic Arrays: Tricky Q&A

[INT][ACCEL][SYSTOLIC-ARRAYS]

Q: Why can a larger systolic array deliver lower real throughput for some models?

A:
Because peak MAC count scales with area, but real workloads may not map cleanly to larger dimensions. Small or irregular tiles increase edge idling, and memory bandwidth may become the limiting factor before compute saturates. Without matching tiling and feed bandwidth, bigger arrays can show lower utilization and only modest wall-clock gains.

FOLLOW-UP TRAP: Assuming array dimension growth automatically yields proportional throughput gains.

[INT][ACCEL][SYSTOLIC-ARRAYS]

Q: When is weight-stationary preferable to output-stationary?

A:
Weight-stationary is usually better when filter reuse is high and weight-fetch energy dominates, such as many batched inference scenarios. Output-stationary is stronger when partial-sum movement would otherwise be the major cost and local accumulation can be preserved until completion. The right choice follows measured traffic composition, not a fixed rule.

FOLLOW-UP TRAP: Selecting dataflow style without analyzing dominant data-movement cost.

[INT][ACCEL][SYSTOLIC-ARRAYS]

Q: What common mistake reduces utilization in a systolic mesh despite correct functionality?

A:
A frequent issue is mismatched tile and mesh dimensions that create long fill and drain phases with many idle PEs. Another is failing to overlap data prefetch with compute, causing bubbles between wavefront launches. Functional correctness can still pass while performance collapses due to scheduling inefficiency.

FOLLOW-UP TRAP: Validating only functional outputs and ignoring occupancy timelines.

[INT][ACCEL][SYSTOLIC-ARRAYS]

Q: How do you decide accumulator size for output-stationary execution?

A:
Start from reduction depth, operand precision, and acceptable numerical error, then budget headroom for worst-case value growth. Validate with layer-wise statistics to estimate overflow risk and required saturation behavior, and compare local accumulator cost against expected spill traffic if size is reduced. The decision is a joint numeric and memory-energy tradeoff.

FOLLOW-UP TRAP: Sizing accumulators from datatype width alone without reduction-depth analysis.

SYMPTOM -> BOTTLENECK CLASS -> ARTIFACT -> OWNER ACTION -> VALIDATION

SYSTOLIC WAVEFLOW

operand stream -> wavefront launch -> PE mesh compute -> reduction/writeback
                         ^ bubbles and feed stalls reduce realized throughput