AI Accelerator Interview Q&A Bank

[INT][ACCEL][CROSS-TOPIC]

Q: Why is TOPS an insufficient metric when comparing NPUs, GPUs, and TPUs?

A:
TOPS ignores memory behavior, precision constraints, software overhead, and real operator mix. Two devices with similar peak compute can produce very different delivered throughput and latency once attention kernels, data movement, and compiler quality are included. Comparison should use workload-representative metrics like inferences per second per watt at required accuracy and SLA.

FOLLOW-UP TRAP: Choosing hardware from peak compute specs without workload-level measurement.

[INT][ACCEL][CROSS-TOPIC]

Q: What is the first step to improve workload mapping on a new accelerator?

A:
Start by profiling layer-level bottlenecks to separate compute-bound and memory-bound operators, then adjust tiling, fusion, and layout transforms accordingly. Mapping changes without this profile often move bottlenecks instead of removing them. A useful baseline is sustained utilization and stall breakdown per major operator class.

FOLLOW-UP TRAP: Applying generic kernel tuning before identifying actual bottlenecks.

[INT][ACCEL][CROSS-TOPIC]

Q: How do batching decisions affect throughput, latency, and energy together?

A:
Larger batches generally raise arithmetic utilization and energy efficiency per operation, improving throughput, but they can increase queueing delay and tail latency. Smaller batches reduce wait time but may underutilize compute and worsen energy per inference. The right batch policy depends on SLA targets, traffic burstiness, and thermal budget.

FOLLOW-UP TRAP: Assuming maximum batch size is always the best production setting.

[INT][ACCEL][CROSS-TOPIC]

Q: When does a hybrid edge-datacenter accelerator strategy make sense?

A:
Hybrid deployment works when latency-sensitive preprocessing or fallback inference must happen near the user while high-compute or periodically updated models run centrally. Success depends on clear split points, compatible model formats, and orchestration that minimizes transfer and synchronization overhead. Without those controls, hybrid systems can add complexity without cost or SLA benefits.

FOLLOW-UP TRAP: Splitting workloads across edge and cloud without accounting for orchestration overhead.

[INT][ACCEL][CROSS-TOPIC]

Q: When does input-stationary dataflow provide the biggest benefit?

A:
It performs best when activation movement dominates memory energy, such as workloads with high feature-map reuse across many MAC operations. Keeping inputs local can cut repeated fetches from upper memory levels, but only if weights and reductions are scheduled without introducing stalls. Benefit should be validated with traffic breakdown, not assumed from the label.

FOLLOW-UP TRAP: Choosing input-stationary without confirming activation traffic is the dominant cost.

[INT][ACCEL][CROSS-TOPIC]

Q: Why can output-stationary lose its advantage in production?

A:
Its main gain comes from local partial-sum accumulation, but undersized accumulators or misaligned timing force frequent spills to shared memory. Once spill traffic rises, latency and energy can degrade quickly and erase theoretical savings. Accurate sizing requires workload-aware reduction-depth and precision analysis.

FOLLOW-UP TRAP: Assuming output-stationary is always superior for any reduction-heavy layer.

[INT][ACCEL][CROSS-TOPIC]

Q: What is the practical risk in weight-stationary execution?

A:
Stationary weights reduce parameter fetch cost, but activation delivery and partial-sum transport can become the new bottleneck if the feeder network is underprovisioned. In that case, PEs hold weights yet idle waiting for inputs. Evaluating reuse alongside interconnect and buffer pressure is essential.

FOLLOW-UP TRAP: Optimizing for weight reuse alone while ignoring activation and reduction bandwidth limits.

[INT][ACCEL][CROSS-TOPIC]

Q: How should teams decide whether to deploy a hybrid dataflow policy?

A:
Use layer-level profiling and a cost model that includes both kernel efficiency and switching overhead between dataflows. Hybrid wins when per-layer gains exceed orchestration, retile, and control complexity costs across the full model path. The decision should be made from end-to-end latency and energy evidence, not isolated kernel benchmarks.

FOLLOW-UP TRAP: Mixing dataflows by intuition without accounting for transition overhead.

[INT][ACCEL][CROSS-TOPIC]

Q: Why can a larger systolic array deliver lower real throughput for some models?

A:
Because peak MAC count scales with area, but real workloads may not map cleanly to larger dimensions. Small or irregular tiles increase edge idling, and memory bandwidth may become the limiting factor before compute saturates. Without matching tiling and feed bandwidth, bigger arrays can show lower utilization and only modest wall-clock gains.

FOLLOW-UP TRAP: Assuming array dimension growth automatically yields proportional throughput gains.

[INT][ACCEL][CROSS-TOPIC]

Q: When is weight-stationary preferable to output-stationary?

A:
Weight-stationary is usually better when filter reuse is high and weight-fetch energy dominates, such as many batched inference scenarios. Output-stationary is stronger when partial-sum movement would otherwise be the major cost and local accumulation can be preserved until completion. The right choice follows measured traffic composition, not a fixed rule.

FOLLOW-UP TRAP: Selecting dataflow style without analyzing dominant data-movement cost.

[INT][ACCEL][CROSS-TOPIC]

Q: What common mistake reduces utilization in a systolic mesh despite correct functionality?

A:
A frequent issue is mismatched tile and mesh dimensions that create long fill and drain phases with many idle PEs. Another is failing to overlap data prefetch with compute, causing bubbles between wavefront launches. Functional correctness can still pass while performance collapses due to scheduling inefficiency.

FOLLOW-UP TRAP: Validating only functional outputs and ignoring occupancy timelines.

[INT][ACCEL][CROSS-TOPIC]

Q: How do you decide accumulator size for output-stationary execution?

A:
Start from reduction depth, operand precision, and acceptable numerical error, then budget headroom for worst-case value growth. Validate with layer-wise statistics to estimate overflow risk and required saturation behavior, and compare local accumulator cost against expected spill traffic if size is reduced. The decision is a joint numeric and memory-energy tradeoff.

FOLLOW-UP TRAP: Sizing accumulators from datatype width alone without reduction-depth analysis.

[INT][ACCEL][CROSS-TOPIC]

Q: Why can tensor-core peak FLOPs be far above delivered throughput in real workloads?

A:
Peak FLOPs assume ideal tile occupancy and uninterrupted data supply, but real kernels often stall on memory movement, synchronization, or shape misalignment. If shared-memory staging, register reuse, and warp scheduling are not tuned, tensor pipelines idle even when the hardware supports much higher compute. Delivered throughput is therefore constrained by dataflow quality, not just arithmetic capability.

FOLLOW-UP TRAP: Treating low tensor-core throughput as purely a hardware limit instead of a dataflow issue.

[INT][ACCEL][CROSS-TOPIC]

Q: How does mixed-precision matmul improve speed without always hurting model quality?

A:
Lower-precision math increases tensor-core throughput and cuts bandwidth, while wider accumulation and selective precision controls preserve numerical stability. Training can use scaling and precision overrides for sensitive layers, and inference can rely on calibration and outlier handling to maintain accuracy. The approach works when precision choices are operator-aware rather than globally forced.

FOLLOW-UP TRAP: Assuming one low-precision format should be applied uniformly to every operator.

[INT][ACCEL][CROSS-TOPIC]

Q: What makes structured sparsity easier to accelerate than unstructured sparsity?

A:
Structured patterns constrain where zeros appear, so hardware and compilers can skip work with predictable scheduling and minimal control overhead. This regularity preserves parallel efficiency and memory coalescing, enabling practical speedups on supported tensor-core paths. Unstructured sparsity is more flexible but introduces irregular indexing and execution paths that are harder to optimize.

FOLLOW-UP TRAP: Assuming any sparsity pattern automatically maps to fast sparse tensor-core execution.

[INT][ACCEL][CROSS-TOPIC]

Q: When is unstructured sparsity still worth considering despite runtime overhead?

A:
It is valuable when high compression ratios materially reduce memory footprint or bandwidth and when quality targets are easier to maintain with unconstrained pruning. It can also help deployment scenarios constrained by model size more than raw throughput. Teams should choose it only after benchmarking end-to-end latency, energy, and fallback behavior against structured alternatives.

FOLLOW-UP TRAP: Choosing unstructured sparsity from pruning percentage alone without system-level benchmarks.

[INT][ACCEL][CROSS-TOPIC]

Q: How do you decide whether two operators should be fused?

A:
Fuse when intermediate tensor traffic and launch overhead savings outweigh added register pressure, code size, and scheduling complexity. The decision should come from profiling memory traffic, occupancy, and spill behavior on target shapes rather than from a blanket fusion rule. Good fusion plans are shape-aware and often differ between training and inference paths.

FOLLOW-UP TRAP: Applying maximum fusion everywhere without checking spill and occupancy regressions.

[INT][ACCEL][CROSS-TOPIC]

Q: Why can increasing batch size improve throughput but hurt user experience?

A:
Larger batches usually improve accelerator utilization and reduce per-item overhead, so throughput rises and energy per inference can improve. But requests wait longer to be grouped, which increases queueing delay and can inflate P99 latency under bursty traffic. User experience is governed by tail latency, not throughput alone.

FOLLOW-UP TRAP: Optimizing only for peak throughput and ignoring P99 latency targets.

[INT][ACCEL][CROSS-TOPIC]

Q: When should dynamic scheduling be preferred over static partitioning on multi-core accelerators?

A:
Dynamic scheduling is preferable when workload shape or request mix varies enough that static partitions leave some cores idle while others saturate. It improves balance but must be constrained with topology-aware placement and contention controls to avoid shared-resource thrash. If workload variance is low and determinism is critical, static partitioning may still win.

FOLLOW-UP TRAP: Assuming dynamic scheduling is always superior regardless of contention costs.

[INT][ACCEL][CROSS-TOPIC]

Q: What is the benefit of feeding runtime telemetry back into compiler mapping?

A:
Telemetry reveals real shape distributions, hot paths, and memory bottlenecks that differ from offline assumptions, allowing compiler heuristics to be retuned to production reality. This feedback loop reduces persistent mismatches in placement, tile selection, and fusion boundaries. Over time it improves delivered performance stability across workload drift.

FOLLOW-UP TRAP: Treating compile-time mapping as fixed and never updating it from production behavior.

[INT][ACCEL][CROSS-TOPIC]

Q: Why does adding more on-chip SRAM not always improve accelerator throughput?

A:
Extra SRAM helps only when data placement, tiling, and refill policy let workloads actually reuse that capacity. If buffers are poorly partitioned or refill cadence is mismatched, larger arrays can still stall on refetch and eviction churn. Throughput tracks effective reuse and feed efficiency, not raw SRAM size.

FOLLOW-UP TRAP: Assuming SRAM capacity growth alone guarantees higher performance.

[INT][ACCEL][CROSS-TOPIC]

Q: When should a kernel prefer scratchpad control over cache-based access?

A:
Scratchpads are usually better when access patterns are regular, synchronization points are known, and software can schedule movement explicitly. Cache paths are often better for irregular or rapidly changing locality where static planning is brittle. The decision should come from measured miss or spill behavior and engineering complexity tradeoffs.

FOLLOW-UP TRAP: Choosing scratchpad or cache by default without locality measurements.

[INT][ACCEL][CROSS-TOPIC]

Q: How do teams avoid overestimating available HBM bandwidth during design planning?

A:
They model real traffic shape, concurrency, arbitration overhead, and controller behavior instead of relying on peak spec bandwidth. Planning includes burst efficiency, read/write mix, and contention from simultaneous kernels to estimate sustainable delivery. Guardbands and stress scenarios are essential before committing compute sizing decisions.

FOLLOW-UP TRAP: Sizing compute arrays from theoretical HBM peak numbers alone.

[INT][ACCEL][CROSS-TOPIC]

Q: What is a common symptom of memory bank conflicts in on-chip SRAM systems?

A:
You often see high compute idle time even though total memory bandwidth looks adequate, because requests serialize on a few hot banks. Profilers usually show elevated bank-conflict counters and periodic stall bursts tied to specific access strides. Layout remapping or stride padding typically restores parallelism.

FOLLOW-UP TRAP: Diagnosing conflict stalls as generic bandwidth shortage without bank-level traces.

[INT][ACCEL][CROSS-TOPIC]

Q: How should teams choose between INT8, FP16, BF16, and FP8 for production inference?

A:
Select format by measured quality and efficiency on target workloads, not by peak hardware claims. INT8 often gives the best efficiency when calibration is robust, FP16 and BF16 help with dynamic range sensitivity, and FP8 can unlock additional gains with careful scaling and accumulation policy. The right answer is usually mixed precision at operator granularity.

FOLLOW-UP TRAP: Picking one precision mode globally without operator-level sensitivity analysis.

[INT][ACCEL][CROSS-TOPIC]

Q: Why is quantization-aware design considered a full-stack problem?

A:
Because model behavior, compiler lowering, and hardware execution all influence post-quantization quality and speed. Training-time quantization simulation, deployment calibration, and runtime numeric handling must align to avoid production drift. Treating quantization as only a model or only a hardware task usually creates hidden regressions.

FOLLOW-UP TRAP: Applying post-training quantization alone and expecting stable production quality.

[INT][ACCEL][CROSS-TOPIC]

Q: What is the first practical step in improving accelerator performance per watt?

A:
Profile real workloads to separate compute inefficiency, memory movement waste, and idle scheduling gaps before tuning. This identifies where each watt is spent and prevents optimizations that simply shift bottlenecks. A targeted loop of measure, prioritize, and re-benchmark typically outperforms broad tuning sweeps.

FOLLOW-UP TRAP: Starting optimization from synthetic peak benchmarks instead of workload profiling.

[INT][ACCEL][CROSS-TOPIC]

Q: How do thermal behavior and DVFS affect precision and performance planning?

A:
Thermal limits and DVFS states change sustained frequency, which directly changes throughput and sometimes the preferred precision operating point. A precision policy that looks ideal at cool startup may underperform once throttling begins. Planning should therefore use long-window thermal runs and frequency residency data, not short burst measurements.

FOLLOW-UP TRAP: Validating precision and throughput only in short, non-thermal stress tests.

[INT][ACCEL][CROSS-TOPIC]

Q: What makes an accelerator verification strategy tapeout-ready rather than just broad?

A:
Tapeout-ready strategy ties tests and coverage to the highest architecture and product risks, not only generic functional completeness. It combines simulation, formal checks, and software-integrated scenarios with explicit signoff criteria. The key is evidence that critical failure modes are either closed or bounded with documented mitigations.

FOLLOW-UP TRAP: Declaring verification complete from high aggregate coverage without risk-based closure.

[INT][ACCEL][CROSS-TOPIC]

Q: Why do bring-up teams invest in detailed performance counters before silicon arrives?

A:
Because counters are the fastest path to separating hardware limits from software mapping issues once first silicon is running. Well-defined counters allow direct attribution of stalls, underutilization, and memory pressure, reducing blind tuning cycles. Early counter design also ensures observability for classes of bottlenecks that are difficult to infer externally.

FOLLOW-UP TRAP: Assuming profiler insights can be reconstructed later without hardware observability planning.

[INT][ACCEL][CROSS-TOPIC]

Q: How is a power-virus test different from regular benchmark stress?

A:
Regular benchmarks represent realistic workloads, while power-virus tests intentionally drive worst-case switching and thermal stress to validate safety margins. They expose throttle behavior, hotspot dynamics, and control-loop weaknesses that normal application traces may never trigger. Both are needed: one for product performance and one for reliability limits.

FOLLOW-UP TRAP: Using only application benchmarks to sign off thermal and power robustness.

[INT][ACCEL][CROSS-TOPIC]

Q: What is the most common reason post-silicon debug takes too long?

A:
Lack of reproducible traces and cross-layer context is the dominant delay, causing teams to chase symptoms instead of causes. Effective debug depends on synchronized hardware, firmware, and workload evidence plus a disciplined triage process. Without that, ownership ambiguity and repeated lab reruns dominate cycle time.

FOLLOW-UP TRAP: Starting deep root-cause analysis before building a reproducible failure capture path.

SYMPTOM -> BOTTLENECK CLASS -> ARTIFACT -> OWNER ACTION -> VALIDATION

workload -> mapping -> memory and compute behavior -> SLA outcome