AI Accelerator Design · All levels
Scheduling & Workload Mapping: Tricky Q&A
Senior interview and review questions for Scheduling & Workload Mapping.
Section Q&A bank
Use these drills after completing all topics in Scheduling & Workload Mapping. Answer with workload context, mechanism proof, artifact, owner, and release decision.
How do you decide whether two operators should be fused?
diagram
[INT][ACCEL][SCHEDULING-MAPPING]
Q: How do you decide whether two operators should be fused?
A:
Fuse when intermediate tensor traffic and launch overhead savings outweigh added register pressure, code size, and scheduling complexity. The decision should come from profiling memory traffic, occupancy, and spill behavior on target shapes rather than from a blanket fusion rule. Good fusion plans are shape-aware and often differ between training and inference paths.
FOLLOW-UP TRAP: Applying maximum fusion everywhere without checking spill and occupancy regressions.Why can increasing batch size improve throughput but hurt user experience?
diagram
[INT][ACCEL][SCHEDULING-MAPPING]
Q: Why can increasing batch size improve throughput but hurt user experience?
A:
Larger batches usually improve accelerator utilization and reduce per-item overhead, so throughput rises and energy per inference can improve. But requests wait longer to be grouped, which increases queueing delay and can inflate P99 latency under bursty traffic. User experience is governed by tail latency, not throughput alone.
FOLLOW-UP TRAP: Optimizing only for peak throughput and ignoring P99 latency targets.When should dynamic scheduling be preferred over static partitioning on multi-core accelerators?
diagram
[INT][ACCEL][SCHEDULING-MAPPING]
Q: When should dynamic scheduling be preferred over static partitioning on multi-core accelerators?
A:
Dynamic scheduling is preferable when workload shape or request mix varies enough that static partitions leave some cores idle while others saturate. It improves balance but must be constrained with topology-aware placement and contention controls to avoid shared-resource thrash. If workload variance is low and determinism is critical, static partitioning may still win.
FOLLOW-UP TRAP: Assuming dynamic scheduling is always superior regardless of contention costs.What is the benefit of feeding runtime telemetry back into compiler mapping?
diagram
[INT][ACCEL][SCHEDULING-MAPPING]
Q: What is the benefit of feeding runtime telemetry back into compiler mapping?
A:
Telemetry reveals real shape distributions, hot paths, and memory bottlenecks that differ from offline assumptions, allowing compiler heuristics to be retuned to production reality. This feedback loop reduces persistent mismatches in placement, tile selection, and fusion boundaries. Over time it improves delivered performance stability across workload drift.
FOLLOW-UP TRAP: Treating compile-time mapping as fixed and never updating it from production behavior.Q&A drill guide
diagram
SYMPTOM -> BOTTLENECK CLASS -> ARTIFACT -> OWNER ACTION -> VALIDATIONSketch while answering
diagram
SCHEDULING PIPELINE
compile plan -> runtime queue -> core placement -> completion and tail behaviorCommon traps
Inspect tail first, not average
Check fairness across streams
Validate fusion against memory constraints
Key takeaways
Tie every accelerator claim to a reproducible workload slice and one primary metric trend.
Prefer bounded fixes with clear owner and rollback boundary over broad tuning bundles.
Common pitfalls
Optimizing synthetic kernels without production-shape validation.
Reading average latency while ignoring p95 and p99 behavior.
Declaring sparse or precision wins without fallback and quality evidence.