Skip to content
DenseChip
Learn
PracticeCode labInterview

DenseChip

Navigate

Menu

LearnPracticeCode labInterview

Tools

All toolsSV editorCode repoResume
DenseChip

Practice, courses, and chip-design tools in one focused workspace.

LearnPracticeCode labInterviewToolsLearning paths

© 2026 DenseChip

AboutContactPrivacyTerms
Skip to lesson

Course contents

  • How to Use This Course
  • Reading Path
  • Cache Coherency Glossary
  • The Shared Course Model
  • Cache Coherency Revision Sheet
  • Why Coherence Exists8
  • States and Protocols12
  • Transactions12
  • Races and Controller Design8
  • Progress and the Fabric16
  • Ordering and Synchronization12
  • Performance and Scalability12
  • Real Protocols12
  • SoC Integration12
  • Power, Clocking and Reliability12
  • Verification12
  • Debug and Bring-up12
  • Interview and Design Review17
    • Whiteboard Frameworks for Coherency
    • Staff-Level Design Reviews
    • Program Risk and Escalation Playbooks
      • Coherence Risk Register
      • Late Bug Decisions
      • An Escalation Scenario
      • Sign-off Criteria
    • Coherency Interview Q&A Bank
  • Capstone: A Coherent Model Lab5
Loading lesson

‹Program Risk and Escalation Playbooks
  1. Topics
  2. Cache Coherency
  3. Interview and Design Review
  4. Program Risk and Escalation Playbooks
  5. Late Bug Decisions

Part 13 · Interview and Design Review · All levels

Late Bug Decisions

A coherence bug found just before tape-out or on silicon: compare an RTL fix, a chicken bit, a software workaround and an erratum by safety, verification time and blast radius.

On this page

    A race is found three weeks before tape-out. Fix it, switch it off, or ship?

    Late in a program every option has a cost, and the cheapest-looking one is often the riskiest. The decision is not 'is the bug real' (it is) but 'which response leaves the least risk at tape-out, given the time left to verify it'. A staff engineer makes that decision explicit, with criteria written down before the argument starts.

    The four options

    Option

    What changes

    What must be verified

    Typical cost

    Fix the RTL and re-verify

    The design. Post-silicon this means a new stepping

    The fix and everything it touches: full regression, affected proofs and coverage, on the final RTL

    Schedule; risk of a new bug in the fix

    Chicken bit: disable the feature

    A reset or boot-time configuration value, if the bit exists

    The fallback path at full regression strength

    Performance of the disabled optimisation

    Software workaround

    Firmware, kernel or driver behaviour

    That software can reliably avoid the trigger

    Software effort and often performance

    Ship with an erratum

    Documentation only

    That the trigger cannot occur in supported use, or is harmless

    Customer trust; unacceptable for silent corruption

    Decision criteria, in priority order

    Criterion

    Question

    Rule

    Safety class

    Silent wrong value, detected failure (hang, error report) or performance only?

    A silent wrong value must be made unreachable by hardware or verified configuration. An erratum alone is never enough.

    Verification time

    Can the change be re-verified to the same evidence standard before freeze?

    A fix that cannot be re-verified in time is a new unverified risk, not a fix.

    Blast radius

    Which paths and configurations does the bug, and the fix, touch?

    Prefer the change with the narrower radius when both remove the bug.

    Workaround cost

    What do customers lose with the bit set or the software change?

    Measure it on real workloads; state it in the errata.

    Reversibility

    Can the choice be undone later without silicon?

    A reset default can be changed by firmware; RTL cannot.

    Flowchart: if the bug can return a wrong value silently, use a verified chicken bit if one exists, otherwise fix only if the fix can be re-verified before freeze, otherwise escalate to move tape-out or remove the feature. Hangs and performance bugs are fixed if re-verifiable, otherwise worked around or documented.
    Late-bug decision flow: the safety class is decided first, then whether a verified fallback exists, then whether a fix can be re-verified in time.

    Worked decision (illustrative): a late PutM in direct transfer

    Fictional program, three weeks before tape-out. A new directed test from the eviction-versus-intervention crosses finds the race of An Illustrative Escape, pre-silicon this time: C0 holds X in M / 1 and evicts it (MI^A, PutM delayed), C1's GetM is served three-hop by C0 directly, C1 stores 2, and H's Put handler then accepts C0's late PutM because it checks only that the entry is exclusive. Memory becomes 1, the directory becomes U, and C0's next read returns a stale 1 while C1 holds M / 2. The trace and the snapshot table are in that lesson; the decision is new here.

    The bug can return a wrong value silently, so an erratum alone is out. The fix is small: H accepts PutM data only from the recorded exclusive holder, and only when no transaction for X is pending; otherwise the Put waits in X's pending entry until that transaction's forward is answered. A Put from a cache H no longer records gets Put-Ack with no directory or memory change, the course's late-writeback rule.

    design.sv
    1
    2
    3
    4
    5
    6
    7
    8
    9
    // snippet
    // Put handler at H: a PutM changes memory and the directory only if it comes from the
    // recorded exclusive holder and no transaction for X is pending (otherwise the Put waits)
    wire putm_from_holder = put_is_putm && dir_is_exclusive && (dir_holder_id == put_src_id)
                            && !line_pending;
    assign put_wait         = put_valid && line_pending;       // held in X's pending entry
    assign put_write_memory = put_valid && putm_from_holder;
    assign put_clear_dir    = put_valid && putm_from_holder;
    assign put_ack_only     = put_valid && !line_pending && !putm_from_holder; // non-holder: Put-Ack, no change

    Option

    Safety

    Verification time (illustrative)

    Blast radius

    Cost

    Verdict

    Fix the Put handler

    Removes the race on both routes

    Full regression in both bit settings, rerun of the small-model proof: about 6 of 21 days

    Every writeback passes through the changed handler

    None

    Do it

    Default the bit to four-hop

    Removes the race in this design: as in An Illustrative Escape, the four-hop Data (marked PutM outstanding) reaches H, and its H marks the coming PutM as stale. That is a choice of this design, not a universal rule

    Fallback already had its own random regression (checked first)

    Routing of dirty misses only

    One extra leg on each dirty-miss critical path

    Do it as well, for first silicon

    Software workaround

    Software cannot control when a PutM is delayed in the fabric

    -

    -

    Making shared data non-cacheable would cost far more than the bit

    Reject

    Erratum only

    Leaves silent corruption reachable

    -

    -

    -

    Reject

    The decision is both: fix the RTL, and ship first silicon with the chicken bit defaulting to the four-hop route of Three-Hop Versus Four-Hop. Two independent defences cover the case where the fix itself is wrong in some way the regression did not reach. Once post-silicon litmus and stress runs with the bit clear are clean, firmware clears the bit and customers get the three-hop latency back. If the same bug were found on silicon, the RTL fix waits for the next stepping, so the chicken bit plus an erratum naming it is the only immediate answer, which is the path the escape lesson follows.

    Common pitfall

    Fixing a race in the last week without rerunning the full regression. The fix touches a handler that every eviction uses; a targeted test proves only that this race is gone, not that writebacks still work.

    Common pitfall

    Choosing the chicken bit because it needs no RTL change, without checking that its fallback was ever stressed. An unregressed fallback is a second, untested protocol.

    Interview

    Structure the answer as: safety class first, then whether a verified fallback exists, then the time to re-verify a fix, then blast radius and cost. Say that a silent wrong value is never left to an erratum alone, and that a fix plus a default-on fallback is a legitimate answer.

    Check yourself The bug is a rare hang, not a wrong value, and the fix cannot be re-verified before freeze. Is shipping with an erratum acceptable?

    It can be, if the hang is detected (a watchdog fires, no data is corrupted), the trigger and its rate are understood, and a workaround or recovery exists, such as a chicken bit or a software restriction. It must be a stated decision with the measured cost and the fix planned for the next stepping, not a default. If the hang could hide a lost write, it is a safety problem and the rule for wrong values applies.

    Related topics

    • Shared course model

    • Part 12: chicken bits and workarounds

    • The race, traced

    • The two routes

    Previous

    155 / 167

    Next: An Escalation Scenario

    Up next

    Up nextAn Escalation Scenario

    Sign in to save progress

    Practice this lessonReport an error

    See also

    • Sign-off Criteria