Skip to content
DenseChip
Learn
PracticeCode labInterview

DenseChip

Navigate

Menu

LearnPracticeCode labInterview

Tools

All toolsSV editorCode repoResume
DenseChip

Practice, courses, and chip-design tools in one focused workspace.

LearnPracticeCode labInterviewToolsLearning paths

© 2026 DenseChip

AboutContactPrivacyTerms
Skip to lesson

Course contents

  • How to Use This Course
  • Reading Path
  • Cache Coherency Glossary
  • The Shared Course Model
  • Cache Coherency Revision Sheet
  • Why Coherence Exists8
  • States and Protocols12
  • Transactions12
  • Races and Controller Design8
  • Progress and the Fabric16
  • Ordering and Synchronization12
  • Performance and Scalability12
  • Real Protocols12
  • SoC Integration12
    • LLC Data and Directory Policies
    • Devices and DMA
      • Coherent Versus Non-Coherent DMA
      • I/O-Coherent Ports
      • Device Caches and CXL.cache
      • Software Maintenance for DMA
    • The Memory-Controller Boundary
  • Power, Clocking and Reliability12
  • Verification12
  • Debug and Bring-up12
  • Interview and Design Review17
  • Capstone: A Coherent Model Lab5
Loading lesson

‹Devices and DMA
  1. Topics
  2. Cache Coherency
  3. SoC Integration
  4. Devices and DMA
  5. Software Maintenance for DMA

Part 9 · SoC Integration · All levels

Software Maintenance for DMA

Write the Linux DMA API sequence for each direction, the ownership rule and the descriptor-ring ordering.

On this page

    Who may touch the buffer now?

    The Linux kernel documentation (Dynamic DMA mapping Guide, DMA-API-HOWTO, and the DMA API reference) gives the sequence for streaming mappings. On a non-coherent platform each sync call performs the clean or invalidate defined in Part 3. On a coherent platform the same calls do no cache maintenance, but the sequence and ownership rule still apply.

    Ownership rule: between dma_sync_*_for_device and dma_sync_*_for_cpu (or unmap), the buffer belongs to the device and the CPU must not touch it.

    Step

    To the device (DMA_TO_DEVICE)

    Effect in the X example

    1

    CPU writes the buffer

    C0 M / 1, memory = 0

    2

    dma_sync_*_for_device(DMA_TO_DEVICE): clean to the point of coherence

    memory = 1 before the doorbell

    3

    MMIO doorbell with writel(), which orders earlier memory writes before the MMIO store

    D starts and reads 1

    Step

    From the device (DMA_FROM_DEVICE)

    Effect in the X example

    1

    dma_sync_*_for_device(DMA_FROM_DEVICE) before start: clean or invalidate

    No dirty CPU copy of X left to be written back over D's data

    2

    Device writes

    memory = 2, CPU does not touch the buffer

    3

    Completion: interrupt or status read

    D has finished writing X

    4

    dma_sync_*_for_cpu(DMA_FROM_DEVICE): invalidate

    C0 invalidated, so the CPU load misses and reads 2

    5

    CPU reads

    CPU sees 2

    The CPU writes input, syncs for the device, rings the doorbell. Before the device writes output the buffer is synced for the device again. After completion it is synced for the CPU, which then reads.
    The Linux streaming-DMA sequence: clean before the device reads, pre-sync before the device writes, invalidate before the CPU reads.

    Cache-line sharing

    Maintenance works on whole 64-byte lines. Suppose a CPU variable shares X's line with a DMA_FROM_DEVICE buffer. The invalidate after completion discards any CPU update to that variable, and a clean, or the eviction of a dirty line, writes the CPU's stale copy of X back over D's data. So DMA buffers must be cache-line aligned and padded to whole lines: no CPU data may share their lines.

    Coherent descriptor rings

    For coherent descriptor memory no cache maintenance is needed, but coherence does not order the doorbell. The producer writes the descriptor fields, then dma_wmb(), then sets the valid bit, then rings the doorbell with writel(), which waits for the earlier memory writes before the MMIO store. On completion the consumer reads the status, then dma_rmb(), then reads the payload, so the payload is not read before the status says it is ready (Linux kernel memory barriers document). A fence is not a cache flush, and cache maintenance is not a device-completion wait: see the fence lesson.

    The X example separates failures. If C0 produced 1 but a non-coherent D reads 0, the to-device sync was missing. If D produced 2 but the CPU reads an old 1, the for-CPU sync after completion was missing. If D's 2 was overwritten by 1 later, the pre-sync before the device wrote, or the line alignment, was missing. If the DMA address points elsewhere, no sync or fence repairs that bug.

    Common pitfall

    Touching the buffer from the CPU between sync_for_device and sync_for_cpu, or using writel_relaxed() for the doorbell, which does not order earlier memory writes.

    Interview

    Walk both directions with the API calls and the cache effect of each, then the descriptor-ring order: fields, dma_wmb, valid bit, writel.

    Check yourself Can a correct cache-maintenance operation prove the device has finished writing?

    No. Maintenance reconciles data. The device's completion event (interrupt or status read) establishes that its writes are complete.

    Check yourself A driver's counter shares a 64-byte line with the tail of a DMA_FROM_DEVICE buffer. What goes wrong?

    The invalidate after completion discards CPU updates to the counter, or a clean or dirty eviction of the line writes the CPU's stale copy over D's data. Align and pad DMA buffers to whole cache lines.

    Related topics

    • Shared course model

    • Path classification

    • Named maintenance targets

    Previous

    105 / 167

    Next: The Memory-Controller Boundary

    Up next

    Up nextThe Memory-Controller Boundary

    Sign in to save progress

    Practice this lessonReport an error

    See also

    • Coherent Versus Non-Coherent DMA
    • I/O-Coherent Ports