The question

The interviewer asks what changes when a pipeline gains a stage. The expected answer names two numbers moving in opposite directions and says what each costs.

What is really under test is whether the candidate separates per-item delay from aggregate rate. Confusing the two is the classic junior error this question hunts.

Model answer

Adding a stage adds one cycle of latency and, once the pipe is full, changes nothing about throughput. The clock rate can rise if the cut stage was critical; the first result arrives one cycle later.

The testbench below uses four register ranks and twenty consecutive items. Input 1 is accepted at edge 1; it is visible after NBA at edge 4 (three elapsed periods), and a downstream register consumes it at edge 5 (four elapsed periods). Then one result is consumed per cycle.

Each slot shows all four rank outputs just after a clock edge, with a new item entering every cycle. Slots are numbered from 0, so slot 0 is the picture after edge 1, the same edge numbering as the table above.

• Slot 3 (after edge 4): item 1 is visible at the output rank, three periods after it entered. A downstream register reading the pre-edge output consumes it at edge 5, four periods after acceptance.

• Slots 4 to 6 (edges 5 to 7): items 2, 3 and 4 arrive one per cycle. The pipe is full, so throughput holds at one result per cycle.

• Cost: each extra rank adds one cycle of latency and one rank of registers, while full-pipe throughput stays flat.

Latency is a cycle count per item; throughput is items per cycle in steady state. Only the testbench timing both at once keeps them apart.

Follow-ups

When does pipelining not help? In a feedback loop where the next input needs the previous output, such as an accumulator. Extra stages cannot overlap iterations that depend on each other.

How does the consumer know which cycles carry real results? A valid bit travels beside the data through the same stages. The testbench checks data only on valid cycles.

What if the input has gaps? Throughput dips for exactly the gap cycles and recovers after. Latency per item is unchanged; bubbles propagate like data.

Common wrong answers

A weaker candidate says the pipeline is now four times faster. Only the steady-state rate for streams improved, and only if the clock rose; each item is slower than before.

Another defines throughput as one over latency. A four-cycle pipeline delivering one result per cycle has latency four and throughput one; the two numbers describe different things.

A third calls the registers free. Each rank adds area, clock power and verification state, which is why shallow logic is never pipelined for its own sake.

Solution and testbench

A four-deep valid-tracking pipe plus the testbench that measures first-result latency and per-cycle rate.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

module scale_pipe #(parameter int S = 4, parameter int W = 8) (
  input  logic         clk, rst_n,
  input  logic         in_valid,
  input  logic [W-1:0] in_data,
  output logic         out_valid,
  output logic [W-1:0] out_data
);
  logic         v [S];
  logic [W-1:0] d [S];
  always_ff @(posedge clk or negedge rst_n)
    if (!rst_n) begin
      for (int i = 0; i < S; i++) begin v[i] <= 1'b0; d[i] <= '0; end
    end else begin
      v[0] <= in_valid; d[0] <= in_data;
      for (int i = 1; i < S; i++) begin v[i] <= v[i-1]; d[i] <= d[i-1]; end
    end
  assign out_valid = v[S-1];
  assign out_data  = d[S-1];
endmodule

Related topics

• Throughput and latency