Energy per Operation and the Memory Gap · arch

Computer Architecture · All levels

On this page

From power to energy per operation

Power divided by throughput is energy per operation. It is the efficiency number architects compare across designs.

Assume the lesson-1 block: 0.288 W at 3 GHz and one instruction per cycle. Each instruction costs 96 pJ.

Diagram

E_per_op = P / (f x IPC)
  = 0.288 W / (3e9 Hz x 1)
  = 9.6e-11 J = 96 pJ

The compute-memory energy gap

Moving data costs far more than computing on it. Textbooks commonly place an off-chip access roughly two orders of magnitude above a simple on-chip ALU operation.

The ratios below are illustrative orders of magnitude of that textbook class, not vendor measurements. Keep data local and reuse it.

Operation	Illustrative relative energy
Integer ALU op	1x
Register file access	A few x
On-chip SRAM access	Tens of x
Off-chip DRAM access	About 100x

Assume one memory-heavy kernel with this illustrative split. The shares sum to 100 percent up to rounding.

Sink	Share of task energy
Off-chip DRAM movement	45 percent
On-chip SRAM and registers	25 percent
Datapath arithmetic	15 percent
Clock and interconnect	15 percent

Toy model: clock gating

The model below counts toggles over a fixed 16-cycle pattern with activity on every third cycle. It is a runnable toy of activity-based energy accounting.

design.sv

module tb_clock_gate_activity;
  localparam int N = 16;
  localparam int CLK_COST = 1;
  localparam int DATA_COST = 1;
  localparam int STATIC_COST = 1;
  int errors = 0;
  int ungated_clk = 0;
  int gated_clk = 0;
  int data_toggles = 0;
  int e_ungated = 0;
  int e_gated = 0;

  initial begin
    for (int i = 0; i < N; i++) begin
      ungated_clk = ungated_clk + CLK_COST;
      e_ungated = e_ungated + STATIC_COST;
      e_gated = e_gated + STATIC_COST;
      if ((i % 3) == 0) begin
        gated_clk = gated_clk + CLK_COST;
        data_toggles = data_toggles + DATA_COST;
      end
    end
    e_ungated = e_ungated + ungated_clk + data_toggles;
    e_gated = e_gated + gated_clk + data_toggles;
    if (ungated_clk != 16) errors++;
    if (gated_clk != 6) errors++;
    if (data_toggles != 6) errors++;
    if (e_ungated != 38) errors++;
    if (e_gated != 28) errors++;
    if ((e_ungated - e_gated) != 10) errors++;
    if (errors == 0) $display("PASS: gating trims modeled energy from 38 to 28 units on the 16-cycle pattern");
    else $display("FAIL: %0d count checks differ", errors);
    $finish;
  end
endmodule

Gating saves 10 of 38 units, about 26 percent. The static share accrues every cycle, so it caps the saving.

Common mistakes

Check yourself Throughput doubles at fixed power. What happens to energy per operation?

It halves. The same power spread over twice the operations costs half per operation.

Check yourself The toy pattern runs 16 cycles with activity on every third cycle starting at cycle 0. How many active cycles is that?

Six: cycles 0, 3, 6, 9, 12 and 15.

Related topics

Up nextArchitecture Interview Prep

Practice this lesson Report an error

The compute-memory energy gap

Moving data costs far more than computing on it. Textbooks commonly place an off-chip access roughly two orders of magnitude above a simple on-chip ALU operation.

The ratios below are illustrative orders of magnitude of that textbook class, not vendor measurements. Keep data local and reuse it.

Operation	Illustrative relative energy
Integer ALU op	1x
Register file access	A few x
On-chip SRAM access	Tens of x
Off-chip DRAM access	About 100x

Assume one memory-heavy kernel with this illustrative split. The shares sum to 100 percent up to rounding.

Sink	Share of task energy
Off-chip DRAM movement	45 percent
On-chip SRAM and registers	25 percent
Datapath arithmetic	15 percent
Clock and interconnect	15 percent

Toy model: clock gating

The model below counts toggles over a fixed 16-cycle pattern with activity on every third cycle. It is a runnable toy of activity-based energy accounting.

design.sv

module tb_clock_gate_activity;
  localparam int N = 16;
  localparam int CLK_COST = 1;
  localparam int DATA_COST = 1;
  localparam int STATIC_COST = 1;
  int errors = 0;
  int ungated_clk = 0;
  int gated_clk = 0;
  int data_toggles = 0;
  int e_ungated = 0;
  int e_gated = 0;

  initial begin
    for (int i = 0; i < N; i++) begin
      ungated_clk = ungated_clk + CLK_COST;
      e_ungated = e_ungated + STATIC_COST;
      e_gated = e_gated + STATIC_COST;
      if ((i % 3) == 0) begin
        gated_clk = gated_clk + CLK_COST;
        data_toggles = data_toggles + DATA_COST;
      end
    end
    e_ungated = e_ungated + ungated_clk + data_toggles;
    e_gated = e_gated + gated_clk + data_toggles;
    if (ungated_clk != 16) errors++;
    if (gated_clk != 6) errors++;
    if (data_toggles != 6) errors++;
    if (e_ungated != 38) errors++;
    if (e_gated != 28) errors++;
    if ((e_ungated - e_gated) != 10) errors++;
    if (errors == 0) $display("PASS: gating trims modeled energy from 38 to 28 units on the 16-cycle pattern");
    else $display("FAIL: %0d count checks differ", errors);
    $finish;
  end
endmodule

Gating saves 10 of 38 units, about 26 percent. The static share accrues every cycle, so it caps the saving.

Common mistakes

Check yourself Throughput doubles at fixed power. What happens to energy per operation?

It halves. The same power spread over twice the operations costs half per operation.

Check yourself The toy pattern runs 16 cycles with activity on every third cycle starting at cycle 0. How many active cycles is that?

Six: cycles 0, 3, 6, 9, 12 and 15.