01 · The patient
A correct kernel that uses a fifth of the machine
The capstone starts where day 16 ended: a working, tree-shaped reduction. The workload: sum 256M FP32 values — 1 GiB — so a perfect kernel's floor is 1 GiB ÷ 504 GB/s ≈ 2.2 ms. The baseline lands at ~11.5 ms: ~93 GB/s, roughly 18% of the 4070's bandwidth. Nothing is broken — the tree is right, the answer is right — and yet four-fifths of the machine's memory system sits idle. Day 18's loop starts here: baseline written down, Nsight attached, hypothesis formed.
02 · Five rungs
Each rung is one change and one measurement
The ladder, with the day-18 metric that justified each step. No rung was discovered by inspiration — each was read off a profile:
BASELINE · 18%
93 GB/s
day 16's two-pass tree: one thread per element, millions of blocks, all the epilogue that implies
RUNG 1 · 40%
202 GB/s
grid-stride loop — fewer threads, each looping (day 14); coalescing was already fine, thread count wasn't
RUNG 2 · 55%
277 GB/s
shrink the grid to ~8 blocks per SM; per-block setup and the second-kernel merge amortize over more work
RUNG 3 · 70%
353 GB/s
warp shuffles replace shared memory for the intra-warp stages (day 16) — fewer barriers, no bank traffic
RUNG 4 · 82%
413 GB/s
each lane loads and sums 4 floats per iteration — ILP covers load latency (day 01, at micro scale)
RUNG 5 · 88%
444 GB/s
vectorized 16-byte loads (float4) and a final tuned block count — day 22's search, done once by hand
Notice what the ladder is not: it isn't five clever tricks. It's the same kernel reorganized five times around one question — what is the profiler saying is idle? Threads idling because there were too many of them (rung 1). Blocks idling in setup and merge overhead (rung 2). Lanes idling at barriers (rung 3). Memory pipes idling between a lane's loads (rung 4). Whole 128-bit transactions idling behind 4-byte loads (rung 5).
03 · What the trace said at each rung
The metrics were ahead of the code
Rung 1's trace showed achieved occupancy near zero and a grid of 100k+ blocks — day 14's decoupling problem, visible as scheduling overhead. Rung 2 showed the launch and merge phases as visible gaps on the Nsight Systems timeline — day 17's lesson about serial phases. Rung 3 showed shared-memory transactions per instruction dropping off a cliff while DRAM throughput barely moved — the win was real but this wasn't yet the wall. Rungs 4–5 finally moved the SOL memory bar, 82 → 88%, and past that point the bar stops climbing: when the memory bar is high and flat, the kernel is done. The remaining gap to 100% is ECC overhead, refresh, and physics.
Also worth confessing: two changes tried along the way lost — a shared-memory layout "optimization" that triggered bank conflicts (day 08), and an over-aggressive block shrink that cratered occupancy (day 06). They were reverted because each was measured alone. That's the loop protecting you from your own good ideas.
04 · The meta-lesson
4× from reorganization, zero from cleverness
From 18% to 88% with the arithmetic untouched — no new algorithm, no tensor cores, no assembly. That's the normal shape of kernel optimization, and it's why day 11's chart and day 18's loop are the two tools this journey keeps returning to: the chart says what's possible, the loop gets you there. The same patient now awaits day 29's build — except this time you'll design the data flow correctly from the start, using instincts these five rungs just installed.
- 01Write the baseline down before touching anything. ~18% of bandwidth, measured. Without the number, rung 1's improvement is a feeling — and feelings don't survive review.
- 02One change per rung, justified by one metric. Grid-stride for scheduling overhead, fewer blocks for setup cost, shuffles for barriers, ILP for idle pipes, vectorization for narrow loads.
- 03When the SOL memory bar is high and flat, stop. ~88% of theoretical bandwidth is the practical ceiling — ECC, refresh, and access granularity own the rest. Chasing it costs weeks and buys nothing.
- 04Revert losing changes without sentiment. Two of seven ideas here made the kernel slower. The one-change-per-measurement loop is what made them cheap to discover and cheap to undo.
Next: day 29 — the capstone itself: a fused residual-RMSNorm kernel designed from scratch, end to end, with every habit from this journey applied on purpose.