The Residual Stream Understanding computation from one layer below the API — AI internals, GPUs, and the physics of computation. https://theresidualstream.ai en Building this site The site itself is the first experiment — an engineering notebook exploring what runs one layer below the API. A workshop, not a manifesto

This site is a workshop open to the street. There are no categories yet, no grand About page, no manifesto — just a stream of things I make: short notes, interactive toys, and experiments that poke at the layer below the one everyone else stops at.

The stack is deliberately boring where it can be (SvelteKit, markdown files in a git repo, static hosting) and interesting only where it pays (Svelte components embedded directly inside posts, so an essay can carry a live simulation in its pocket).

The first live artifact

The first such essay is already here — it has a GPU roofline you can play with.

]]> https://theresidualstream.ai/posts/building-this-site https://theresidualstream.ai/posts/building-this-site Thu, 16 Jul 2026 00:00:00 GMT How fast can your GPU actually run a transformer? A playable roofline model — pick a GPU, shape a transformer, and watch where the FLOPs go.

Draft stub — the real essay goes here. The point of this post in the skeleton is to prove that prose and live computation can share a page.

Everyone quotes peak TFLOPs. Almost nobody hits them. During decode, a transformer reads every weight once per token step — so until your batch is large enough, the GPU spends its life waiting on HBM, not multiplying matrices.

Find the ridge

Try it. Watch the "bound by" readout flip as you push the batch size past the ridge:

The ridge point. Arithmetic intensity is useful work per byte moved. Batched decode raises that intensity until compute, rather than memory bandwidth, becomes the ceiling. Bridge ≈ Fpeak / BWmem

Interactive roofline calculator: choose a GPU, model width, layer count, and batch size to compare memory-bound and compute-bound transformer decoding. Open the web article to use the controls.

Read the result

Two things to notice:

  1. The ridge point is a property of the GPU, not the model. Peak FLOPs divided by memory bandwidth. On an A100 that's ~153: below that batch size, you are paying for silicon you cannot feed.
  2. Utilization is a choice you make at design time — d_model, layer count, and batch geometry decide your ceiling before a single kernel is written.

The full essay will add the KV cache (which makes everything worse), quantization (which moves the ridge), and real measured numbers against this toy model.

KV-cache overhead. The toy counts only model weights. Real decode also streams cached keys and values, so longer contexts add memory traffic without increasing weight reuse.

]]> https://theresidualstream.ai/posts/gpu-flops https://theresidualstream.ai/posts/gpu-flops Thu, 16 Jul 2026 00:00:00 GMT