Understanding computation from one layer below the API.

Featured · GPU Architecture · 2 min read

How fast can your GPU actually run a transformer?

A playable roofline model — pick a GPU, shape a transformer, and watch where the FLOPs go.