t11s @transmissions11 · 27 Aug 2026

prefill tok/$ ≈ FLOPs/$ decode tok/$ ≈ HBM BW/$ but not always! consider a chip with poor interconnect pipeline parallelism can rescue prefill, but decode needs many seqs resident to fill a deep pipeline, eating KV capacity and shrinking per-chip batch size + throughput!

Open on X →