IIntroduction to AI Systems Performance EngineeringTechnical textbook
Appendix A

Interactive models

The figures make a few important relationships manipulable. Predict the direction before changing a value; treat the result as a lower bound or consistency check, never as a substitute for measurement.

01

Roofline explorer

Connect peak arithmetic, memory bandwidth, and arithmetic intensity.

Predicted regimebandwidth-bound
Ridge point40.0 FLOP/BUpper bound60.0 TFLOP/s

Next question: if measurement is well below this roof, which dependency, instruction, parallelism, or traffic assumption is wrong?

02

KV-cache capacity

Estimate cache memory for a decoder using layers, KV heads, head dimension, precision, tokens, and concurrency.

Cache footprint128.00 GiB
Per token128.0 KiBPer request16.00 GiB

This excludes block tail waste, page tables, allocator metadata, prefix sharing, tensor-parallel sharding, workspaces, and safety headroom. Add those before sizing a service.

1

Predict

Write the regime, trend, or break-even point before touching a profiler.

2

Measure

Capture a controlled baseline with correctness and workload context.

3

Explain

Reconcile model and observation. The mismatch is often the lesson.