May 9, 2026•LLM Architecture
Why GQA Changes the KV Cache Bill Before Quantization
How grouped-query attention changes the KV cache formula before FP8 or INT4: MHA vs MQA vs GQA, decode bandwidth, and serving capacity math.
Read articleBack to Blog
LLM Architecture
LLM Architecture
Transformer internals, context behavior, and model design trade-offs.
LLM Architecture focuses on the internal mechanisms that shape model behavior, from attention heads and GQA to context limits and representation design.
Feb 26, 2026•LLM Architecture
Attention in Practice: Visualizing Q/K/V and why scaling heads changes behavior
A walkthrough of scaled dot-product attention (Q/K/V), softmax temperature, and why increasing head count shifts attention statistics and behavior.
Read article