MHA, MQA, GQA, MLA: The KV-Cache Trade-offs Behind Every Modern Transformer
Ask someone to sketch a transformer on a whiteboard and they’ll draw the 2017 version: softmax attention over every pair
Continue readingMHA, MQA, GQA, MLA: The KV-Cache Trade-offs Behind Every Modern Transformer