Run two unrelated documents through a large language model at the same time, and you assume the model keeps them
Continue readingYour Transformer Can Hold Two Thoughts at Once: The New Linear Superposition Result
Another week, another pile of repositories crossing the ten-thousand-star mark in days. Most weekly roundups stop at the star count,
Continue readingTrending GitHub Repos This Week: Typed Decision Models Take Over the Trending Page
DeepSeeded? No — DeepSeek. In September 2026 the Chinese AI lab released a paper titled “DeepSeek-V4.1-Flash: Pushing the Limits of
Serving a large language model to thousands of concurrent users is, underneath all the marketing, a memory management problem. Every
Continue readingDeepSeek-V4.1-Flash: Why the Most Interesting AI Paper This Month Is About Storage
Autoregressive decoding is the reason large language models feel slow: every token is generated by a full forward pass, and
When an LLM serves a 2,000-token response to a prompt of 10,000 tokens, it does something that looks absurd from
Continue readingPagedAttention and the KV Cache: How OS Paging Explains Modern LLM Serving
Every RAG system has two halves: a retriever that decides which text the model sees, and a generator that answers
Continue readingRAG Chunking Strategies in 2026: What the Benchmarks Actually Show
If you run a large language model behind a product, you have probably internalized an uncomfortable trade-off. The big frontier
Continue readingLLM Routing and Cascading: Send Every Query to the Cheapest Model That Can Handle It
Serving a 7B model in fp16 takes roughly 14 GB of VRAM. A 70B model takes around 140 GB —
Continue readingLLM Quantization Explained: GPTQ, AWQ, GGUF, and When Each One Wins
Watch a GPU while a large language model generates text and you’ll see something strange: for most of every forward
Continue readingSpeculative Decoding: Getting 2-3x More Out of Every GPU Pass