DeepSeeded? No — DeepSeek. In September 2026 the Chinese AI lab released a paper titled “DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression,” and the name does not oddly oversell it: the whole paper is about making a giant language model cheaper to run by shrinking something called the KV cache. If that phrase means nothing to you, good — this post explains every term from scratch, the way I wish someone had explained it to me.
The vocabulary you need first
Token
LLMs do not read words; they read tokens — short chunks of text (a word, part of a word, or a punctuation mark) converted to numbers. “Chatting” might be 2–3 tokens. Every price quote and context window in AI is measured in tokens.
Context window
The context window is how much text the model can “see” at once — its working memory. V4.1-Flash handles up to one million tokens, roughly 1,500 pages. Older models handled a few thousand, which is why early chatbots forgot what you said ten messages ago.
Attention (and why it needs a cache)
The core operation inside a transformer is attention: for each new token, the model looks back at all previous tokens and decides which ones matter for predicting what comes next. To do this efficiently, the model stores a compact summary of every past token so it never has to reconstruct them. Those summaries come in two flavors — keys (K) and values (V) — and the collection of them for the whole conversation is the KV cache.
Here is the problem the paper attacks: the KV cache grows with every token you feed the model. A million-token conversation means a million-token-sized cache. That cache has to live somewhere, and “somewhere” is expensive hardware.
HBM and SSD
HBM (High Bandwidth Memory) is the extremely fast, extremely expensive memory stacked on top of the GPU itself. The active KV cache must live here, or every generated word stalls. SSD storage is the slow, cheap tier used to save caches of previous conversations so they can be resumed without recomputation. HBM capacity is often the real bottleneck on how many users a server can serve.
Prefill vs decode
Serving a request has two phases. Prefill: read the entire prompt (all those tokens of conversation history) and build the KV cache. Decode: generate the answer one token at a time, reusing that cache. Agent workloads — where an AI reads files, tool results, and logs before emitting a short answer — are input-heavy: huge prefill, small decode. DeepSeek’s insight: optimize each phase separately.
Mixture-of-Experts (MoE)
A dense activates every parameter for every token. A Mixture-of-Experts model contains many “expert” sub-networks and routes each token to only a few of them. V4.1-Flash has 552B total parameters but activates only a slice per token — big brain, small effort. That is how it gets flagship-class quality with prices that embarrass Western labs.
The big idea: a Causal Encoder–Decoder
Most LLMs today are one stack of identical layers that both reads your prompt and writes the answer. V4.1-Flash splits the job. Its 40 layers divide into a 20-layer causal encoder that processes your input, and a 20-layer decoder that generates output. “Causal” means the encoder still only sees left-to-right (no peeking at future tokens), preserving normal LLM behavior.
Why split? Because reading and writing have different costs. The encoder run is cheap — only 8B parameters activate per token during prefill. The decoder is heavier — 16B per token during decode. Read more than you write? You pay the read price, not the write price. The paper credits the YOCO (“You Only Cache Once”) line of research as inspiration for this design.
Compressed Sparse Attention 2 (CSA2)
Next trick: stop every layer from keeping its own full KV cache. CSA2 assigns each of the decoder’s attention layers one of three roles:
- Full: a “producer” layer. It reads the whole history, builds the global KV cache, and selects the top-k most relevant passages (the “Top-K” sparse attention indices).
- Reindex: a “borrower with opinions.” It reuses the producer’s cached KV but runs its own small scout to pick a fresh set of relevant positions.
- Reuse: a “full borrower.” It takes both the cached KV and the position choices from an earlier layer, skipping the search entirely.
Every layer still keeps a small sliding window — a short-term memory of the last ~128 tokens — so local text flows remain perfectly precise. The huge global cache exists once instead of twenty times.
FP4 storage
Models normally cache keys and values in 16-bit numbers. V4.1-Flash trains from the start to store its global cache in FP4 — four bits, a 4× size cut — using quantization-aware training (the model learns to live with the lower precision rather than having it imposed afterwards). The sliding window stays at FP8 because the authors measured that squeezing it further hurts quality.
SWA Bounded Replay
When an old conversation resumes from SSD, the sliding-window state is normally gone (it changes every turn, so nobody stores it). Exactly rebuilding it would mean re-reading thousands of tokens through every layer. SWA Bounded Replay instead reconstructs it approximately — replaying just the most recent 128 tokens per layer — which the paper reports costs almost no quality.
The rest of the novelty pile
The paper also mentions Engram conditional memory (a 196B-parameter lookup table the model consults sparsely for facts and patterns) and DSpark speculative decoding (a small drafter proposes several tokens at once and the big model verifies them in batch — same answer, faster). These matter, but the KV-cache work is the headline.
Why the numbers are a big deal
- Global KV cache: 890 bytes per token in HBM — about 1/4 of its predecessor V4-Flash, despite being smarter.
- Persistent SSD cache footprint: about 1/8 of the previous generation.
- Decode cost across a 4K → 1M token context: grows by only ~25%, where older models balloon.
- Trained on 45 trillion tokens, with reinforcement-learning post-training that pushed it ahead of DeepSeek’s own flagship V4-Pro on several benchmarks — for less money.
DeepSeek is live with it on their API (model name deepseek-flash), retired the previous generation, and is routing V4-Pro traffic to this smaller, cheaper model. Public weights are on Hugging Face.
The one-sentence takeaway
DeepSeek-V4.1-Flash’s paper argues that the road to cheap million-token AI agents is not a bigger GPU — it is stopping every layer from hoarding its own copy of the conversation. Share the cache, shrink it to four bits, make reading cheap, and rebuild the small stuff only when asked. The rest of the industry is going to be copying this playbook.