Ask an inference server to answer one question and you get one GPU doing something odd: running a matrix multiplication
Continue readingContinuous Batching: The Scheduling Model Behind Every LLM Serving Stack
Every conversation with a chatbot hides a growing accounting problem. Each token the model generates must attend to every token
Continue readingThe KV Cache: The Memory Bill Behind Every Token Your Model Generates
Every Go service that talks to PostgreSQL sits on top of a pool, and almost every incident review that mentions
Continue readingConnection Pooling Is a Queue: Sizing pgxpool Against PostgreSQL’s Process Model
Every allocation your program makes is a promise the kernel keeps lazily. When you write make([]byte, 512<<20) in Go, you
Continue readingVirtual Memory Is Deferred Work: Page Faults, RSS, and Why Allocation Numbers Lie
The most expensive lie a system can tell is that it accepted work it cannot process in time. It usually
Continue readingBackpressure: What Your System Should Do When Consumers Fall Behind
Every so often a profiler hands you a paradox. Your server is running eight worker goroutines, the load is perfectly
Continue readingFalse Sharing: When Your Parallel Code Is Secretly Serial
Ask a language model for JSON and most of the time it complies. Every so often it wraps the object
Continue readingConstrained Decoding: How LLMs Get Forced Into Valid JSON
Python 3.15 landed on October 9, and for the free-threading effort it is the most consequential release since the whole
Continue readingLife After the GIL: What Python 3.15’s Free-Threading Milestones Actually Change
Every database you use is a B-tree or an LSM tree underneath. PostgreSQL and MySQL’s InnoDB are B-trees. RocksDB, Cassandra,
Continue readingB-Trees vs LSM Trees: The Storage Engine Trade-Off Behind Every Database
Most services read the same rows over and over. A user profile endpoint gets hammered by the same traffic pattern
Continue readingClient-Side Caching with Redis: Invalidation Is the Whole Design