Skip to content

WorthPosting

  • About
  • Resource Links

Tag: vLLM

Cat Links AI News

Speculative Decoding: Getting 2-3x More Out of Every GPU Pass

Posted on September 6, 2026September 7, 2026

Watch a GPU while a large language model generates text and you’ll see something strange: for most of every forward

Continue readingSpeculative Decoding: Getting 2-3x More Out of Every GPU Pass

Cat Links AI News

vLLM Prefix Caching: The Prefill Optimization You’re Already Running

Posted on September 3, 2026

Every request that hits an LLM server pays the same tax before the first generated token appears: the prompt must

Continue readingvLLM Prefix Caching: The Prefill Optimization You’re Already Running

Cat Links Software Engineering

Speculative Decoding in vLLM: Draft, Verify, and Cut Latency Without Losing the Distribution

Posted on August 27, 2026August 27, 2026

Large language model inference has an awkward performance profile: the GPU does enormous math, then waits. Every token requires a

Continue readingSpeculative Decoding in vLLM: Draft, Verify, and Cut Latency Without Losing the Distribution

Posts navigation

Newer posts
  • Home
  • About
Copyright © 2026 WorthPosting | Signify by WEN Themes
Scroll Up