Concept detail
Prefix caching
2 tagged videos · 3 connection videos · degree 0
Publication timeline
- 2026-094 videos
Source videos
Observed neighbors
Agentic loops
Allows repeated system/tool prefixes to be shared, saving prefill compute across many agent calls.
1 source videos
Deep dive on LLM Inference at Scale — Harshul Jain, Audible & Tanmay Sah, Independent AI Researchercache-aware routing
routing must be locality-aware to preserve prefix cache hits across a distributed cluster
1 source videos
The Frontier AI Inference Cloud for Agents — Byung-Gon (Gon) Chun, FriendliAIprompt design
Prompt layout determines cache hit rate; the cache contract constrains how prompts may be authored and versioned.
1 source videos
CMU AI Agents 2026: 3. Long Context Modeling for Agents