Context Window Management for Multi-Agentic Platforms
Eight strategies ordered by leverage-per-complexity — from prompt caching to bandit-scored tool selection — with the economics, code, and latency budget for each.
⚙⚙⚙⚙⚙ 2026.07.01
A publication for engineers shipping inference
The engineering
layer of AI.
Deep technical writing on LLM, GPU, and ML systems internals — decoded from silicon to system to algorithm, for the engineers who already know what RAG is.
fp4 quantizer E2M1 · 4-bit float
input · fp32 3.7194
quantized · fp4 4.0
quantization error 0.281
−6−2026
/01
Pick your depth Three layers, one publication
Silicon 03
The hardware that runs the model.
HBM hierarchy, NVLink topology, tensor-core generations, CUDA streams — kernel-level reality.
System 09
The infra that serves the model.
KV cache, paged attention, continuous batching, parallelism strategies, NCCL collectives.
Algorithm 11
The math that is the model.
Attention variants, MoE routing, quantization, alignment, long context, positional encoding.
/02
View all → Latest
The KV Cache: A Definitive Engineering Analysis
Naive autoregressive inference recomputes the entire attention matrix at every decoding step. KV caching stores the immutable K and V projections instead, converting per-step O(t·d) cost to O(d) exactly — no approximation.
⚙⚙⚙⚙○ 2026.06.27
Flash Attention: A Complete Architectural Autopsy
From O(N²) HBM catastrophe to Hopper-native warp-specialized pipelines: every trick, every number, every tradeoff decoded. Standard attention is not bottlenecked by arithmetic — it is bottlenecked by memory bandwidth.
⚙⚙⚙⚙⚙ 2026.06.27
PagedAttention: How vLLM Borrowed Virtual Memory to Unlock GPU Serving at Scale
Static KV cache pre-allocation wastes 60–80% of GPU memory under production traffic patterns. PagedAttention applies OS-style virtual memory paging to the KV cache, enabling 3× higher concurrency on the same hardware.
⚙⚙⚙⚙○ 2026.06.27
Continuous Batching: The Scheduling Insight That Made LLM Serving Actually Work
Static batching blocks an entire batch until its longest sequence finishes. Moving the scheduling boundary from request to iteration frees GPU slots at every decode step, taking utilization from 20–35% to 70–85%.
⚙⚙⚙○○ 2026.06.27
The Missing Decision Guide: Parallelism Strategies in LLM Training and Inference
NVLink delivers 9–18× more bandwidth than InfiniBand NDR. Every parallelism decision is a negotiation with that ratio — this is the decision tree from hardware topology and model architecture to a concrete TP/PP/SP/DP configuration.
⚙⚙⚙⚙⚙ 2026.06.27
/03 — Editorial thesis
We decode AI one layer at a time.
01 Silicon
What
the hardware can do
hbm · nvlink · tensor cores 02 System
How
inference actually runs
kv cache · batching · parallelism 03 Algorithm
Why
the math works
attention · moe · quantization WHAT WE ARE
Dense
Every article cites the paper, derives the math, and shows the code. No padding.
WHAT WE PROVE
Verifiable
Numbers you can reproduce on your own hardware — never vendor benchmarks.
WHAT WE REFUSE
No filler
Written for engineers building inference infra, not engineers explaining what inference is.