What is LLM weight streaming?
Weight streaming executes a model whose weights don't fit in GPU memory by continuously transferring layers from host (or network) storage into the GPU while compute proceeds — trading bandwidth for capacity. This site measures what that actually achieves, per GPU, in the real rental market.
How is this different from public benchmark leaderboards?
Three ways: (1) streaming-specific metrics — stall %, sustained ingress, TTFT-from-cold — not peak throughput; (2) every artifact snapshots the full marketplace offer row, so results stay reproducible as the fleet churns; (3) we publish our own prediction error next to every measurement.
What does "prediction error" mean?
Before renting a device, our calibrated simulator predicts its streaming behavior from its public listing. The published error is the gap between forecast and measurement. It's how you decide whether to trust a prediction for a device we haven't measured yet.
Why should I trust these numbers?
Every figure is labeled MEASURED (with a citable artifact: machine ID, offer row, UTC timestamp) or MODELED. Where competing systems — vLLM sleep mode, TensorRT-LLM weight streaming, Run:ai streamer — own a result, we say so in print.
Does the census cover AMD / MI300X?
Not yet. Our honesty rails prohibit claims on hardware we haven't measured, and the vast.ai fleet has no MI300X supply at census time. When we rent one, it ships — labeled, measured, timestamped.
How often is the fleet re-measured?
Sweeps run on a rolling schedule against the live marketplace, stratified across PCIe bandwidth × HBM bandwidth × architecture × host type. The dataset regenerates after every sweep.
Can autonomous agents use the census?
Yes — that's a design goal. Agents can fetch the free dataset (/data/atlas.json) or pay per prediction query via x402 ($0.01, USDC on Base, no account). Discovery via /llms.txt.
What does the census cost to produce?
Sweeps run on interruptible bids with precomputed minimums. A full 30–60 machine sweep costs roughly $20–80. We publish aggregate spend — measurement should be cheap enough to be honest.