Methodology

1 · The standard benchmark

Every device runs the same artifact (gpu_bench_streamos.py):

• Raw pinned H2D bandwidth — sustained host-to-device copies at 437.5 MB and 875 MB block sizes. This is the actual ingress ceiling.
• Streamed-weights decode — a standardized workload that streams an 80-layer, 70B-class weight layout through the device during decode. Variants: INT4 (437.5 MB/layer) and FP8 (875 MB/layer) wire formats, batch 1 and 32. (Engine implementation details are withheld pending patent filing; the measurement protocol and all results are published.)
• Reported per layer: copy ms, compute ms, stall ms — plus aggregate tokens/sec, stall %, and (via nvidia-smi sampling) average watts and joules/token.

2 · Stratified fleet sampling

The census targets 30–60 machines per sweep, stratified across the live vast.ai marketplace by PCIe bandwidth × HBM bandwidth × architecture × verified/community. Sampling is reliability-weighted and deliberately includes the ugly tail — community hosts, PCIe risers — because that's where calibration degrades and buyers get burned. Stated bias: the rentable fleet is not an unbiased census of the world's GPUs; results describe the marketplace, not the population.

3 · Artifact discipline

Every run snapshots the full marketplace offer row (advertised specs + price + geolocation) and the machine_id into the artifact. The marketplace is non-stationary; an artifact without its offer row is unreproducible and doesn't ship. Schema: kofta.atlas.v1 (see API docs).

4 · The calibration model

Before renting any device, our simulator predicts its streaming behavior from its public listing: effective H2D bandwidth, stall %, tokens/sec, TTFT. Predictions are anchored to measured constants (first anchor: NVIDIA H100 SXM, July 2026) and scaled by device specs. The predicted-vs-measured error ledger is published per device — the median error on this site's homepage is ours, not a competitor's.

5 · Cost discipline

Sweeps run on interruptible bids with precomputed minimum bids and retry-elsewhere on preemption. Weight caches live on local volumes pinned to machine IDs, making 50-run sweeps practical in an evening. Every dollar is ledgered per experiment and published in aggregate.

6 · Honesty rails

• Every figure is MEASURED (artifact cited) or MODELED. No third state exists.
• We cite vLLM sleep mode, TensorRT-LLM weight streaming, Run:ai model streamer, ServerlessLLM, and PipeBoost/ParaServe/HydraServe wherever they own a result — including results that beat us.
• No ROCm/MI300X claims until we rent and measure one. No optical numbers derived from electrical experiments. TCO only from marketplace price data.
• Quantized-transfer claims cite HOBBIT; our steady-state claims are dense per-token re-streaming.