The fundamental tradeoff: scale vs. query cost. At 2M records, full replay is not viable. Bounded retrieval is.
A long-running AI system accumulates an operational archive. When the system needs to answer a question about past events, one approach is to replay the full archive as model context. This is reliable but the cost grows linearly with archive depth.
At 1,000,000 records, a full replay at 50 tokens per record requires 50 million tokens of context per query. At 2,000,000 records, 100 million. Context windows do not scale this way. At production telemetry rates, full replay is not a viable architecture.
Token cost comparison
Query cost is determined by the token budget, not archive depth. The retrieval set size is bounded and configurable. Over 2M records, the cost per probe is the same as at 1,000 records.
Cost does not grow with archive size.
Query cost grows linearly with the number of archived records. At 1M records with 50 tokens/record, each probe requires 50M context tokens. Most LLM providers cap context windows well below this.
Not viable at production telemetry scale.
Telemetry terminal benchmark
Run on real-time ADS-B aviation telemetry. 2,365 events ingested, 239 aircraft tracked. Probe: factual recall of specific aircraft state at a past timestamp.
Label note: The 106-token bounded figure is measured. The 49,894-token full-replay figure is estimated from event count and average token size. The 471x ratio reflects this estimated comparison baseline.
nuPlan AV benchmark
At 2,000,000 archived telemetry records, a full-replay approach at an average of 100 tokens per record would require 200 million context tokens per probe. No commercial LLM context window supports this. Bounded retrieval on the same archive used approximately 400 tokens per probe (measured, Ref 002 configuration).
The efficiency claim is not the primary claim. The primary claim is 100% verified recall within benchmark scope, at bounded evidence-packet cost. The efficiency data demonstrates that the bound is practical: 400 tokens per probe is a cost that fits within any LLM context window, enabling real-time querying of arbitrarily large archives.
A system that achieves 100% verified recall but requires 200M tokens per probe is not deployable. A system that achieves 100% verified recall at 400 tokens per probe is.