What MLPerf measures

MLPerf scope

MLPerf is an industry-standard benchmark suite measuring machine learning system performance: inference throughput, inference latency, and training throughput. It covers image classification, object detection, natural language processing, recommendation, and speech recognition workloads.

MLPerf answers: how fast can this system run inference? How many inferences per second? At what latency percentile?

MLPerf does not answer: can this system maintain accurate memory of operational history over millions of records? Can it retrieve a specific past event from a growing archive with bounded token cost?

MLPerf measures

  • ✓  Inference throughput (queries/sec)
  • ✓  Inference latency (p50, p99)
  • ✓  Training throughput
  • ✓  Model accuracy on fixed validation sets
  • ✓  Hardware platform comparisons

MLPerf does not measure

  • ✗  Operational memory recall over long archives
  • ✗  Bounded retrieval efficiency vs. full context replay
  • ✗  Archive depth vs. query cost scaling
  • ✗  Sustained AI runs over millions of continuous records
  • ✗  Factual recall from growing telemetry archives

Where TeleMemetry fits

A different layer

MLPerf measures the inference layer: how fast a model processes a given input. TeleMemetry measures the memory layer: what input gets constructed for the model when the operational archive is too large to pass directly.

A system could score well on MLPerf inference benchmarks and still be unable to handle a 2M-record operational archive without full context replay. MLPerf does not characterize this problem because it does not model growing archives. TeleMemetry does.

These are complementary measurements of different layers in an AI system stack.

Why there is no MLPerf result here

MLPerf submission requires running on specific hardware configurations in controlled environments, with results validated by MLCommons. TeleMemetry has not submitted to MLPerf.

More importantly, MLPerf workloads are not designed to measure memory recall over growing archives. A TeleMemetry MLPerf submission would require a new workload category that does not currently exist: sustained operational memory with factual retrieval recall as the metric.

TeleMemetry throughput context

For context: TeleMemetry's measured throughput numbers

These are TeleMemetry benchmark numbers, not MLPerf numbers. They are not comparable to MLPerf submissions because the workload definition, hardware certification, and metric definitions are different.

52.02
Peak records/sec
Measured · Throughput 003 · 50K records · RTX 4000 Ada
41.72
Records/sec · Ref 001
1M records · 6h 39m · RTX 4000 Ada
397
Tokens per probe
Bounded · does not grow with archive depth
12.46W
Avg GPU power · 1M run
Low-Power Watch 002 · at 47.05 RPS

What a TeleMemetry MLPerf category would look like

Hypothetical workload definition

A TeleMemetry workload in an MLPerf-style framework would define:

Archive depth: N records (e.g., 1M, 10M, 100M).
Probe frequency: One factual probe per K records.
Token budget: Maximum context tokens per probe (fixed).
Metric: Recall % at that token budget, across that archive depth.
Scaling law: How does recall change as N grows, for fixed budget?

The fundamental measurement would be: what is the recall@budget curve as archive depth scales? Systems that maintain recall at depth with a fixed budget would score well. Systems that require growing context windows would not be viable at scale.

This is speculative. No such MLPerf category exists. TeleMemetry has not submitted to MLPerf. This section is for technical buyers evaluating how to contextualize TeleMemetry relative to existing benchmark frameworks.