TL;DR
- An average response time describes one summary of a defined set of requests. It does not say how long each buyer will wait.
- Keep the mean, percentiles and task boundary separate. A fast component does not prove that a complete workflow is fast.
- Historical systems research shows why slow responses matter, while separate modeling work shows that a mean can still be useful with other inputs and assumptions.
- Publish the clock, workload, sample and spread behind a speed claim. Use an approved test for the buyer's task rather than borrow another system's result.
Keep the tested task beside the time
This matters when product pages feed a buyer's research, including answers in AI search. “Average response: 200 ms” may be accurate for a test and still leave the central buying question open: can this service finish the task I need, under the conditions I face? Clear evidence gives that question a better starting point.
The Task Boundary Changes the Number
Dean and Barroso's 2013 article, The Tail at Scale, reports measurements from a Google service. A root node sent work to many leaf servers. Its table separates waiting for one random leaf from waiting for most or all leaf requests. The chart takes the 99th-percentile column from that table. These times were observed at the root node inside Google's service, not by an end user's client.
| Completion condition, measured at the root | 99th-percentile latency, milliseconds |
|---|---|
| One random leaf finishes | 10 |
| 95% of all leaf requests finish | 70 |
| All leaf requests finish | 140 |
Source: Dean and Barroso, Communications of the ACM, February 2013, Table 1. Historical service measurements; the paper does not give a sample count or collection dates for this table. These are different completion conditions, not a current vendor benchmark or a before-and-after test.
The 10 ms, 70 ms and 140 ms figures answer different questions about the same service structure. The 99th percentile is a cutoff near the slow end: roughly 99% of measured times fall at or below it, under the stated method. It is not the mean or the maximum. Here, “95% finish” describes which leaf work must finish, while “99th percentile” describes the spread of measured times for that condition. Those percentages play different roles.
For a buyer, the lesson is to name what must be complete. An API's first reply, a fully received result and a finished report may use different clocks. A component reading should not become a whole-workflow promise just because the smaller number fits a headline. The engineering team must choose a test that matches the task.
Dean and Barroso, February 2013, Table 1. Root-node clock, not a client measurement; completion conditions differ. Sample count and dates not stated. Historical 99th percentiles, not means or current vendor benchmarks. Original article and table.
An Average Can Be Useful Without Being Sufficient
The evidence does not support discarding means. Nguyen and colleagues' 2016 HotCloud paper used mean and variance as inputs to a tail-latency model. Variance describes spread around the mean. For its system-level model and hybrid cases at 90% load, prediction errors for the 99th percentile were within 10%. That is model error under tested conditions, not a latency reduction.
The distinction changes what a public claim can say. A mean used with spread, a model and checked assumptions can support engineering work. A mean printed alone on a sales page does not carry those extra inputs with it. Neither paper gives a target response time for every B2B service.
| Evidence | Actual finding | What it does not establish |
|---|---|---|
| Dean and Barroso, 2013, real service table | Root-measured 99th percentiles of 10, 70 and 140 ms for different completion conditions | A universal ratio from component time to buyer wait; current cloud performance |
| Nguyen et al., 2016, modeled and hybrid systems | Within-10% error for system-level 99th-percentile predictions at 90% load in the tested cases | A mean-only prediction; accuracy for every workload or low-load regime |
| Nguyen et al., test scope | Mean and variance feed a model that assumes independent task response times; homogeneous subsystems simplify it | Full production validation: a complete Solr-based EC2 search-engine test was still underway |
Source: 2013 article and 2016 paper, Sections 2.2–2.3. The studies come from different research teams and are not a ranking of products. Google authors reported their service; the university team tested a model. The 2016 work used a measured Google leaf-node distribution in some tests. It was not an independent replication of the 2013 service table. Neither tested the effect of web copy or AI citations.
Read the table as a reason to ask for scope, not as a reason to promise one of its numbers. Historical results can explain a measurement problem without forecasting your buyer's wait. A public claim needs evidence from the product, version and workload it names.
What Belongs Beside a Public Latency Claim?
First, state the clock. Does it begin when the client sends the request, when the server receives it, or when processing starts? Does it stop at the first byte or at the full result? These boundaries affect what the buyer can infer even before you choose a statistic.
Then describe the work and the sample. The relevant facts may include the operation, payload size, region, load, cache state, test period, request count and treatment of failures. Do not quietly remove timeouts from a result that appears to cover all attempts. If the approved report excludes them, keep that limit clear and report failure behavior separately.
Also label the summary. “Typical” can mean different things to different readers. Say arithmetic mean or median when that is what was measured. A percentile needs its own label and method. If data is split by region or endpoint, avoid making an untested claim about all users from one slice.
Our API allowance guide addresses how much work fits a usage rule. This question is about how long defined work takes and how those times vary. The uptime guide addresses a different measure again: what counts as available.
| Field | Fictional record | Claim it permits or blocks |
|---|---|---|
| Task | Receive a complete result for one named search operation | Does not describe report generation or every API endpoint |
| Clock | Client send to full result received | Includes the measured client path; not just server processing |
| Sample | An invented set of 100 response times, used only for the calculation below | No real test, product result or industry benchmark |
| Summary and spread | 99 times of 100 ms; one of 10,100 ms; mean 200 ms | A 200 ms mean does not mean all responses took 200 ms |
| Owner's release check | Match real copy to the approved test record, method and version | A service promise cannot be created from this fictional set |
The draft in this fictional case says “Every search returns in 200 ms.” Its evidence is the invented set in the table. The arithmetic is (99 × 100 + 10,100) ÷ 100 = 200 ms. The mean is correct; the word “every” is not. One response takes 10.1 seconds. No response in the set took 200 ms, and its median is 100 ms. This is a worked calculation, not a test we ran.
A truthful description of the example is: “This illustrative set has an arithmetic mean of 200 ms. It contains 99 responses of 100 ms and one of 10,100 ms.” A real product page would also need its actual workload, conditions and approved results. The editor checks those records with the performance owner before replacing a broad speed promise.
The useful outcome is a claim whose scope a buyer can inspect. It does not require a long technical report in the hero section. A concise result with a clear label and a link to the full method can keep the meaning intact. A number with no basis cannot do that job.
Latency Claims: Common Questions
Is the median the same as the average?
The median is the middle of an ordered set; the arithmetic mean uses every value in its sum. They can differ. Label the measure you actually used instead of treating either word as a general promise about each request.
Does a 99th percentile cap the slowest request?
No. It describes a percentile of the measured set under its stated method. Some observations may be slower. Keep maximum, timeout rules and any contractual deadline separate from that statistic.
Can we compare two vendors' response times?
Only after checking whether their task, clock, workload and summary match well enough for the buying question. A server-only mean and a client-observed percentile are different measures. If the bases do not match, explain the gap rather than name a winner from the smaller number.
Will more percentiles improve AI visibility?
That was not tested here. Add measures that help answer the buyer's question, with their scope and source. Extra numbers, headings or structured data do not guarantee retrieval, citation or a better sales result.
Clarify the claim behind the speed figure
If buyers or AI comparisons turn your test average into a universal speed promise, SemShift's AI visibility audit can help align the public answer with the approved evidence. Bring the page, test record and buyer task. Performance judgments remain with your engineering team.