If an API runs on several replicas, their individual p95 values cannot tell you the p95 across all requests. A busy replica accounts for more of the traffic than a quiet one. In Prometheus, combine the bucket counts across replicas and calculate p95 once from the combined distribution. See the introduction to latency percentiles for how to interpret the resulting quantile.
Check the buckets, request counts, and units
The examples use a classic histogram named http_request_duration_seconds. They assume instrumentation records one duration in seconds per request. Labels such as service, job, and instance depend on your setup.
| Series | Value |
|---|---|
http_request_duration_seconds_bucket{le="0.3"} | Cumulative number of observations at or below 0.3 seconds |
http_request_duration_seconds_bucket{le="+Inf"} | Unbounded bucket with the cumulative count of all observations |
http_request_duration_seconds_count | Cumulative count of all observations |
http_request_duration_seconds_sum | Cumulative sum of observed durations in seconds |
The le boundary is inclusive. A 100 ms observation contributes to both le="0.1" and le="0.3", so adding those buckets would count it twice. To count observations in (0.1, 0.3], subtract the smaller cumulative count from the larger one. Use the same bucket boundaries and units across the replicas you plan to aggregate. See the
Prometheus metric type documentation
.
Calculate p95 across the service
histogram_quantile(
0.95,
sum by (le) (
rate(http_request_duration_seconds_bucket{service="checkout"}[5m])
)
)
rate(...[5m])estimates the per-second increase in each bucket counter over five minutes. Apply it before aggregation so Prometheus can handle resets in each individual series.sum by (le)adds the rates across replicas for each boundary. To return separate results per service, usesum by (service, le)and select the intended services in the metric selector. Check that your selector does not include multiple measurements of the same request.histogram_quantile(0.95, ...)estimates p95 for the combined distribution in seconds. Multiply by 1,000 if you want the output in milliseconds.
The five-minute range looks backward from each evaluation timestamp. A range that is too short relative to the scrape interval can leave some series without enough samples for rate. On low-volume services, percentiles can also fluctuate because they describe very few requests. Plot the request volume alongside them. See the function references for
histogram_quantile
and
rate
.
Work through two replicas
Assume the counters grow at steady rates between scrapes. These are the five-minute rate values.
| Replica | Within 100 ms | Within 300 ms | Within 500 ms | Total |
|---|---|---|---|---|
| A | 88 req/s | 90 req/s | 90 req/s | 90 req/s |
| B | 0 req/s | 2 req/s | 10 req/s | 10 req/s |
| Combined | 88 req/s | 92 req/s | 100 req/s | 100 req/s |
The 95% rank corresponds to 95 req/s. It lies between the cumulative rates of 92 req/s at 300 ms and 100 req/s at 500 ms. Classic histogram interpolation yields 300 + (500 − 300) × (95 − 92) / (100 − 92) = 375 ms. If all eight requests per second in that bucket actually took 310 ms, the true p95 would be 310 ms and the histogram estimate would still be 375 ms.
The two replicas have estimated p95 values of about 97 ms and 488 ms. Their arithmetic mean is about 292 ms, and their request-weighted mean is about 136 ms. Both differ from the combined histogram estimate of 375 ms. The counts within each bucket are what you can add across replicas.
Measure a 300 ms SLO with a bucket ratio
For an objective like “99% of eligible requests finish within 300 ms,” divide requests within the threshold by all eligible requests. When the threshold aligns with a bucket boundary, you can compute this from the counter rates without estimating the distribution inside a bucket.
sum(rate(http_request_duration_seconds_bucket{service="checkout",le="0.3"}[5m]))
/
sum(rate(http_request_duration_seconds_count{service="checkout"}[5m]))
The example gives 92 / 100 = 0.92, or 92% over five minutes. If the SLO uses a 30-day window, sum increase(...[30d]) for good events and total events separately before dividing. A simple average of five-minute ratios gives quiet windows the same weight as busy ones. If only successful requests are eligible, apply the same definition and filter to numerator and denominator. If errors count as SLO failures, count all eligible requests in the denominator and successful requests within the threshold in the numerator.
Distinguish boundary mismatches, missing data, and zero traffic
- Different boundaries across replicas. If only some replicas export
le="0.3", the numerator omits the others and the compliance ratio is biased downward. Use consistent boundaries and keep cohorts with different boundaries separate during a rollout. If no replica exports the threshold bucket, the numerator is an empty vector. - Zero requests or missing series. When the counters exist but there are no requests, the ratio is
0/0, orNaN. The p95 is alsoNaN. An absent input series may instead yield an empty result. Plot request volume alongside the ratio and check scrape health withupor monitored target counts. Partial data loss can still leave a plausible-looking aggregate. - p95 plateaus at the largest finite boundary. If the quantile falls inside the
+Infbucket,histogram_quantilereturns the largest finite boundary for a classic histogram. When that boundary is 500 ms, a result of 500 ms alone cannot tell you how far above it the slow requests were. Subtract the last finite bucket from the+Infbucket to measure traffic beyond the boundary.
Include the SLO threshold as a boundary in the instrumentation. For p95 estimation, add useful boundaries around the range where the quantile typically lands to reduce uncertainty from interpolation. See the Prometheus guide to histograms and summaries .
FAQ
How do I calculate p95 across multiple replicas?
Apply rate to each bucket series, aggregate with sum by (le), and then run histogram_quantile. To return one value per service, use sum by (service, le).
How do I measure the fraction of requests within 300 ms?
With a 300 ms bucket boundary, sum the rate of buckets with le=“0.3” and divide by the summed rate of _count over the same population and window. For a metric in seconds, 0.3 is 300 ms.
What should I display when there is no traffic?
With zero observations, p95 and the threshold compliance ratio are undefined. Display request volume and scrape health separately so you can distinguish zero traffic from missing measurements.