Skip to main content

Boardroom Answers · Technology & Platform · Operational Excellence

Show me your p95 latency and your performance benchmarks under realistic load.?

The question a Chief Technology Officer (CTO) asks.

The short answer

AI latency — the one that matters here — is measured live with an SLO. API latency and availability are declared targets, not measurements, until monitoring lands; the platform itself refuses to blur that line.

The full executive answer

Here is exactly what is measured and what is not — and the distinction is built into the product itself. What is measured live today: AI generation latency, from a latency ledger inside our AI gateway that records every model call. That is the latency that dominates user experience in this product, because the expensive operation is the AI analysis, and it carries a real, measured service-level objective. What is not yet measured: end-to-end API latency and availability. In our reliability code those objectives — for example, 95% of public API reads under 800 milliseconds — are explicitly flagged as targets awaiting monitoring, with a comment forbidding anyone from presenting them as measured commitments.

Why this posture rather than a benchmark slide: we are pre-launch with no production traffic history, and synthetic benchmarks of an unloaded system are close to meaningless — Google's SRE discipline is that you set service-level objectives, instrument them, and manage to an error budget, rather than quoting lab numbers. The instrumentation groundwork is already in: OpenTelemetry distributed tracing across the stack, Sentry for errors, and an SLO alerting job that runs on schedule.

The roadmap is: external synthetic probes on the critical user journeys plus APM ingestion, turning those three flagged targets into measured SLOs with public error budgets — before we sign any latency or availability SLA. If a vendor ever shows you a clean p95 chart with no production traffic behind it, that is the number to distrust.

Grounded in: Google SRE — SLOs and error budgets

Want this answered live, on your data?