Skip to main content

Boardroom Answers · Technology & Platform · Technical Architecture & Scalability

When something is slow but not broken, how do you see it? Describe your observability stack honestly.?

The question a VP Engineering / Head of Platform asks.

The short answer

OpenTelemetry traces, Sentry errors, PII-masked structured logs, and a per-call AI latency ledger driving the one SLO measured live today — with the unmeasured SLOs flagged as such in the code itself.

The full executive answer

Three signals, per the standard observability model. Traces: OpenTelemetry — the open, vendor-neutral instrumentation standard — is wired through the stack with OTLP export, so a request can be followed across route handler, database calls and AI provider calls, and 'slow but not broken' becomes a spans-on-a-timeline question rather than guesswork. Errors and exceptions: Sentry, with full stack context. Structured logs: a logging layer that machine-masks personal data by default, so debugging never becomes a privacy incident. Product-side, PostHog gives usage analytics, so 'slow' can be correlated with 'where users actually are'.

The AI layer gets its own dedicated telemetry, because in this product the model calls are the performance story: a latency ledger records every generation — duration, provider, token counts, whether fallback fired, cache hits — and that ledger feeds the one service-level objective that is genuinely measured live today, AI generation latency, with a scheduled job alerting on breach.

The honest boundary, which I flag before you ask: platform-wide availability and API-latency SLOs are defined in code but explicitly marked unmeasured — external synthetic probes and APM ingestion are the wiring that turns them live, and that is scheduled ahead of any SLA conversation. So: deep instrumentation on the paths that matter most today, one live-measured SLO, and a declared — not hidden — gap on the rest. Instrument, measure, then promise, in that order.

Grounded in: OpenTelemetry / OTLP standard; Google SRE golden signals

Want this answered live, on your data?