Skip to main contentMain content

Boardroom Answers · Technology & Platform · Technical Architecture & Scalability

When something is slow but not broken, how do you see it? Describe your observability stack honestly?

The question a VP Engineering / Head of Platform asks.

The short answer

OpenTelemetry spans (exported over OTLP when a collector is configured, structured log lines otherwise), Sentry errors, PII-masked structured logs, a cost ledger for every model call, and two SLOs measured from our own request logs — with the unmeasured ones, AI latency and availability, flagged as such in the code itself.

The full executive answer

Three signals, per the standard observability model. Traces: API requests, AI generations, AI-gateway calls, connector syncs and outbound calls each run inside an OpenTelemetry span — the open, vendor-neutral instrumentation standard. An OTLP exporter is built in and exports those spans to your collector of choice when a collector endpoint is configured for the deployment; where none is configured, each span is written as a structured log line with its name, duration and status, so 'slow but not broken' is still answerable — from the logs rather than a trace timeline. Errors and exceptions: Sentry, with full stack context, when a Sentry DSN is configured. Structured logs: a logging layer that machine-masks personal data by default, so debugging never becomes a privacy incident. Product-side, PostHog usage analytics is built in, consent-gated and off until its project key is configured; with it on, 'slow' can be correlated with 'where users actually are'.

The AI layer gets its own telemetry, because in this product the model calls are the performance story: a cost ledger records every billed model call — provider, model, token counts and cache hits — and each insight records whether the fallback provider was used. What the ledger does not record yet is how long each generation took, so AI generation latency is a target, not a measured objective, until that is added.

The honest boundary, which I flag before you ask: the availability and AI-generation latency objectives are defined in code but explicitly marked unmeasured — an external probe and per-generation timing are the wiring that turns them live, and that is scheduled ahead of any SLA conversation. API read latency and the server error rate are measured today, from a daily rollup of our own request logs, with a scheduled job alerting on breach. So: deep instrumentation on the paths that matter most, two measured SLOs, and a declared — not hidden — gap on the rest. Instrument, measure, then promise, in that order.

Grounded in: OpenTelemetry / OTLP standard; Google SRE golden signals

Want this answered live, on your data?