Boardroom Answers · Technology & Platform · Technical Architecture & Scalability
When something is slow but not broken, how do you see it? Describe your observability stack honestly?
The question a VP Engineering / Head of Platform asks.
The short answer
OpenTelemetry spans (exported over OTLP when a collector is configured, structured log lines otherwise), Sentry errors, PII-masked structured logs, a cost ledger for every model call, and two SLOs measured from our own request logs — with the unmeasured ones, AI latency and availability, flagged as such in the code itself.
The full executive answer
Three signals, per the standard observability model. Traces: API requests, AI generations, AI-gateway calls, connector syncs and outbound calls each run inside an OpenTelemetry span — the open, vendor-neutral instrumentation standard. An OTLP exporter is built in and exports those spans to your collector of choice when a collector endpoint is configured for the deployment; where none is configured, each span is written as a structured log line with its name, duration and status, so 'slow but not broken' is still answerable — from the logs rather than a trace timeline. Errors and exceptions: Sentry, with full stack context, when a Sentry DSN is configured. Structured logs: a logging layer that machine-masks personal data by default, so debugging never becomes a privacy incident. Product-side, PostHog usage analytics is built in, consent-gated and off until its project key is configured; with it on, 'slow' can be correlated with 'where users actually are'.
The AI layer gets its own telemetry, because in this product the model calls are the performance story: a cost ledger records every billed model call — provider, model, token counts and cache hits — and each insight records whether the fallback provider was used. What the ledger does not record yet is how long each generation took, so AI generation latency is a target, not a measured objective, until that is added.
The honest boundary, which I flag before you ask: the availability and AI-generation latency objectives are defined in code but explicitly marked unmeasured — an external probe and per-generation timing are the wiring that turns them live, and that is scheduled ahead of any SLA conversation. API read latency and the server error rate are measured today, from a daily rollup of our own request logs, with a scheduled job alerting on breach. So: deep instrumentation on the paths that matter most, two measured SLOs, and a declared — not hidden — gap on the rest. Instrument, measure, then promise, in that order.
Grounded in: OpenTelemetry / OTLP standard; Google SRE golden signals
The natural next questions
Related governed answers
- Serverless functions have hard time limits. Your AI analyses are slow by nature. So your platform dies by timeout exactly when it is doing its most valuable work — during my board meeting. Explain?
- Your entire platform sits on Vercel, Supabase and Clerk — three venture-backed vendors. What is your concentration risk and what is my exit if one of them dies?
- Your platform is built around today's AI patterns — prompts, RAG, vector search. When the AI paradigm shifts again in eighteen months, does your data architecture survive?
Want this answered live, on your data?