Models drift. Providers silently update endpoints. Prompts rot. What is your drift story — detection, retraining, and what happens when Claude changes under you?

The full executive answer

First, the framing that changes the answer: we do not train models, so classical retraining-on-drifting-data does not apply. Our drift surface is three things — provider model behaviour changing, our prompt stack evolving, and the world our advice describes moving — and each has a distinct control. On provider change: our fast-tier model is pinned to a dated version, model resolution is centralised and configuration-driven, and any model change alters the cache fingerprint, so outputs from an old model are never silently served as if from the new one. The same is true of prompts — the prompt stack carries an explicit version that busts the cache on material edits, so prompt changes are deliberate, versioned events, not rot.

On detection: every generation gets a deterministic quality score stored alongside it, and a monitoring layer aggregates average quality, worst-decile quality, provider-fallback rate, cache hit rate and latency into a health status of healthy, watch, or degraded, with a scheduled job that alerts operations on degradation. A behavioural shift in a provider model shows up there as a quality or fallback anomaly. The regression harness also re-runs on every build, so a provider change that breaks output contracts is caught before deploy in CI, and the guardrail and provenance gates catch it per-generation in production.

On the world moving: the calibration loop logs our forecasts as scoreable predictions and grades them against later real assessments — if our advice drifts from reality, the calibration record shows it numerically. Now the honest gap, stated before you probe it: we are pre-launch, so there is zero production drift history — the instruments exist, the time series does not. What I can show is the monitoring code, the alert wiring and the versioned invalidation; what only time can provide is the track record.