Skip to main content

Boardroom Answers · AI & Data · Product Strategy & Roadmap

Prompt engineering is where most AI products are secretly held together with tape. Walk me through how your generation layer is actually structured, and how you change it without silently regressing quality.?

The question a Chief Artificial Intelligence Officer (CAIO) asks.

The short answer

A four-layer prompt constitution — platform rules, drafting rules, adversarial council, per-module charter — all versioned as code, gated by an offline eval harness on every push, and rolled out behind a feature flag. Prompt changes get migration-grade ceremony because they carry migration-grade blast radius.

The full executive answer

The v2 generation layer is a layered constitution, not a pile of prompt strings. At the base sits a platform constitution — the invariant rules every output obeys, including the provenance and honesty discipline. Above it, drafting rules govern how boardroom prose is written; above that, a council core defines the adversarial review perspectives applied to a draft; and at the top, each module carries its own charter — the single decision that module’s output must enable — with a domain-specific principal whose job is to veto generic advice. Every layer is versioned code in the repository, reviewed like any other code, not configuration someone edits in a dashboard.

On regression safety: changes to this layer go through an offline evaluation harness with golden fixtures per module family — run on every push with zero live model calls, so a prompt change that breaks an output contract fails the build the same way a broken function would. Structural validators then check generated output at runtime against the same expectations. And the new layer itself shipped behind a feature flag: it rolls out deliberately, compared against the incumbent layer, rather than being swapped underneath customers overnight.

The honest boundary: offline evals catch contract and structure regressions; they cannot fully measure judgment quality, which is why the flag-gated rollout and the cross-provider consensus check on flagship modules stay in the loop. We treat prompt changes with the same ceremony as schema migrations, because in this product they carry the same blast radius.

Grounded in: Layered prompt architecture (constitution → drafting rules → council → module charter); eval-driven development with golden fixtures; feature-flag progressive delivery.

Want this answered live, on your data?