LLMs are stochastic. If a regulator asks you to reproduce the exact analysis a board saw six months ago, can you — and if not, what CAN you prove?

The full executive answer

I will separate what is reproducible from what is provable, because conflating them is how vendors get caught. Bit-identical regeneration from a live model is not something anyone can honestly promise — large language models are not perfectly deterministic even at low randomness settings, and providers update infrastructure. We run at a low, fixed temperature — think of that as turning the model’s creativity dial most of the way down, applied identically to both providers — which makes outputs stable, not identical.

What we can prove is stronger than replay: the exact artifact itself. Every insight the board saw is persisted with its content, model, provider, token counts, grounded sources, framework citations, quality score and timestamp, under an append-only audit log — so the question "what did the board see and where did it come from" is answered by retrieval, not regeneration. The cache fingerprint documents exactly which input state, prompt version and model produced it. And because identical inputs hit the cache rather than the model, asking the same question twice returns the same stored, validated answer — operational determinism where it matters.

This is deliberately the SR 11-7 posture from banking model risk: you do not prove a model would say the same thing again; you prove what it said, from what inputs, under which controls, with which human accountable. Six months later I can hand a regulator the full generation record, the consensus diff if one ran, and the audit chain — that is a better evidentiary position than a replay claim that any expert witness would dismantle.