Model risk management has a well-documented history in financial services. SR 11-7, the Federal Reserve’s 2011 guidance on model risk management, established a framework that influenced how banks, insurers, and eventually many large enterprises thought about model governance. Validate before deployment. Document assumptions. Establish independent review. Reassess when use or environment changes.
That framework made sense when models were relatively stable artifacts — scorecards, statistical models, rules engines — with formal development cycles, defined input ranges, and predictable output distributions. You could validate a model before deployment and have reasonable confidence that the thing you shipped was the thing running in production six months later.
Foundation model deployments break those assumptions. They break most of them quietly.
What traditional MRM was built to catch
SR 11-7-era MRM is designed around a linear workflow: development, validation, approval, deployment, periodic review. The model under review is a bounded artifact. Its structure, training data, and parameters are fixed at the time of validation. Risk assessments assume a stable relationship between inputs and outputs.
This works when you are validating a credit scoring model that runs on a defined set of features and produces a probability between zero and one. The validator can test it against held-out data, examine its assumptions, and certify that the model behaves as documented.
Even pre-foundation model machine learning deployments stretched these assumptions — model behavior could shift as production data drifted away from training distributions — but the basic structure held. You still shipped a specific version of a specific model.
The silent update problem
Foundation model deployments introduce a different failure mode: behavior change without a change event.
When an organization deploys a product or internal tool built on top of an API-accessed foundation model, the underlying model can be updated by the provider without formal notification, without version locking, and without triggering internal change control processes. The organization may not know the model changed. Their MRM documentation may still reference a model version that is no longer running.
This is not hypothetical. Providers have documented behavior changes across minor model versions that affected output formatting, refusal behavior, reasoning patterns, and task performance in ways that downstream applications did not anticipate. Safety fine-tuning updates, capability additions, and alignment adjustments can all alter the model’s effective behavior without anything in the organization’s change management system recording that anything happened.
The validation that happened at launch is now a description of something that no longer exists.
Prompt engineering as uncontrolled change
Even when the underlying model is stable, the effective model behavior is not.
Prompt engineering changes — system prompt modifications, few-shot example additions, instruction reordering, context window adjustments — alter how the model interprets inputs and generates outputs. These changes often happen outside formal release processes. A product team adjusts a system prompt to improve output quality. A developer tunes few-shot examples in a shared configuration file. An analyst updates a retrieval-augmented generation (RAG) template to reduce hallucination on a specific document type.
None of these trigger an MRM review. None of them appear in a version history the model risk function is tracking. But they can substantively change the model’s behavioral profile — its failure modes, its output distribution, its tendency to refuse or comply with edge-case requests.
Traditional MRM frameworks have no clean category for this. It is not a new model. It is not a material change in most organizations’ formal definitions. It exists in the gap between development and operations, where governance tends to atrophy.
Why launch-time validation cannot account for drift
Even if the underlying model stays constant and prompts are properly versioned, behavioral drift can still occur through changes in production data.
Foundation models embedded in workflows that incorporate retrieved context — customer data, document content, conversation history — are effectively operating on a continuously changing input distribution. The validation that characterized model behavior before launch was performed against a snapshot of that distribution. As the distribution changes, performance on the actual population of production queries can diverge from validated behavior without any model change, prompt change, or documented event.
This is analogous to the data drift problem in traditional ML, but harder to monitor because foundation model outputs are often high-dimensional and human-evaluated rather than numeric predictions against clear ground truth. Detecting that a summarization model has gradually become more prone to confident fabrication on a certain document type requires ongoing evaluation infrastructure, not a one-time validation exercise.
What credible ongoing MRM would require
Getting this right is not simple, but the requirements are not mysterious.
First, behavioral versioning at the output level. If the model or its prompts change, there needs to be a record of what changed and when, along with sampling of pre- and post-change output to characterize the effect. This means treating prompts and retrieval configurations as change-controlled artifacts.
Second, provider model change notification processes. Organizations relying on API-accessed foundation models need contractual or operational mechanisms to know when the underlying model changes. Some providers offer model version pinning for exactly this reason. Where version pinning is not available, the MRM framework should document that assumption as a residual risk, not quietly assume stability.
Third, ongoing behavioral evaluation, not just pre-launch validation. This means maintaining evaluation sets representative of production use, running them regularly, and having a threshold at which behavioral deviation triggers review. The SR 11-7 concept of periodic reassessment is correct in principle; the execution needs to happen at a cadence that matches the rate of silent change in foundation model environments.
Fourth, scope clarity on what counts as a model change. Most organizations currently lack a definition of “material change” that covers prompt modification, retrieval configuration changes, or provider-side model updates. Without that definition, the change control process cannot trigger because nothing looks like a change.
Bottom Line
MRM frameworks are not wrong. The core principles — independent validation, documentation of assumptions, ongoing monitoring, escalation when behavior changes — are sound and apply directly to foundation model deployments.
The gap is in operationalizing them for an environment where the model is not a static artifact, where changes can arrive silently from third-party providers, and where the most consequential behavioral modifications often happen in configuration files rather than formal release cycles.
A model risk program that validates at launch and then reassesses annually is not providing coverage for the things that actually change in production. It is providing documentation that something was reviewed, which is a different thing from managing risk.
The difference matters when something goes wrong and the audit trail shows a well-validated model that nobody monitored.