Every third-party risk management (TPRM) program in a regulated enterprise rests on one quiet assumption: the product you assessed at onboarding is the product you are running today. Foundation models consumed through an application programming interface (API) break that assumption as a matter of routine, and most programs have no control that notices.
The playbook was written for vendors whose product holds still
The reference document for bank TPRM is the June 2023 interagency guidance, issued by the Office of the Comptroller of the Currency (OCC) as Bulletin 2023-17 together with the Federal Reserve and the Federal Deposit Insurance Corporation (FDIC). It replaced the OCC’s 2013 guidance and its 2020 FAQs, and it “outlines the third-party risk management life cycle and identifies risk management principles applicable to each stage of the life cycle.” It also states that “not all third-party relationships present the same level of risk or criticality,” which is the sentence most programs use to tier vendors and decide how deep to look.
The life-cycle structure is sound, and it is the structure we recommend keeping. The problem is the cadence it implies. In practice the life cycle runs as a sequence of snapshots: a due-diligence package at onboarding, a contract that fixes the service description, an annual or risk-tiered reassessment, and an exit plan. Each snapshot describes the vendor’s product as it existed on a date. For a core-banking platform that is adequate, because the product changes through announced releases the customer can test. A hosted foundation model is a different kind of thing. The vendor retrains, fine-tunes and re-routes the model behind a stable endpoint name, often with no release note the customer will see, and the behavior your due-diligence file certified is not the behavior answering your users.
Models change under you, and the change is measurable
This is not hypothetical. In a study published in the Harvard Data Science Review in March 2024, Lingjiao Chen, Matei Zaharia and James Zou compared the March 2023 and June 2023 snapshots of GPT-4 and GPT-3.5 on identical tasks. GPT-4’s accuracy at identifying prime numbers fell from 84.0 percent to 51.1 percent. Its compliance with the authors’ answer-extraction instructions fell from 99.5 percent to 0.5 percent. Other tasks moved the other way; GPT-4’s exact-match rate on a multi-step question-answering benchmark rose from 1.2 percent to 37.8 percent. The direction is not the point. The point is that the authors found the behavior of the same named service “can change substantially in a relatively short amount of time,” and that “these unknowns make it challenging to stably integrate LLMs into larger workflows,” where LLM is the large language model behind the service.
Read that against a TPRM file. A due-diligence review completed in March would have certified a model that no longer existed in June, and nothing in a conventional program — not the contract, not the annual reassessment, not the vendor’s assurance report — would have surfaced the change. The authors name the missing control: “Our findings highlight the need to continuously monitor LLMs’ behavior over time.”
The due-diligence file describes the model you assessed, not the model answering your users today.
The due-diligence file describes the model you assessed, not the model answering your users today.
The model-risk rulebook was rewritten in April, and it stepped around generative AI
The natural place to look for help is model risk management, which has fifteen years of practice in validating models a bank did not build. On April 17, 2026, the Federal Reserve, the OCC and the FDIC issued revised interagency model risk guidance that, as KPMG’s regulatory alert records, supersedes SR 11-7 and OCC Bulletin 2011-12 and is published by the OCC as Bulletin 2026-13. The alert reads the revision as one that “reaffirms that vendor and third-party models are subject to the same model risk management principles as internally developed models,” and states plainly that “an important element of model risk management is the validation of vendor products.”
Then it draws a line. Orrick’s analysis notes that a footnote excludes generative and agentic AI models from the guidance’s scope, describing them as “novel and rapidly evolving,” and directs banks to rely on their broader risk management and governance practices for those tools. KPMG reads the same passage the same way: generative and agentic models “are not within the formal scope of the guidance but should follow a banking organization’s established risk management and governance practices.” The agencies anticipate “a forthcoming request for information on AI models, including generative AI and agentic AI.” Until that produces something, the models most likely to change under you are the models the model-risk guidance declines to govern, which leaves TPRM holding the problem with a toolkit built for software.
Outside banking there is less, not more. The National Institute of Standards and Technology (NIST) published its Generative AI Profile in July 2024 as a companion to the AI Risk Management Framework, a framework NIST describes as “intended for voluntary use.” It is a shared vocabulary, not a control standard, and it does not tell procurement what to put in a contract.
What a third-party program that survives model change looks like
Our assessment is that the TPRM life cycle does not need replacing; three stages need a model-specific control, and one needs a different cadence.
- Due diligence records a version, not a vendor. The entry in the third-party register names the exact model identifier and snapshot date the assessment covered, and the application inventory ties each use case to that identifier. Where the vendor offers pinned or dated model versions, pin them; where it does not, record that fact as an accepted risk with a named owner.
- The contract obliges the vendor to tell you when the model changes. The European Union has already written this expectation into law for high-risk systems. Article 25(4) of the EU AI Act requires providers of high-risk AI systems and their component suppliers to specify “by written agreement” the “necessary information, capabilities, technical access and other assistance” the provider needs to meet its obligations, and it provides that the AI Office may develop and recommend voluntary model terms “free of charge in an easily usable electronic format.” A U.S. enterprise does not need to be in scope to borrow the clause: change notification with a defined lead time, access to version documentation, and a right to run your own evaluation before a forced migration.
- Ongoing monitoring becomes a scheduled evaluation, not an annual questionnaire. A fixed, task-specific test set run against the production endpoint on a cadence — weekly for customer-facing uses, monthly for internal ones — is the control that detects the drift the Chen study measured. The output is a trend line with a threshold that triggers second-line review.
- Termination planning assumes the vendor will retire the model before you are ready. The exit plan names the second model you have already evaluated, the tests that would need to be re-run, and the owner who signs off on the switch.
None of this is expensive relative to the program it sits inside. The register and contract changes are a quarter’s work for the third-party risk team with counsel; the evaluation harness is a small engineering investment the AI platform team owns thereafter. What it buys is the ability to answer the question an examiner or a board member will eventually ask: which model made that decision, and did we know it had changed?
The vendor-risk discipline you already have is the right structure for foundation models; its assumptions are not. Software vendors ship releases you can see. Model vendors ship behavior you have to measure. Until the regulators decide whose rulebook generative AI belongs in, the enterprise that pins versions, contracts for change notice, and tests its models on a schedule is the one whose third-party risk file is still true on the day it is read.