AI Model Documentation
Executive Summary
Key Takeaways
- ✓ AI model documentation requires elements beyond standard financial model documentation, training data or source material description, technique category, known limitations, and validation history over time.
- ✓ Describing the training data or source material a model was built on supports assessing whether it remains representative of current conditions, directly relevant to the model drift risk addressed in AI Model Governance.
- ✓ Stating the specific technique category, machine learning, NLP, or generative AI, applied supports applying the correct explainability standard and reliability expectations addressed in AI Explainability.
- ✓ Documenting known limitations explicitly, rather than only the model's intended capabilities, supports a downstream user correctly judging when the model's output should be weighted more or less heavily.
- ✓ Maintaining validation history over time, not only the most recent validation result, supports identifying gradual accuracy trends that a single point-in-time validation record would not reveal.
Objective¶
This guide sets out documentation requirements specific to AI models used in finance, complementing standard financial model documentation practice within AI Financial Modelling & Artificial Intelligence in Finance.
Documentation Elements Specific to AI Models¶
Training data or source material description. What data or source material the model was built on, or in the case of a generative AI application, what source material it is grounded to draw from. This supports assessing whether the model remains representative of current conditions, directly connected to the drift risk addressed in AI Model Governance.
Technique category. Whether the model applies machine learning, natural language processing, or generative AI, addressed generally in Artificial Intelligence in Finance. This supports applying the correct explainability standard set out in AI Explainability, since the three categories carry genuinely different explainability characteristics.
Known limitations. What the model does not capture, or where it is known to be less reliable, documented explicitly rather than left implicit. This supports a downstream user, a decision-maker following the framework in AI Decision Support, correctly judging how much weight to give the model's output.
Validation history. A maintained record of validation results over successive periods, not only the most recent result, supporting identification of a gradual accuracy trend that a single point-in-time record would not reveal, feeding into the periodic re-validation schedule addressed in AI Model Governance.
Why Each Element Supports a Specific Downstream Use¶
Each documentation element exists to answer a question a specific downstream user will need answered: a governance reviewer assessing drift risk needs the training data description; an auditor applying the correct explainability standard needs the technique category; a decision-maker weighing the model's output needs the known limitations; and a governance committee reviewing model health over time needs the validation history. Documentation that omits any of these elements leaves that specific downstream question unanswerable without reconstructing the information after the fact.
Common Construction Pitfalls¶
Documenting only the model's intended use, not its known limitations. A documentation record describing only what a model is designed to do, without what it does not capture well, gives a downstream user an incomplete basis for weighing its output.
Retaining only the most recent validation result. Without a maintained history, a gradual accuracy decline across several validation cycles can go unnoticed even though each individual result might appear acceptable in isolation.
Applying financial model documentation templates unmodified to AI models. A template built for static formula-based models will not prompt for training data, technique category, or the AI-specific limitations this guide addresses.
Recommended Practices¶
- Document training data or source material, technique category, known limitations, and validation history for every AI model used in finance.
- Maintain validation history as a running record, not only the latest result.
- Extend, rather than replace, existing financial model documentation templates to capture these AI-specific elements.
Continue Reading¶
Related Pillars¶
Related Technical Guides¶
How OXXON tests thisRun a free structural check with FMAE
Frequently Asked Questions
What does AI model documentation need beyond standard financial model documentation?
A description of the training data or source material the model was built on, the specific AI technique category applied, its known limitations, and its validation history over time, each supporting a specific downstream use a standard financial model's documentation does not need to address.
Why document the training data or source material specifically?
Because it supports assessing whether the model remains representative of current conditions, directly relevant to the model drift risk addressed in AI Model Governance, a question standard financial model documentation does not need to answer since a formula's logic does not depend on training data.
Why document the specific technique category applied?
Because it supports applying the correct explainability standard and reliability expectations addressed in AI Explainability, since machine learning, NLP, and generative AI each carry different explainability characteristics that a downstream reviewer needs to know which standard to apply.
Why document known limitations, not just intended capabilities?
Because a downstream user needs to know where a model's output should be weighted more or less heavily, information a description of only the model's intended capabilities and successes does not provide.
Why maintain validation history over time rather than only the latest result?
Because a single point-in-time validation record cannot reveal a gradual accuracy trend, while a maintained history of validation results over successive periods can surface drift before it becomes a material reliability problem.
Related Articles
AI Financial Modelling & Artificial Intelligence in Finance
AI financial modelling is the application of machine learning and generative AI techniques within the financial modelling process itself, driver identification, construction assistance, scenario generation, and narrative drafting, while artificial intelligence in finance is the broader application of those same technique categories across the finance function generally. This page is the hub for the Knowledge Centre's AI financial modelling content: the foundational distinction between machine learning, natural language processing, and generative AI; how AI accelerates modelling construction without replacing the auditable calculation layer beneath it; a staged framework for adopting AI reliably; enterprise applications across FP&A, forecasting, valuation, and investment analysis; governance and risk practice; and the institutional best practice synthesis this domain builds toward.
AI Model Governance
AI model governance establishes ownership, documented scope and limitations, change control, and periodic re-validation for machine learning and generative AI models used within a finance function. This guide sets out the governance elements specific to AI models, distinct from but complementary to the financial model governance a firm already applies to its spreadsheet and system models, and why an AI model's statistical nature requires governance triggers a static formula-based model does not.
AI Explainability
Explainability, the ability to state why an AI model produced a specific output, means something different for machine learning than for generative AI, and something different again from the formula traceability standard applied to a deterministic financial model. This guide sets out each of these distinct explainability standards, why conflating them creates unrealistic expectations for what an AI model can actually explain about itself, and the practical documentation, feature importance, training data description, known limitations, that supports explainability in finance practice.
AI Audit Trail
An audit trail for AI-assisted financial work should capture more than the final output: the prompt or task input, the specific model or technique version used, the source material supplied, the verification checkpoint outcome, and the human decision applied to the result. This guide sets out what a complete AI audit trail captures and why each element matters specifically for defending an AI-assisted conclusion after the fact, to an auditor, regulator, or internal governance review.
AI Assurance Framework
Ongoing assurance over AI-assisted finance work depends on connecting three activities that are often run separately, quality assurance sampling, internal control testing, and periodic model re-validation, into a single assurance cycle with a shared reporting line. This guide sets out how these three activities complement each other, why running them in isolation leaves gaps each is well positioned to catch for the others, and how to structure a combined assurance cycle.