Model Validation
Executive Summary
Key Takeaways
- ✓ Model validation is the independent assessment of a model's conceptual soundness, assumption basis, and implementation accuracy.
- ✓ It must be conducted by a reviewer independent of the model's developer.
- ✓ It covers conceptual soundness, data and assumptions, and implementation accuracy.
- ✓ Validation frequency is determined by the model's risk tier.
- ✓ Material findings must be resolved; validation that identifies errors and does not ensure their correction provides limited assurance.
- ✓ The audit of a model governance framework should confirm validation independence, coverage, scope adequacy, findings resolution, and documentation.
Definition¶
Model validation is the structured, independent process of assessing whether a financial model is conceptually sound, mathematically correct, implemented as intended, and fit for its approved purpose. It is conducted by a reviewer who is independent of the model's developer and produces a documented assessment of the model's strengths, limitations, and any findings requiring remediation.
Model validation is a component of model governance. The governance framework defines when validation is required, who conducts it, and what the validation must assess. The validation itself is the technical execution of that requirement.
Why It Matters¶
Model validation exists because the developer of a financial model is not the appropriate reviewer of that same model. The developer has contextual knowledge that allows them to overlook errors they made (confirmation bias), and they have an interest in the model producing expected or desired outputs. Independent validation removes both of these limitations.
Without validation, an organisation relies on model outputs that have been reviewed only by the person who created them. Given the complexity of modern financial models and the materiality of the decisions they inform, this is an inadequate control.
Model validation is required by:
- Lenders as a condition precedent to financial close in project finance transactions (see Model Audit Certificate)
- Model governance frameworks for models above a defined risk tier
- Investment committees as a standard element of due diligence
- Regulatory frameworks for financial institutions that rely on internal models for risk measurement or reporting
Note: Specific regulatory requirements for model validation vary by jurisdiction and institution type.
Technical Background¶
The Three Components of Model Validation¶
A comprehensive model validation addresses three components:
1. Conceptual soundness Assessment of whether the theoretical framework underlying the model is appropriate for its intended purpose. For a project finance model, this includes whether the DSCR definition is correct, whether the debt sculpting logic correctly reflects the intended structure, and whether the financial statements are constructed on the appropriate accounting basis.
2. Data and assumptions Assessment of whether the input data and assumptions used in the model are appropriate, sourced from reliable references, and consistently applied. This does not constitute independent market research — the validator is assessing the logic and internal consistency of the assumptions, not independently forecasting markets.
3. Implementation accuracy Assessment of whether the model correctly implements its intended logic. This is the most technically intensive component and includes:
- Formula verification: Are the formulas calculating what they are supposed to calculate?
- Reference accuracy: Are cell references pointing to the correct source cells?
- Circularity check: Does the model contain circular references, and if so, are they intentional and correctly resolved?
- Consistency check: Are formulas applied consistently across like rows?
- Hardcoded number check: Are there hardcoded numbers in formula sections that should be formula-driven?
- Output verification: Do the model's key outputs (DSCR, IRR, NPV) reconcile to an independent recalculation?
Validation vs Audit: Terminology¶
In financial model practice, the terms "validation" and "audit" are sometimes used interchangeably and sometimes used to describe distinct activities:
| Term | Common Usage |
|---|---|
| Model validation | Internal process within an organisation's governance framework; conducted by an internal model risk function or an external firm acting in a validation capacity |
| Model audit | External independent review, typically commissioned for a transaction; produces a model audit certificate as a condition precedent to financial close |
| Model review | Generic term for any structured assessment of a model; may or may not produce a certificate |
The substantive content of a validation, audit, or review may be similar or identical. The key distinctions are independence (internal vs external) and the formal output (governance record vs certificate).
Validation Scope¶
A validation scope document defines what the validator will and will not assess. A standard scope for a financial model validation includes:
In scope: - Arithmetic and formula verification - Internal consistency of the three financial statements (where applicable) - Reconciliation of key outputs against an independent recalculation - Review of the model's documentation for completeness and accuracy - Assessment of whether the model is being used within its approved purpose
Out of scope (unless specifically included): - Commercial reasonableness of assumptions (the validator is not providing market or technical advice) - Legal advice on the interpretation of contractual provisions - Tax advice - Independent verification of third-party forecasts or technical reports
The scope boundary matters because parties relying on the validation should understand what it does and does not assure.
Who Can Conduct a Model Validation¶
Independence is the defining characteristic of a valid reviewer. A model validation conducted by the model's developer, by a member of the developer's team, or by the developer's manager does not constitute independent validation. The validator must be structurally independent — in a different reporting line from the developer, or from a separate organisation.
For formal governance frameworks, the model risk or model validation function is typically structured to be independent of the business lines that develop and use models.
Validation Frequency¶
The frequency of validation is determined by the model's tier classification and the requirements of the model governance framework:
| Tier | Typical Validation Frequency |
|---|---|
| Tier 1 (High risk) | Full validation at initial development; re-validation after any material change; periodic full review (e.g. annually) |
| Tier 2 (Medium risk) | Validation at initial development; re-validation after material change; periodic review (e.g. every 2–3 years) |
| Tier 3 (Low risk) | Validation at initial development or on a risk-based sample basis; no mandatory periodic review |
Material change triggers are: significant changes to model logic, changes to the purpose or use case of the model, or material changes to the scope of assumptions.
Outcome Testing¶
For models that have been in use over time, validation should include outcome testing (also called backtesting): comparing the model's historical predictions against actual outcomes to assess whether the model performs as expected. Where significant divergences exist between predictions and outcomes, the model's assumptions and structure should be reviewed.
Audit Considerations¶
When auditing an organisation's model validation framework, the following should be examined:
1. Independence of Validators¶
Confirm that validators are structurally independent of model developers. A validation function that sits within the same business unit as the model development team is not independent.
2. Validation Coverage¶
Confirm what proportion of the model population has been validated, and whether the validated models include all Tier 1 and Tier 2 models. A governance framework that validates only a subset of high-risk models is not providing full assurance.
3. Scope Adequacy¶
For a sample of completed validations, confirm that the scope was adequate for the model's risk tier. A validation that checked only formatting but did not verify key outputs is not an adequate validation of a Tier 1 model.
4. Findings Resolution¶
Confirm that material findings from completed validations have been resolved. A validation that identifies critical errors that are never corrected provides no protection.
5. Validation Documentation¶
Confirm that validation reports are documented, retained, and accessible. Validation that is conducted informally and leaves no written record cannot be relied upon as a governance control.
Common Errors¶
| Error | Description | Risk |
|---|---|---|
| Developer self-validation | Model reviewed by its developer | No independent assurance |
| Inadequate scope | Validation covers formatting only, not formula verification | Critical errors remain undetected |
| No outcome testing | Historical predictions not compared to actuals | Model performance deterioration undetected |
| Findings not resolved | Material findings documented but not remediated | Known errors remain in operational models |
| No re-validation trigger | Model materially changed without re-validation | Validation covers a different version than the one in use |
Best Practices¶
Define validation independence standards explicitly in the model governance policy. State clearly that a validator cannot validate a model they developed or that was developed by their direct reports.
Require that validation scope is agreed in writing before the review begins, and that the scope document is retained with the validation report. This prevents scope disputes after the review is completed.
Track validation status for every model in the model inventory, including the date of last validation, the tier of the model, and the date by which re-validation is due. Use this information to drive a validation schedule that is proactive rather than reactive.
Continue Reading¶
Prerequisites¶
- What Is Model Risk? — the parent pillar
Related Pillars¶
Related Technical Guides¶
Related Glossary¶
How OXXON tests thisRun a free structural check with FMAE
Frequently Asked Questions
What is the difference between model validation and model audit?
Model validation typically refers to the internal governance process of reviewing a model within an organisation's risk framework. Model audit typically refers to an external independent review producing a certificate for a transaction. The substantive technical content can be identical. The key differences are independence structure and formal output.
Can a model be used before it has been validated?
Under a model governance framework, a model should not be used for its intended purpose until it has been validated, unless an explicit exception has been approved by the relevant governance authority with documented compensating controls. Using an unvalidated model for a material decision is a governance breach.
How long does model validation take?
This depends entirely on the model's complexity, the scope of the validation, and the quality of the model's documentation. A simple Tier 3 model may be validatable in a few hours. A complex Tier 1 project finance model may require several days of detailed review.
What happens if a model fails validation?
There is no formal pass/fail in most model validation frameworks. Instead, the validation produces findings classified by severity. Material findings must be resolved before the model is used for its intended purpose. The governance framework defines what "resolved" means — typically that the developer has corrected the finding and the validator has confirmed the correction.
Related Articles
Model Governance
Model governance is the organisational framework through which an institution defines, implements, and enforces policies and controls for the development, approval, use, validation, change, and retirement of financial models. It establishes accountability for model quality, a structured process for model oversight, and a documented record of model use and validation history. Effective model governance ensures that decisions made using financial models are based on outputs that have been developed to an appropriate standard, validated by a party independent of the developer, and used within the bounds for which they were designed.
What Is Model Risk?
Model risk is the risk that a decision is wrong not because the underlying business or investment case was flawed, but because the model used to evaluate it was. It is a distinct category of risk from market risk, credit risk, or operational risk, and it applies to any organisation that relies on a financial model, spreadsheet or otherwise, to support a material decision. Most published model risk content addresses statistical and regulatory capital models used inside banks. This page defines model risk specifically as it applies to Excel based financial models, the kind used every day for investment decisions, lending, and transaction evaluation, which is a related but distinct problem from the quantitative model risk literature most search results return.
Model Tiering
Model tiering is the process of classifying financial models into risk-based categories — tiers — that determine the level of governance oversight, validation rigour, documentation standards, and review frequency applied to each model. Higher-tier models, which are more complex, more material to decision-making, or more difficult to verify, receive more intensive governance than lower-tier models. Model tiering allows organisations to apply governance resources proportionately. Without tiering, an organisation must either apply heavy governance to every model (impractical) or apply light governance to every model (insufficient for high-risk models). Tiering resolves this by concentrating oversight where it matters most.
Model Materiality
Model materiality is the threshold at which an error, deviation, limitation, or uncertainty in a financial model is considered significant enough to affect a decision, require remediation, or warrant disclosure. A finding is material if, had it been known, it would or could have changed a decision made using the model's outputs. Model materiality is a judgement — it depends on the purpose of the model, the magnitude of the finding, and the sensitivity of the key outputs to the finding. The same error may be material in one context and immaterial in another.
Audit Methodologies for Financial Models
Financial model audit methodologies fall into three primary categories: manual line-by-line review, automated structural analysis, and deterministic rule-based checking. Each methodology differs in scope, speed, consistency, and the types of errors it is designed to detect. The appropriate methodology depends on transaction complexity, time constraints, and institutional risk appetite.
Model Audit Certificate
A model audit certificate (also referred to as a model audit report or model assurance certificate) is a formal written document issued by an independent auditor or model review firm confirming that a financial model has been independently reviewed, describing the scope of the review, identifying findings, and providing a level of assurance about the model's arithmetical accuracy and internal consistency. In project finance, a model audit certificate is typically a condition precedent (CP) to financial close, meaning that lenders will not fund the first drawdown until the certificate has been delivered by an approved independent reviewer.
What Is a Financial Model Audit?
A financial model audit is an independent, structured examination of an Excel based financial model to confirm that its mechanics, logic, and outputs are reliable enough to support a decision. It is not a check of whether the assumptions are optimistic or conservative. It is a check of whether the model actually calculates what its author believes it calculates. Every year, lenders extend debt, investment committees approve capital, and boards sign off on transactions using numbers that came out of a spreadsheet nobody outside the immediate deal team has independently verified. A financial model audit exists to close that gap before it becomes expensive.