RP-006: Model Validation vs. Model Audit — A Methodological Comparison

Institutional publication. Not peer-reviewed.

Abstract¶

This paper compares model validation and model audit as distinct methodological disciplines within financial model risk management, treating them as answering categorically different questions rather than representing two intensities of the same underlying exercise.

1. The Two Questions¶

Model audit asks: does this model, as built, calculate what it claims to calculate? This is a question about mechanical correctness — whether the formulas, references, and structural logic actually implement the calculation the model's labels and documentation claim they implement, independent of whether that calculation is the right one to be running.

Model validation asks: is the modelling approach and are the assumptions chosen appropriate for this model's specific intended purpose? This is a question about fitness for purpose — whether the methodology itself, correctly implemented, is the right methodology given what the model is being used to decide.

2. Why Audit Admits a Deterministic Methodology and Validation Does Not¶

Mechanical correctness is a structural property: a formula either does or does not implement a stated calculation correctly, and this can be checked systematically against a fixed methodology applied consistently to every formula in the model. This is precisely what makes audit amenable to a deterministic, rule-based engine — the kind FMAE's own Rule Engine & Rule Packs implements — since the check does not require judgement about the model's purpose, only about whether its internal logic is self-consistent and free of the specific structural defects a rule targets.

Appropriateness, by contrast, is inherently contextual. The same modelling choice — a simplified discount rate assumption, a particular treatment of working capital — can be entirely appropriate for one decision (an early-stage feasibility screen) and inappropriate for another (a binding financial close). Validation therefore inherently involves judgement in a way audit, as defined here, does not.

3. Independence of Outcomes¶

Because the two disciplines test different properties, their outcomes are independent rather than correlated in any necessary direction:

• A model can pass audit and fail validation: every formula calculates exactly as its author intended, but the underlying modelling approach — perhaps an oversimplified treatment of a material risk — is not appropriate for the decision the model supports.
• A model can fail audit and pass validation: the chosen methodology is sound and well-suited to the model's purpose, but a formula error (a hardcoded value, a broken reference, an inconsistent row) undermines the model's actual calculated output regardless of how sound the underlying approach is.

This independence is the methodological basis for treating audit and validation as complementary rather than substitutable exercises — each addresses a risk the other does not test for at all.

4. Practical Implication¶

For a model supporting a genuinely material decision, relying on only one of the two disciplines leaves the other risk category completely untested, not merely less thoroughly tested. This is a direct extension of the buyer-facing comparison already published on the Knowledge Centre's Audit vs Validation page, restated here in methodological rather than commercial register.

Related Reading¶

• Audit vs Validation — the Knowledge Centre's buyer-facing treatment of this same distinction.
• FMAE Rule Engine & Rule Packs — the execution model that makes deterministic audit, specifically, possible.