FMAE Scoring Engine — SM-2.0 Methodology
Executive Summary
Key Takeaways
- ✓ The active scoring formula (SM-2.0) is: score = 100 − (triggered rule weight ÷ 207.0 × 100), where 207.0 is the sum of every active rule's declared weight.
- ✓ SM-1.0, the engine's original methodology (basis 170.0, covering R001–R021), is retained as a frozen historical reference, not deleted, when SM-2.0 was introduced to cover R022–R026.
- ✓ Five rules — R001, R002, R004, R006, R023 — are critical-override rules. If any one of them triggers, the model's letter grade is capped below A or B regardless of the numeric score.
- ✓ A model can score in the high 70s or 80s and still be graded C, not B, if the triggered findings include a critical-override rule — the override exists precisely so a numeric score alone cannot certify a structurally broken model as sound.
The SM-2.0 Formula¶
The active FMAE scoring methodology, SM-2.0, computes a model's overall risk score as:
score = 100 − (triggered rule weight ÷ 207.0 × 100)
Where triggered rule weight is the sum of the declared weight value of every rule that produced at
least one finding in the audit, and 207.0 is the fixed basis — the sum of the declared weights of every
rule in the active rule set (R001 through R026). The score is bounded to the range 0–100.
Every rule's weight is a fixed value declared once, in the rule's own source file — it is not computed at audit time and does not vary by workbook. The basis (207.0) is likewise a fixed constant, re-derived and pinned whenever a new rule is added to the active set, and guarded by a test that the sum of every active rule's weight equals the pinned value — so the basis cannot silently drift out of step with the rule set it is meant to represent.
SM-1.0 → SM-2.0: What Changed and What Didn't¶
FMAE's scoring methodology has one version history, published here rather than left undocumented:
| SM-1.0 (frozen historical reference) | SM-2.0 (active) | |
|---|---|---|
| Status | Frozen — retained for audit-trail comparability, not used for new audits | Active — used for every current audit |
| Rule coverage | R001–R021 | R001–R026 |
| Basis | 170.0 | 207.0 |
| Critical-override rules | R001, R002, R004, R006 | R001, R002, R004, R006, R023 |
SM-1.0 was not deleted when SM-2.0 was introduced. It is retained in the engine as a frozen historical reference specifically so a score computed under SM-1.0 against an older audit remains interpretable on its own terms, rather than silently reinterpreted under a methodology it was never scored against.
SM-2.0 exists because five new rules (R022–R026) were added to the active rule set after SM-1.0 was established. Extending the existing basis rather than replacing the formula preserves the same scoring logic — the formula itself, 100 minus normalized triggered weight, has not changed between versions; only the rule coverage and the basis constant have grown to match it.
The Five Critical-Override Rules¶
Five rules, if triggered, cap a model's letter grade regardless of how high its numeric score is: R001 (Hardcoded Cells), R002 (Broken Links), R004 (Formula Inconsistency), R006 (Propagated Errors), and R023 (Aggregation Range Gap). Full detail on each is published on its own page in the Rule Reference.
A critical override means: if any one of these five rules triggers at least one finding, the model's letter grade is capped so it cannot be graded A or B, no matter what the weighted numeric score comes out to. The numeric score itself is unaffected by the override — only the letter grade is capped. This distinction exists because these five findings each represent a structural failure serious enough that no combination of otherwise-clean formulas should be able to average it out into a passing grade.
Grade Thresholds¶
The numeric score maps to a letter grade on a fixed threshold scale:
| Score | Grade |
|---|---|
| ≥ 90 | A |
| ≥ 75 | B |
| ≥ 60 | C |
| ≥ 40 | D |
| < 40 | F |
If a critical-override rule has triggered and the threshold table above would otherwise assign an A or a B, the grade is capped at C.
Worked Example¶
A model triggers four findings, from four different rules:
| Rule | Finding | Weight |
|---|---|---|
| R001 | Hardcoded value overriding a formula | 15.0 |
| R004 | Formula inconsistency across a row | 15.0 |
| R013 | IFERROR masking a broken reference | 8.0 |
| R017 | External link count above threshold | 6.5 |
| Total triggered weight | 44.5 |
score = 100 − (44.5 ÷ 207.0 × 100) = 78.5
Read against the threshold table alone, 78.5 falls at or above the B threshold (≥ 75). But R001 and R004 are both critical-override rules. Because at least one triggered, the letter grade is capped below A or B — the model is graded C, not B, even though its numeric score of 78.5 would read as a solid B in isolation.
Every weight and the basis figure in this example are read directly from gcc_audit/scoring/risk_score.py
and will be re-derived here if either changes — this page carries a live dependency on the engine's actual
rule weights, not a static illustrative figure.
Related Reading¶
- FMAE Rule Taxonomy — the classification each rule (including the five critical-override rules) belongs to.
- Rule Reference — individual pages for R001, R002, R004, R006, and R023, and every other rule this formula scores.
- FMAE Audit Engine — Architecture Overview — where scoring sits in the overall audit pipeline.
- Model Risk — the buyer-facing explanation of model risk this methodology quantifies.
- RP-008: Materiality and Severity — Distinguishing Two Axes of Model Risk — the fuller, engine-independent framework the critical-override mechanism on this page is formalized into.
How OXXON tests thisRun a free structural check with FMAE
Frequently Asked Questions
How is a financial model risk score calculated?
Every triggered rule's declared weight is summed, then the score is 100 minus that sum divided by a fixed basis (207.0 under the active SM-2.0 methodology) times 100. If any critical-override rule triggered, the resulting letter grade is capped below A or B.
What is SM-2.0?
SM-2.0 is the active scoring methodology, covering all 26 rules (R001–R026) against a basis of 207.0. It succeeds SM-1.0, which covered R001–R021 against a basis of 170.0 and is retained as a frozen historical reference rather than deleted.
What is a critical-override rule?
One of five rules (R001, R002, R004, R006, R023) whose triggering caps a model's letter grade below A or B, regardless of the numeric score the weighted formula produces. The override exists so a model with a structurally serious finding cannot be graded as though it were merely good with minor issues.
Can a model score 90 and still not get an A?
Yes, if a critical-override rule triggered. The numeric score and the letter grade are computed from the same weighted formula, but the grade is capped afterward if any of the five critical-override rules is among the triggered findings.
Why does a rule's weight matter more than its severity label alone?
Severity (critical, high, medium, low) is a category; weight is the specific numeric value that the score formula actually sums. Two critical-severity rules do not necessarily carry the same weight — R001 is weighted at 15.0 and R023 at 10.0, for example — so a model's numeric score depends on which specific rules triggered, not just how many critical-severity findings it has.
Does the scoring basis ever change?
Yes, when a new rule is added to the active rule set, its weight is added to the basis, and the basis is re-pinned to the new total. The engine guards this with a test that recomputes the sum of all active rule weights and fails if it does not equal the pinned basis, so the basis cannot silently drift out of sync with the rule set.
Related Articles
FMAE Audit Engine — Architecture Overview
The FMAE audit engine processes a workbook through five stages executed in a fixed order — parsing and workbook ingestion, dependency graph construction, rule engine execution, risk scoring, and report generation. Each stage consumes the previous stage's output and exposes nothing about how any later stage will use it, so the stages can be documented, tested, and read about independently. This page is the map; the detail behind each stage lives on its own Technical Documentation page as those pages are published.
FMAE Rule Taxonomy
Every rule in the FMAE structural rule pack declares a category attribute at the point it is defined in source — this is not a classification imposed on the rules afterward for documentation purposes, it is the classification the engine itself uses. Six categories cover all 26 rules — Structural (18 rules), Assumptions Governance (1), Integrity Controls (2), Structural Hygiene (1), Aggregation Logic (1), and Model Governance (3). This page publishes that taxonomy as the FMAE equivalent of a control catalog's classification scheme, cross-linked to the Rule Reference page for each member rule.
Rule Reference Index
The Rule Reference is a set of individually addressable pages, one per FMAE structural rule, living at /fmae-knowledge-centre/rule-reference/ — the FMAE equivalent of an AWS error-code reference or a NIST control catalog. Each rule page states its purpose, detection logic at a conceptual level, severity, evidence produced, remediation guidance, and its sibling rules within the same taxonomy category. All 26 rule pages are now published; this index page explains how the Rule Reference is organized and links to every rule page in taxonomy order.