RP-005: Evidence-Driven Rule Calibration — The R015 Case Study

Institutional publication. Not peer-reviewed.

Abstract¶

This paper documents a single, dated calibration event in the FMAE rule engine's history — the redesign of rule R015, Explicit Aggregation Review, from an earlier and more aggressive form — as a case study in evidence-driven rule design for deterministic structural audit engines. The event is grounded entirely in the engine's own changelog and current source code, verifiable independently of this paper. The paper deliberately withholds the specific client engagement and exact finding counts that motivated the review, reporting instead the rule's own objectively verifiable before-and-after parameters.

1. Background¶

FMAE's rule engine, documented in full on the Rule Engine & Rule Packs page, executes 26 deterministic structural rules against every audited workbook. Each rule's severity, weight, and detection threshold are declared, static values — not learned or adjusted per audit. This raises a natural question for any such engine: if a rule's thresholds are fixed, how are they ever corrected when evidence suggests they do not track genuine risk well? R015's history is FMAE's own documented answer.

2. The Original Rule¶

R015, then named "Addition Chain Risk," fired on any formula joining five or more individual cell references with a bare addition operator (+), rather than a SUM()-style aggregation function, regardless of whether the referenced rows formed a contiguous range. It carried high severity.

3. The Review Finding¶

A review of this rule's findings, conducted against a real institutional financial model, found the overwhelming majority of triggered findings to be legitimate modelling practice rather than genuine structural risk, falling into two distinct patterns:

• Non-contiguous addition chains — a formula deliberately summing a specific, named, non-adjacent set of components (for example, an EBITDA bridge that intentionally skips certain line items). A SUM()-style formula over the enclosing range would have been incorrect in these cases, since it would have included rows the model's author specifically chose to exclude.
• Contiguous addition chains — a formula representing a fixed, named list of business components (a revenue total, an operating cash flow subtotal) where explicit addition is standard, legible modelling practice, preferred by experienced practitioners specifically because a reader can see exactly which components are included without needing to trust an opaque range.

4. The Redesign¶

Following the review, the rule (renamed to Explicit Aggregation Review) was changed on four dimensions, each traceable to a specific finding from the review rather than an isolated tuning decision:

The remediation guidance was also rewritten: rather than unconditionally recommending replacement with SUM(), the rule now asks a conditional question — retain the explicit form when components are fixed and named, consider SUM() only when the range is genuinely expected to grow.

5. Why This Case Generalizes¶

The specific mechanism that improved this rule's precision was not a blanket loosening of its trigger condition — it was a structural distinction the original rule did not draw: whether the referenced rows are contiguous. A contiguous addition chain is a candidate for the risk this rule addresses (a later-inserted row silently excluded from the total); a non-contiguous chain is, by construction, evidence of deliberate component selection, since a contiguous SUM() range could not have produced that reference pattern by accident. This is a generalizable principle for any structural rule comparing an observed formula pattern against an expected aggregation shape: the presence or absence of a specific structural signal can turn an otherwise-noisy heuristic into a precise one, without abandoning the underlying concern the rule exists to address.

6. Scope and Limitation of This Paper¶

This paper reports only the rule's own objectively verifiable parameters, each independently checkable against gcc_audit/rules/r015_manual_addition_chain.py and CHANGELOG.md in the FMAE source repository at the time of writing. It deliberately does not reproduce the specific client engagement, exact finding counts, or any other client-specific detail behind the review that motivated this redesign — those facts belong to a specific client relationship, not to this institutional record of the engine's own calibration history.

Related Reading¶

• R015 — Explicit Aggregation Review — the current rule this case study documents the calibration history of.
• FMAE Rule Taxonomy — where R015 sits within the engine's six-category classification.