TOOL 02 · BIAS DETECTION

Fairness Monitoring Protocol

Check a live credit model's fairness, see what each result means, and who must act.

Reviewer guide · Tool 02

What it is

A protocol for watching a credit model's fairness once it is live. It sets what is measured and how often, the events that force a check, and what happens when a check returns Green, Amber, Red or Not evaluable.

Why it matters

A model that was fair at approval can drift as applicants, products and data change. Many institutions have no defined thresholds, owners or deadlines once a model is in production.

How to read it

Part 1 sets frequency by the model's materiality tier and lists three core measures (parity, equalized odds, AUC) and six early-warning signals, including a scan of features that reflect an applicant's history with the lender and a check on whether staff overrides favor men or women. Part 2 lists events that force an immediate check. Part 3 sets the action, owner and evidence for each status. Part 4 shows where each threshold comes from.

Reading EOD

For equalized odds difference (EOD), the proposed starting values are: Green when the upper bound is 0.02 or below; Amber when the estimate is above 0.02 and up to 0.05; Red above 0.05. Amber means the model owner investigates within one cycle. Red, or Amber left unresolved, goes to the model risk committee, which decides whether to continue, restrict, remediate or suspend the model.

Questions this tool informs

Item	What to look at
Q8–Q9	Whether production fairness monitoring with thresholds and owners exists where you work, and whether its absence is a gap
Q10	Whether computing the measures at the tier-based frequency in Part 1 is operationally feasible
Q11	Whether Green / Amber / Red, with named owners and deadlines, fits your existing model risk escalation
Q12	The EOD starting values: Amber above 0.02, Red above 0.05
Q13	Whether the protocol matches the ongoing-monitoring expectations of SR 26-2, SR 11-7 and/or OSFI E-23
Q14	Barriers to operating the protocol, and the frequency or thresholds you would set instead

Nothing you enter here is stored or sent. Values stay in this browser tab and are cleared when you close it. Use Export to keep a copy.

How to complete this step

Choose the model’s purpose and exposure using institution-defined volume, dollar and de minimis cut-offs. Read the resulting materiality tier and its monitoring schedule.
Record cut-offs and the repayment seasoning window in Tool 04 before scoring. Use the appropriate cohort for each scheduled measure; immature outcomes cannot be treated as a passing result.

Materiality tier

PurposeOutput determines approval, denial or price without individual human review (or overrides are below the de minimis level)Output is one input to a decision with documented human review and tracked overridesShadow mode, challenger, analytics or pilot — no adverse action issued

ExposurePrincipal consumer credit channel, or above the institution's volume and dollar thresholdsBelow those thresholds, above the de minimis floor

Monitoring frequency
Measure	Tier 1	Tier 2	Tier 3
DPR	Monthly cohort	Quarterly	At revalidation or material change
EOD	Monthly cohort	Quarterly	At revalidation
AUC	Quarterly on matured cohort (PSI is the interim signal)	Semi-annually	At revalidation
Cell gate	Every run	Every run	Every run
PSI	Monthly	Quarterly	Annually
Attribution drift	Quarterly	Semi-annually	At revalidation
Proxy scan	Quarterly	Semi-annually	At revalidation
Reason stability	Monthly	Quarterly	Not applicable
Override parity	Monthly	Quarterly	Not applicable

Record your cut-offs, dated and version-controlled, in the Tool 04 pre-registration. Practitioner review may make monitoring lighter for smaller institutions; thresholds change only through a new Tool 04 pre-registration.

Reading the results

Relationship features: File each feature and its correlation with sex to Tool 04 every cycle, including features that do not cross a band.

Overrides: Review changes from denial to approval by sex. Opposite-direction overrides are retained separately and do not replace the parity calculation.

Pricing: The same measures can be applied to price outcomes; this use has not been tested in the research behind the book.

Scope: SR 26-2 excludes generative and agentic AI models from its scope. Institutions should nevertheless determine appropriate governance and controls for tools, processes or systems it does not cover. This protocol is a practitioner monitoring aid, not supervisory guidance or a claim that SR 26-2 applies to those models. SR 26-2 scope

Key terms

DPR: Female approval rate divided by male approval rate; the lower confidence bound informs the monitoring band.
EOD: Equalized odds difference: the larger absolute true-positive or false-positive rate gap between sexes, with uncertainty bounds.
AUC: Predictive discrimination, measured against the locked validation baseline on a matured cohort.
PSI: Population stability index compares expected and current distributions across bins.
Attribution drift: Change in the rank order of model explanations, measured using Spearman correlation and new top-three features.
Relationship feature: A feature tied to the applicant’s relationship with the institution; record its correlation with sex each cycle.
Override parity: Female-to-male ratio of model denials changed to approvals by human review.
Seasoning: Waiting for repayment outcomes to mature before computing outcome-dependent measures.

Thresholds are declared starting points fixed in the Tool 04 pre-registration. They are not regulatory requirements.