TOOL 04 · GOVERNANCE

Pre-Registration Template & Audit-Trail Checklist

Fix how fairness will be judged before training, name who decides when a threshold is crossed, and keep the evidence.

Reviewer guide · Tool 04

What it is

The governance record for a model. Part 1, the pre-registration template, is signed before training and fixes how fairness will be judged. Part 2 names who must decide when a threshold is crossed. Part 3 lists the evidence kept across the model's life, and Part 4 logs every change after sign-off.

Why it matters

When fairness measures and thresholds are chosen after results are known, a team can, knowingly or not, report the most favorable one. Committing in advance removes that choice, and the checklist lets a validator, auditor or examiner rebuild what was decided, when and by whom.

How to read it

Part 1 is grouped into scope, data, measures and thresholds, methods, and governance. It now also records the rules for missing data, the features reflecting an applicant's history with the lender, where each threshold comes from, and who owns each decision. A worked example shows the registration used for the research behind the book.

Questions this tool informs

Item	What to look at
Q22–Q23	Whether measures are usually chosen after results at institutions you know, and whether that invites selective reporting
Q24	Whether completing and signing Part 1 before training fits your development timelines
Q25	Whether your institution keeps, or could keep, each record in Part 3
Q26	Whether Tool 04 would strengthen your documentation position under SR 26-2, SR 11-7 and/or OSFI E-23
Q27	Whether the “Maps to” column links records to the right lifecycle stages and expectations
Q28	Barriers to adopting pre-registration

Nothing you enter here is stored or sent. Values stay in this browser tab and are cleared when you close it. Use Export to keep a copy.

A · Scope

How to complete this step

Before training, identify the model/use case, origin, protected groups and materiality tier. Define institution-specific tier cut-offs and outcome seasoning; research evaluation must be identified explicitly.

1 Model materiality tierTier 1 — HighTier 2 — ModerateTier 3 — LowNot applicable — research evaluation

Tier cut-offs / applicability

2 Model and use case

Describe the model, credit product, population and intended decision.

Model originBuilt in-houseBought from a vendor

3 Protected groups examined

State the attributes as recorded and any excluded or future groups.

4 Outcome seasoning window

Record when repayment outcomes mature, or explain retrospective applicability.

Tool 02 tier helper

B · Data

How to complete this step

Fix exclusion and missing-label rules, the attrition limit and every scoring segment’s minimum counts. List relationship features and definitions so later scans cover them consistently.

5 Data rules

Define decision outcomes, sex exclusions (never imputed), bounding check and attrition limit.

Attrition limit (percentage points)

6 Segments and minimum counts

List every segment and the minimum female approvals AND female denials.

Segment 1 name

Segment 1 minimum

7 Relationship features

List feature names and definitions, or explicitly state none/not applicable.

Relationship feature 1

Relationship definition 1

C · Measures and thresholds

How to complete this step

Select the fairness measures and set thresholds with their sources before results are seen. Fix maximum AUC fall, the random explanation sample size and its pre-correction pass-rate requirement. These PAL starting points are not regulatory requirements.

8 Fairness measures registered

Other measures / measure notes

9 Decision thresholds and sources

Fix values and provenance before locking; these changes apply to this registration only.

Threshold	Value	Source
DPR floor	Threshold DPR floor	convention
DPR sensitivity	Threshold DPR sensitivity	declared
EOD Green maximum	Threshold EOD Green maximum	declared
EOD Red above	Threshold EOD Red above	declared
AUC tolerance	Threshold AUC tolerance	declared
AUC Amber above	Threshold AUC Amber above	declared
Female cell minimum	Threshold Female cell minimum	convention
PSI Amber	Threshold PSI Amber	convention
PSI Red	Threshold PSI Red	convention
Rank stability Green minimum	Threshold Rank stability Green minimum	declared
Rank stability Red below	Threshold Rank stability Red below	declared
Proxy Amber correlation	Threshold Proxy Amber correlation	declared
Proxy Red correlation	Threshold Proxy Red correlation	declared
Proxy top SHAP count	Threshold Proxy top SHAP count	declared
Reason stability Amber	Threshold Reason stability Amber	declared
Reason stability Red	Threshold Reason stability Red	declared
Override parity floor	Threshold Override parity floor	convention
Sufficiency pass rate	Threshold Sufficiency pass rate	declared
Attrition shift maximum	Threshold Attrition shift maximum	declared
Augmentation maximum attempts	Threshold Augmentation maximum attempts	declared
95% normal quantile	Threshold 95% normal quantile	statistical

10 Maximum AUC fall

Maximum allowed AUC fall against the locked baseline.

11 Explanation sample size

Fix sample size and the required pre-correction pass rate.

Explanation pass rate (%)

D · Methods

How to complete this step

Specify held-out validation design, planned interventions, the augmentation attempt budget and statistical procedures, including uncertainty and reproducibility. Keep synthetic data out of validation/test samples.

12 Validation design

Describe folds, hold-outs, baseline and unit of analysis.

13 Planned interventions

Describe correction methods and the maximum augmentation attempts.

Augmentation attempt limit

14 Statistical procedures

Specify uncertainty intervals, tests, multiplicity and significance rules.

E · Governance

How to complete this step

Name accountable people for each decision point in the next tab, then confirm that assignment here. Record each cross-jurisdictional requirement, source jurisdiction and adoption date.

15 Decision points and owners confirmation

Confirm named owners/roles in the Decision points tab, or document non-production applicability.

16 Cross-jurisdictional requirements

e.g. Regulation B adverse-action standard applied as best practice for a Canadian institution.

Requirement 1

Source jurisdiction 1

Date adopted 1

Typed sign-offs

How to complete this step

Obtain first- and second-line acknowledgment before training. Enter each person’s name, role and date; typed names record acknowledgment, not authenticated signatures.

Typed names record acknowledgment; they are not signatures.

First line

First line name

First line role

First line date

Second line

Second line name

Second line role

Second line date

How to complete this step

Complete every required field and all decision owners, then select Lock registration before training. Locking creates a browser timestamp and fingerprint; retain your exported input record and use the Registration ID in downstream tools.
The locked plan cannot be edited. The fingerprint detects changes relative to a retained digest; it does not prove identity. Later changes require a deviation or a linked new registration.

Required before locking

1 Materiality tier and cut-offs
2 Model and use case
3 Protected groups
4 Seasoning
5 Data rules
12 Validation
13 Interventions
14 Statistics
15 Owners
2 In-house or vendor
6 Segments and minimums
7 Relationship features (state none if not applicable)
8 Measures
11 Sample size and pass rate
16 Requirements
Both typed sign-offs

Export your inputs

Key terms

Pre-registration: The plan for data, measures, decision rules, methods and accountable owners, fixed before training or scoring affected data.
Registration ID: The first twelve characters of the SHA-256 fingerprint of the locked registration, decision owners, parent link and lock time.
Fingerprint: A digest of the retained locked JSON. It detects changes relative to the saved digest; it is not a signature or proof of identity and can be recomputed by someone altering both.
Effective challenge: Independent review capable of identifying assumptions, limitations and decisions that require correction.
Deviation: A documented departure from the locked plan, with rationale and second-line acknowledgment before affected results are used.
Revalidation: A new evaluation under a new pre-registration, linked to the prior registration without changing it.

Thresholds are declared starting points fixed in the Tool 04 pre-registration. They are not regulatory requirements.