PBeyond Bias

This instrument is not open for responses.

Data collection begins after Golden Gate University IRB determination.

Confidential practitioner study

Practitioner Validation of the PAL Framework

A confidential 15–20 minute questionnaire with 41 questions on gender-fair AI credit decisioning, for professionals working in model risk, fair lending, and credit AI governance.

15–20 minutes41 questionsNo identifying information
0% complete

Part I

Information and Consent

Please read this consent agreement carefully before you decide to participate in the study.

The purpose of this survey is to study how financial institutions govern gender fairness in AI-enabled credit decisioning, and to test whether a proposed four-layer governance framework — the Protective Alignment (PAL) Framework — and its four accompanying practitioner tools address a governance gap practitioners recognize and could operate within their own institutions.

Researcher: Alexandria Davis, Doctor of Business Administration candidate, Golden Gate University Worldwide.

You are invited because you hold, or have held, responsibility for model risk management, model validation, fair lending compliance, internal audit, or credit AI governance at a financial institution operating in the United States or Canada.

This survey should take you 15 to 20 minutes to complete. There are 41 questions to answer. You will first review a short description of PAL and its four tools, then rate their relevance and feasibility from the perspective of your professional experience. There are no right or wrong answers; the study seeks your professional judgment.

Your participation in this study is voluntary. If any survey question makes you uncomfortable, please skip that question. You may also stop completing this survey at any time before submission by closing your browser window; nothing is retained if you do.

Your information will be kept confidential. Complete confidentiality cannot be guaranteed. No name, email address, employer name, IP address, or other direct identifier is collected. However, some questions ask about your role, your institution type, and your jurisdiction, and in a specialized professional community it may be possible for someone to decipher your identity from a combination of those answers. To reduce that possibility, results are reported only in aggregate; any category with fewer than five respondents is suppressed from reporting; results are never broken down by more than one characteristic at a time; and every open-text response is reviewed before quotation to confirm it contains no identifying detail.

Your information will be stored in a password-protected file — a spreadsheet held in a Google account used only for this study, protected by two-factor authentication and shared with no one. Google's servers are located in the United States and other countries. Only the researcher will see your individual answers, with one exception: a peer doctoral candidate outside the study independently codes a sample of open-text responses as a quality check, and sees those excerpts only.

Because no direct identifier is collected, a submitted response cannot later be located or withdrawn.

We do not anticipate any risks to you for your participation in this study beyond those of everyday professional reflection, though you may find some questions about governance practice uncomfortable to consider. Please do not disclose confidential or proprietary information belonging to your institution in any open-text field. There is no compensation for participation.

While there are no direct benefits to you for participating in this study, the information gathered will expand the understanding of how gender fairness can be engineered into AI credit decisioning systems, and will directly shape four openly published practitioner tools.

The research team may use your information for a future study but not beyond three years. At that point, the survey data, including your information, will be destroyed.

Questions about this research may be directed to Alexandria Davis at adavis582@my.ggu.edu. Questions or concerns about your rights as a research participant may be directed to the Golden Gate University Institutional Review Board at:

Therese F. Martin, DBA

Institutional Review Board
536 Mission Street
Golden Gate University
San Francisco, CA 94105
Telephone: (415) 442-7800
Email: worldwide@ggu.edu

If you agree to participate in this study, please mark the next question as “Yes, I agree to participate in this study.” If you do not agree to participate in this study, please exit this study by closing your window.

I attest that: (check each to confirm)

Agreement to proceed

Part II

Professional Profile

01

All items are categorical. No identifying information is collected.

P1. Which best describes your current (or most recent) role?*
P2. In which line of defense does your role primarily sit?*
P3. Which best describes your institution?*
P4. In which jurisdiction(s) does your institution originate credit?*
P5. Years of experience in model risk, compliance, or credit risk functions*
P6. Which best describes your institution’s use of AI/ML models in credit decisioning?*

Part III

Data Correction

02

Consider demographic representation, fairness-conditioned synthetic augmentation, and documentation in model development. See Tool 01 – Synthetic Augmentation Decision Tool.

Q1. Under-representation of certain demographic groups in historical training data is a real source of unfair outcomes in credit models at institutions like mine.
Q2. My institution’s current model development process gives insufficient structured attention to demographic representation in training data.
Q3. Fairness-conditioned synthetic augmentation is technically feasible for my institution, using either in-house capability or available vendor tooling.
Q4. Tool 01 – Synthetic Augmentation Decision Tool could be applied by a model development team like my institution’s without material external support.
Q5. The tool’s declared decision rules (parity ratio floor of 0.80; AUC tolerance of 0.02) are appropriate reference points for institutions like mine.
Q6. Documenting augmentation decisions as the tool specifies would satisfy the data-quality documentation expectations of my institution’s model risk framework.

Part III

Bias Detection

03

Consider production fairness monitoring, defined thresholds, ownership, and escalation. See Tool 02 – Fairness Monitoring Protocol.

Q8. Fairness metrics for credit models are not currently monitored in production with defined escalation thresholds at my institution (or at comparable institutions I know well).
Q9. Ongoing fairness monitoring with defined escalation ownership addresses a governance gap I recognize from my institutional experience.
Q10. Computing the protocol’s fairness and stability measures on each production scoring cohort at its stated cadence is operationally feasible for my institution.
Q10a. For each measure in the Tool 02 monitoring protocol, how feasible would it be for your institution to produce it at the stated cadence?
Demographic parity ratio (female ÷ male approval rate), with a 95% confidence intervalMonthly
Equalized odds difference, with the true-positive and false-positive gaps reported separatelyMonthly
Population stability index on model inputs and the score distributionMonthly
AUC on matured (seasoned) outcomesQuarterly
Attribution drift: rank agreement of the top ten SHAP features against the validated profileQuarterly
Proxy scan: association between each model input and applicant sexQuarterly
Adverse-action reason stability: share of denials whose top three reasons fall outside the validated reason setMonthly
Override parity: rate at which denials are overridden to approval for women ÷ the rate for menMonthly
Q10b. My institution holds, or could lawfully obtain, applicant sex for the purpose of fairness testing on its credit models.
Q11. The tiered escalation structure (green/amber/red with named owners and timelines) is compatible with my institution’s existing model risk escalation processes.
Q12. The escalation thresholds proposed (amber above 0.02, red above 0.05 equalized odds difference) are reasonable starting values for institutions like mine.
Q13. The monitoring protocol maps accurately onto the ongoing-monitoring expectations of the model risk framework my institution works under (SR 26-2, its predecessor SR 11-7 and/or OSFI E-23).

Part III

Explainability

04

Consider attribution-based explanations, counterfactual verification, proxy review, and adverse-action notices. See Tool 03 – Adverse-Action Explanation Reporting Template.

Q15. Producing specific, accurate principal reasons for adverse action from complex models is a genuine challenge at institutions like mine.
Q16. The explanation tooling currently available at my institution does not reliably produce reasons specific enough for adverse-action notices from complex models.
Q17. Attribution-based explanation with counterfactual verification, as the template specifies, is technically feasible for my institution.
Q18. The per-applicant explanation record could be integrated into my institution’s existing adverse-action workflow without unreasonable burden.
Q19. The template’s proxy flag, routing records to compliance review when a top-ranked feature is associated with a protected attribute, reflects a control my institution would find valuable.
Q20. The template’s mapping of model attributions to regulatory reason language accurately reflects how adverse-action compliance works at my institution.

Part III

Governance

05

Consider pre-registration, audit trails, evidence retention, and model risk documentation. See Tool 04 – Pre-Registration Template and Audit-Trail Checklist.

Q22. In my experience, fairness metrics and thresholds are typically chosen after model results are known, rather than committed to before training.
Q23. Without pre-commitment to metrics and thresholds, fairness assessments at institutions like mine are vulnerable to selective reporting of favorable results.
Q24. Completing the pre-registration template before model training is workable within model development timelines like my institution’s.
Q25. The audit-trail checklist corresponds to records my institution already retains or could reasonably retain.
Q26. Pre-registration and audit-trail records as specified would strengthen my institution’s documentation position under its model risk framework (SR 26-2 or its predecessor SR 11-7 and/or OSFI E-23 lifecycle stages).
Q27. The checklist’s mapping of records to model risk documentation categories (SR 26-2 or its predecessor SR 11-7 / OSFI E-23 lifecycle stages) is accurate in my experience.

Part IV

Integrated Assessment

06

Consider the four-layer PAL Framework as a whole.

Q29. Taken together, the four PAL layers address the most important fairness governance gaps in AI credit decisioning as I experience them.
Q30. The framework’s tools are proportionate—an institution of my institution’s size and resources could adopt them in a reasonable form.
Q31. If the empirical results reported for the framework hold, I would support piloting one or more of its tools at my institution.

This instrument is not open for responses. Data collection begins after Golden Gate University IRB determination.