PBeyond Bias

Confidential practitioner study

Practitioner Validation of the PAL Framework

A confidential 15–20 minute questionnaire with 39 questions on gender-fair AI credit decisioning, for professionals working in model risk, fair lending, and credit AI governance.

15–20 minutes39 questionsNo identifying information
0% complete

Part I

Information and Consent

Please read this consent agreement carefully before you decide to participate in the study.

The purpose of this survey is to study how financial institutions govern gender fairness in AI-enabled credit decisioning, and to test whether a proposed four-layer governance framework — the Protective Alignment (PAL) Framework — and its four accompanying practitioner tools address a governance gap practitioners recognise and could operate within their own institutions.

Researcher: Alexandria Davis, Doctor of Business Administration candidate, Golden Gate University Worldwide.

You are invited because you hold, or have held, responsibility for model risk management, model validation, fair lending compliance, internal audit, or credit AI governance at a financial institution operating in the United States or Canada.

This survey should take you 15 to 20 minutes to complete. There are 39 questions to answer. You will first review a short description of PAL and its four tools, then rate their relevance and feasibility from the perspective of your professional experience. There are no right or wrong answers; the study seeks your professional judgment.

Your participation in this study is voluntary. If any survey question makes you uncomfortable, please skip that question. You may also stop completing this survey at any time before submission by closing your browser window; nothing is retained if you do.

Your information will be kept confidential. Complete confidentiality cannot be guaranteed. No name, email address, employer name, IP address, or other direct identifier is collected. However, some questions ask about your role, your institution type, and your jurisdiction, and in a specialised professional community it may be possible for someone to decipher your identity from a combination of those answers. To reduce that possibility, results are reported only in aggregate; any category with fewer than five respondents is suppressed from reporting; results are never broken down by more than one characteristic at a time; and every open-text response is reviewed before quotation to confirm it contains no identifying detail.

Your information will be stored in a password-protected file — a spreadsheet held in a Google account used only for this study, protected by two-factor authentication and shared with no one. Google's servers are located in the United States and other countries. Only the researcher will see your individual answers, with one exception: a peer doctoral candidate outside the study independently codes a sample of open-text responses as a quality check, and sees those excerpts only.

Because no direct identifier is collected, a submitted response cannot later be located or withdrawn.

We do not anticipate any risks to you for your participation in this study beyond those of everyday professional reflection, though you may find some questions about governance practice uncomfortable to consider. Please do not disclose confidential or proprietary information belonging to your institution in any open-text field. There is no compensation for participation.

While there are no direct benefits to you for participating in this study, the information gathered will expand the understanding of how gender fairness can be engineered into AI credit decisioning systems, and will directly shape four openly published practitioner tools.

The research team may use your information for a future study but not beyond three years. At that point, the survey data, including your information, will be destroyed.

Questions about this research may be directed to Alexandria Davis at adavis582@my.ggu.edu. Questions or concerns about your rights as a research participant may be directed to the Golden Gate University Institutional Review Board at:

Therese F. Martin, DBA

Institutional Review Board
536 Mission Street
Golden Gate University
San Francisco, CA 94105
Telephone: (415) 442-7800
Email: worldwide@ggu.edu

If you agree to participate in this study, please mark the next question as “Yes, I agree to participate in this study.” If you do not agree to participate in this study, please exit this study by closing your window.

I attest that: (check each to confirm)

Agreement to proceed

Part II

Professional Profile

01

All items are categorical. No identifying information is collected.

P1. Which best describes your current (or most recent) role?*
P2. In which line of defense does your role primarily sit?*
P3. Which best describes your institution?*
P4. In which jurisdiction(s) does your institution originate credit?*
P5. Years of experience in model risk, compliance, or credit risk functions*
P6. Which best describes your institution’s use of AI/ML models in credit decisioning?*

PAL Layer 01

Data Correction

02

Consider demographic representation, fairness-controlled synthetic data, and documentation in model development.

Q1. Under-representation of certain demographic groups in historical training data is a real source of unfair outcomes in credit models at institutions like mine.*
Q2. My institution’s current model development process gives insufficient structured attention to demographic representation in training data.*
Q3. Generating fairness-controlled synthetic training data is technically feasible for my institution, using either in-house capability or available vendor tooling.*
Q4. The decision tool shown in Appendix A could be applied by a model development team like my institution’s without material external support.*
Q5. The tool’s declared decision rules (parity ratio floor of 0.80; AUC tolerance of 0.02) are appropriate reference points for institutions like mine.*
Q6. Documenting augmentation decisions as the tool specifies would satisfy the data-quality documentation expectations of my institution’s model risk framework.*

PAL Layer 02

Bias Detection

03

Consider production fairness monitoring, defined thresholds, ownership, and escalation.

Q8. Fairness metrics for credit models are not currently monitored in production with defined escalation thresholds at my institution (or at comparable institutions I know well).*
Q9. Ongoing fairness monitoring with defined escalation ownership addresses a governance gap I recognize from my institutional experience.*
Q10. Computing equalized odds and parity metrics on each production scoring cohort at the protocol’s stated cadence is operationally feasible for my institution.*
Q11. The tiered escalation structure (green/amber/red with named owners and timelines) is compatible with my institution’s existing model risk escalation processes.*
Q12. The escalation thresholds proposed (amber above 0.02, red above 0.05 equalized odds difference) are reasonable starting values for institutions like mine.*
Q13. The monitoring protocol maps accurately onto the ongoing-monitoring expectations of the model risk framework my institution works under (SR 26-2, its predecessor SR 11-7 and/or OSFI E-23).*

PAL Layer 03

Explainability

04

Consider attribution-based explanations, counterfactual verification, proxy review, and adverse-action notices.

Q15. Producing specific, accurate principal reasons for adverse action from complex models is a genuine challenge at institutions like mine.*
Q16. The explanation tooling currently available at my institution does not reliably produce reasons specific enough for adverse-action notices from complex models.*
Q17. Attribution-based explanation with counterfactual verification, as the template specifies, is technically feasible for my institution.*
Q18. The per-applicant explanation record could be integrated into my institution’s existing adverse-action workflow without unreasonable burden.*
Q19. The template’s proxy flag, routing records to compliance review when a top-ranked feature is associated with a protected attribute, reflects a control my institution would find valuable.*
Q20. The template’s mapping of model attributions to regulatory reason language accurately reflects how adverse-action compliance works at my institution.*

PAL Layer 04

Governance

05

Consider pre-registration, audit trails, evidence retention, and model risk documentation.

Q22. In my experience, fairness metrics and thresholds are typically chosen after model results are known, rather than committed to before training.*
Q23. Without pre-commitment to metrics and thresholds, fairness assessments at institutions like mine are vulnerable to selective reporting of favorable results.*
Q24. Completing the pre-registration template before model training is workable within model development timelines like my institution’s.*
Q25. The audit-trail checklist corresponds to records my institution already retains or could reasonably retain.*
Q26. Pre-registration and audit-trail records as specified would strengthen my institution’s documentation position under its model risk framework (SR 26-2 or its predecessor SR 11-7 and/or OSFI E-23 lifecycle stages).*
Q27. The checklist’s mapping of records to model risk documentation categories (SR 26-2 or its predecessor SR 11-7 / OSFI E-23 lifecycle stages) is accurate in my experience.*

Part IV

Integrated Assessment

06

Consider the four-layer PAL Framework as a whole.

Q29. Taken together, the four PAL layers address the most important fairness governance gaps in AI credit decisioning as I experience them.*
Q30. The framework’s tools are proportionate—an institution of my institution’s size and resources could adopt them in a reasonable form.*
Q31. If the empirical results reported for the framework hold, I would support piloting one or more of its tools at my institution.*

Submitting records your response confidentially. Please review your answers before continuing.