# The Ethics of Training and Interpreting Logistic Regression Models
### CS181
### Spring 2023

Suppose that you fit a logistic regression model to predict whether a loan application should be approved. Suppose that you have three covariates:

1. `x_1` representing gender: 0 for male, 1 for female
2. `x_2` for the income
3. `x_3` for the loan amount

Suppose that the parameters you found are:
$$
p(y=1 | x_1, x_2, x_3) = \mathrm{sigm}(-1 + 3 x_1 + 1.5 x_2 + 1.75 x_3).
$$

What are the parameters telling us about the most influential attribute for predicting loan approval? What does this say about our data?

*Note:* If you noticed issues with us calling `x_1` gender, but then coding for "male" and "female", or that we are encoding gender as a binary variable see the section on "Biases Arising from Data Collection and Encoding" below.

We might be pleased that by interpreting our model we can hypothesize **why** the outcome $y$ is a certain way - why is the loan approved or denied. But just because we can collect the data for the covariates does it mean that we should use them? It particular, in our logistic model above, the most impactful factor on a loan decision was gender, does this seem reasonable? Should we use this model in real-life loan decisions?

In models that are built on human data, covariates containing information that infringes on the rights of the subject to remain anonymous or to keep potentially non-relevant and biasing information out of the decision making process are called **sensitive** or **protected attributes**.

### When not to use sensitive/protected attributes

Gender, in the case of loan decisions, is a protected attribute, since under the [Equal Credit Opportunity Act](https://www.consumerfinance.gov/fair-lending/) makes it illegal for a creditor to discriminate in any aspect of credit transaction based on certain characteristics including gender. Thus, any decision process using gender to inform a loan decision can be potentially considered discriminatory.

In fact, most credit models used in industry are trained on data that are stripped of protected attributes - meaning that these models never see covariates like gender during training or deployment!

If we had deployed the logistic model to make real life loan decisions, not only would our decisions be potentially unfair and possibly questionable in terms of financial soundness (since our decisions are heavily influenced by a covariate not directly related to the financial qualifications of a loan applicant), using such a model exposes our company to regulatory sanctions and law-suits.

### When you might want to use sensitive/protected attributes

So should we never collect data on protected attributes? Well, unfortunately, just because we are blind to protected attributes it does not mean that our decisions are fair with respect to these attributes! That is, just because we don't see gender when making decisions, it does not mean that our decisions impact men and women equally. In fact, the Equal Credit Opportunity Act contains explicit language that protects consumers from the [disparate impact](https://www.americanbanker.com/opinion/dont-ditch-disparate-impact) of credit decision systems. Disparate impact is defined as the unequal impact of a credit policy on a legally protected class:

> "A lender's policies, even when applied equally to all its credit applicants, may have a negative effect on certain applicants. For example, a lender may have a policy of not making single family home loans for less than \$60,000. This policy might exclude a high number of applicants who have lower income levels or lower home values than the rest of the applicant pool. That uneven effect of the policy is called disparate impact."

How do modelers and engineers prevent their models from creating disparate impact for protected classes of people? One of the key ways to check for disparate impact is to compare model decisions on protected classes against model decisions against the population (for example, we can compare the percentages of loans the model approves for men and for women). But in order perform these checks, we need access to protected attributes!

Often times, the models we are auditing for disparate impact are "black-boxes", that is, the inner workings of the model (as well as details like the training procedure) are all proprietary information and we only have access to the models inputs and outputs, $(x_2, x_3, y)$ (note that the black-box model is not using the protected attribute gender $x_1$). In these cases, we can train a proxy model on covariates including protected attributes as well as the predictions of the black-box model, and then analyze our proxy model as an approximation to the black-box model. If we suppose that our logistic regression model in the previous slide was a proxy model that is trained to approximate a black-box model, then our proxy model is telling us that the black-box model's decisions are highly correlated with gender and hence it's decisions may cause disparate impact! This is a case where we needed to look at protected attributes in order to check for regulartory compliance!

So why is this happening - why is a black-box model that is trained on data stripped of gender information making decisions that are highly correlated with gender? There are many possible explanations for this, but one common reason for this is that gender is a **confounding variable**, that is, some combination of income and loan amount is secretly encoding for gender (e.g. let's say that in your data set, all the men earn 50k-60k and apply for exactly \$10,000 of loans) and the black-box model is ***implicitly*** relying on gender to make loan decisions.

### Appropriate usage of sensitive/protected attributes

So now we see that protected attributes can be extremely useful when we are auditing models for regulatory compliance, should we collect sensitive or protected attributes all the time and as many of them as possible? Unfortunately, it's not so simple: some industries are required by law to collect sensitive attribute data, while others are prohibited from doing so, still others infer sensitive attributes from collected data for compliance checking purposes (e.g. using income and other covariates to infer gender or zip codes to infer race). While useful for antidiscrimnatory purposes, collecting or infering sensitive protected attribute is nonetheless full of challenges and potential pitfalls (for example, protected attributes you infered can violate the subject's right to non-disclosure!) - references in this section addresses the issues of appropriate usage of sensitive attributes.

### Biases Arising from Data Collection and Encoding
Furthermore, **how** you collect and ecnode data can deeply impact your fairness/compliance analysis. For example, application forms that include two options for gender and three boxes for race may cause us to incorrectly aggregate subjects who do not fall neatly into these boxes under categories that are inappropriate (and by the way, we've not been carefully distinguishing gender and sex in this discussion, but they maybe treated differently under different bodies of laws. Also, are they equivalent concepts to our human subjects from whom we are collecting the data? What are we really trying to measure by collecting gender or sex data?). Such misleading categorization can cause us to miss cases of disparate impact in our analysis, since we have simply missed out on important subgroups of the population.

If you feel like we're now treading in the unfamiliar territories of social science or law, you are not mistaken! When we work with human data, the interpretation of the data and hence our analysis hinges on our ability to meaningfully understand the data in social contexts and in human terms. Just as we need to consult physicist domain experts when we are building a statistical model for physical data, medical experts when we are building models for medical data, when we build models that have social impact we have to proceed carefully under the advisement of experts on our social/legal systems.

**Optional Readings on Issues Involving Proper Use of Protected Attributes**

0. [New Categories Are Not Enough: Rethinking the Measurement of Sex and Gender in Social Surveys](https://journals.sagepub.com/doi/10.1177/0891243215584758)
1. [Estimates of Multiracial Adults and Other Racial and Ethnic Groups Across Various Question Formats]("https://www.pewsocialtrends.org/2015/11/06/chapter-1-estimates-of-multiracial-adults-and-other-racial-and-ethnic-groups-across-various-question-formats/")
2. [Blind Justice: Fairness with Encrypted Sensitive Attributes](https://arxiv.org/pdf/1806.03281.pdf)
3. [Awareness in Practice: Tensions in Access to Sensitive Attribute Data for Antidiscrimination](https://arxiv.org/pdf/1912.06171.pdf)