# Fairlearn: Theory of change
This is an attempt to write out the **theory of change** in the work that has morphed into the "deep dive" meetings.
- [Seed ideas for scenarios](https://hackmd.io/gD8dwdPSRsqH3BxBt9terg?both)
- [Sociotechnical "talking points"](https://hackmd.io/nDiDafJ6TMKi2cYDHnujtA)
- [Brainstorming: Website for "fairness as a sociotechnical challenge" community #498](https://github.com/fairlearn/fairlearn/issues/498)
- [Meta notes](https://hackmd.io/2NBeOFZuS6CpJVQVudhFqg?both)
## Anchor quote
The core idea is similar to what is expressed in [Engaging the Ethics of Data Science in Practice (Barocas and Boyd 2017)](https://cacm.acm.org/magazines/2017/11/222176-engaging-the-ethics-of-data-science-in-practice/fulltext):
> All too often, the data scientists we have encountered are quite sympathetic to the sentiment behind the critiques they hear, but feel maligned and misunderstood, unacknowledged for their efforts, and frustrated by vague recommendations that are not actionable... Data scientists, as the ones closest to the work, are often the best positioned to address ethical concerns, but they often need help from those who are willing to take time to understand what they are doing and the challenges of their practice.
The theory of change for this work is based on a few premises:
1. **Translational work is key.** There's incredible work happening in ML fairness, but much of this is not visible or legible to practicing developers and data scientists. Even translating something from a research article to a blog post can bridge practical and cultural gaps.
2. **Abstraction traps are everywhere.** Folks working as developers and data scientists often value abstraction in ways that can make it challenging to even begin engaging with sociotechnical fairness work. See Selbst et al. (2019).
3. **Language matters**. Words that function as attractive signals in one context (eg, "lived experience" or "oppression") can function differently in other contexts (eg,, "scalable and efficient mechanism" or "clean modular design"). We'll need to notice these nuances, and use different languages for different purposes. Normalizing through comparison with other computing practices (eg, security threat modeling, accessibility) is an opportunity.
4. **Big tech companies are different.** The organizational change practices that find success within large technology companies are likely to be different from other contexts. Most organizations are unlikely to have platform teams to create, teach, and enforce organizational processes. The use of third party services may be more common, and funding levels are likely to be lower, particularly for fairness work. Working styles are less likely to be waterfall and more likely to have visibility only into a few parts of the ML lifecycle.
5. **Growth is personal and slow**. Each person's notion of fairness is tied up with their own identity and beliefs, lived experiences, and disciplinary training. Listing principles is not teaching, and real growth only happens over time.
## Data scientists' opportunities to influence
Data scientists are not a monolithic group, and organizational structures vary wildly across different deployment contexts. But as a rough starting point, data scientists may have opportunity to influence certain kinds of work more easily than others, based on organizational assumptions about their role's responsibilities and other kinds of constraints (in the same spirit of the anchor quote by Barocas and Boyd (2017). See also [Passi and Sengers (2020)](https://journals.sagepub.com/doi/pdf/10.1177/2053951720939605) and [Madaio et al. (2020)](http://www.jennwv.com/papers/checklists.pdf).
[PDF chart v3](https://drive.google.com/file/d/1leJHwmi4LLYl4CQVSGN4ob18IJvnBrAv/view?usp=sharing)
Based on these premises, the theory of change involves:
1. **Small bites**. Web pages, not papers. Code samples, not proofs. Incremental adoption of a small tool, rather than a single big module. Discussion starters that take 60 minutes, not 60 question audits. We should aim to provide something valuable within an hour.
2. **Actionable communication**. For practicing data scientists and developers, the primary action they're going to take is communicative. It's not going to be shipping code. We've been successful if we've given them a new visual or chart or question to bring to their team or other stakeholders. We are not helping them discover insights on their own, but helping them tee up productive communication.
4. **Tangible and concrete**. In order to avoid abstraction traps, being tangible and concrete is critical. We start with examples, and then generalize afterward, and tackle complexity by comparing examples rather by abstracting. We can't work without getting to real deployment contexts, and without building domain expertise.
5. **New pathways.** After those initial small bites, we want to connect them with a pathway towards further group. This path ends at research papers, community groups, reports by nonprofits, participatory ML methods, etc. But there are many middle steps that we may be able to help with as well. Microsoft's marketing of the project is a strength, even if the company has different goals than the open source project.
6. **Building a community**. The project has been incredibly successful in the last few months with recruiting community that's excited to engage with fairness work. This is a strength to build on, particularly since growth is personal and unfolds over time.
## Next steps
See links above. See also - [Brainstorming: Website for "fairness as a sociotechnical challenge" community #498](https://github.com/fairlearn/fairlearn/issues/498), which references a separate project that prototypes what a website oriented around this concept might look like (and how it might be different from the current fairlearn.org website).