---
tags: TEA
description: assurance project workshop
---

:::info
* **Date: 2023-07-2
* **Time**: 10:00
* **Join via**: Teams
* **Call recording**: not available

:::

[TOC]

## Agenda

* **Theme**: TEA
* **Chair**: Chris
* **Links & resources:**
* [Agenda and notes](https://thealanturininstitute.sharepoint.com/:w:/r/sites/tea-dh/_layouts/15/Doc.aspx?sourcedoc=%7B7C1771C5-55F0-425E-9278-5343BEF6FE09%7D&file=project-workshop-21-july.docx&wdOrigin=TEAMS-ELECTRON.p2p.mw&action=default&mobileredirect=true)
* [Workshop planning](https://thealanturininstitute.sharepoint.com/:w:/r/sites/tea-dh/_layouts/15/Doc.aspx?sourcedoc=%7B2ACF2A98-21BE-4B31-9074-2BC62CD3BC7F%7D&file=workshop-planning.docx&wdOrigin=TEAMS-ELECTRON.p2p.mw&action=default&mobileredirect=true)
* [Assurance platform](https://github.com/alan-turing-institute/AssurancePlatform)
* [A principles-based ethics assurance argument pattern for AI and autonomous systems](https://link.springer.com/article/10.1007/s43681-023-00297-2), _AI and Ethics_
* [Regulators & legislators workshop participants](https://thealanturininstitute.sharepoint.com/:x:/r/sites/tea-dh/_layouts/15/Doc.aspx?sourcedoc=%7BDA2BD793-77FB-40AB-85DA-D8CBAE40D7B0%7D&file=regulation-and-policy-workshop-participants.xlsx&action=default&mobileredirect=true)
* ["AI regulation: a pro-innovation approach"](https://www.gov.uk/government/publications/ai-regulation-a-pro-innovation-approach)

### Attendees
* Cass
* Kalle
* Chris
* Not in tese notes:
* Aoife (REG)
* Zoe Porter (York)
* Marten Kaas (York)
* Ibrahim Habli (York)
* Shakir Laher (RAM)

### Abbreviations
GSN = [Goal Structuring Notation](https://safetyengineering.wordpress.com/2008/04/04/the-goal-structuring-notation-gsn/)
CQC = Care Quality Commission
MAAS = Mobility as a Service?
CDEI = Centre for Data Ethics and Innovation

### Notes
#### Regulators workshop
* Regulators and policy makers workshop (==before December?==)
* Zoe: Justice-based account == Modular structure that discusses
* Boxes: 
* Can be opened into: 
* Will focus on fairness and healthcare.
* Goals for workshop:
* To understand the current needs and challenges of regulators and policy-makers for contributing to a robust and trustworthy assurance ecosystem, focusing on the specific context of fairness in digital healthcare.
* To identify opportunities for improving the Assurance Platform and corresponding methodology to build capacity in the assurance ecosystem.
* To build on and link up existing communities of practices to share best practices.
* Modularity = extensibility and reproducibility
* Hope/Aim: people can develope stand-alone modules which they can contribute to the repo
* Note this is not a feature of the code at the moment. Aoife will be looking to acheive this -- Cass to contribute community-building around use-case for this
* [name=Kalle] Clear to me that this is a high-prio for roadmapping - [name=Cass] Agree
* Needs and Challenges Mapping is prepped and done.
* User Testing and Feedback
* There is a loose plan for it, but not super important with this group -- they won't be the primary users of the platform. For other groups, a UX/UI consultancy will be involved leading this part of the workshop.
* Value of engagement should be a white paper or something that is publishable (==to be planned between now and September==)
* User testing and feedback
* Not too much to draw on because they are not the primary users of the platform.
* 10-15 users (aiming for 15, happy with 10)
* Chris has connected with Vicky, Emma and Malvika to make this transferrable to the heath theme
* ==**need more detail on this - Chris will follow up with us**==
* Proposed date 14th September - This clashes with CogX. Chris confirmed that hub team can be split across both events.

#### Practitioners workshop
* Failitated by external UI/UX consultants (Chris has started bringing in quotes)
* Practitioners
* Product owner
* design, development, deployment - who creates these systems who will need to demonstrate these to regulators
* health care professionals, doctors - not right to be brought into these workshops
* Keep it to technical discussion
* Make this clear with UI/UX team that engagine with actual "end users" will be a later point... [name=Cass] I think this is a missed opportunity to not engage with end users early on in the development [name=Kalle] Catching up on notes and **+1 here**
* Format to follow structured interviews which could be fed back to REG

#### Research deliverables
* Deliverables proposed in applicaiton: 1) conference proceedings; 2) ?paper
* Policy white paper which ==Turing will lead on==
* Government white paper ["pro-innovation approach to AI regulation"](https://www.gov.uk/government/publications/ai-regulation-a-pro-innovation-approach)
* Conference paper =
* ==York will lead on, as authors==
* ==Turing will do the community management facilitation + create knowledge-share events.==
* All co-author on each
* [ACM faact conference](https://facctconference.org/)
* held in Chicago 12th-15th June 2023
* Proposal: promising methodology for operationalising ethical goals
* Bringing in more projects across Turing who are working on fairness in healthcare, bringing in the methodology there.
* Bringing project leads into Knowledge Share focused on aligning definitions. Present projects as case studies. Develop an assurance case in the open ("I'm using this bit of AI for image recognition, and I want to make sure that the recognition is fair in X way.", "I'm working with Health Data UK, and only do data wrangling, and only care about representativeness.")
* Show how tools can be built.
* Generalise common threads -- providing fairness in the context of specific projects and how that could work across projects.
* ==Could this be basis for conference paper?==
* We brought researchers together, present the methodology, then: here's how we used methodology to extract commonalities
* Researchers = NHS (facilitated by SIG), Genomics England, [Data Science for Health Equity group](https://www.datascienceforhealthequity.com/), and Roche (which might be complicated with IP).
* [name=Cass] - I'd also be interested to bring in folks from this [project on data sharing governance](https://liberalarts.utexas.edu/psychology/news/wellcome-trust-awards-funding-to-develop-international-governance-framework-for-neuroscience-data-sharing), which I am supporting. It is a departure from AI/Fairness, so maybe better for a later stage.
* Shakir raised making the 5y roll-out plan visible (or not precluded) by the range of projects we inlcude.
* 
* This image can help illustrate the different maturity-levels of different projects to engage with the framework/methodology.
* Shows where to anchor claims
* Workshops running November time = preprint available at end of this project. Submission in January. A draft on Arxiv for the end of this year.
* Reminder: putting health in the title means that you rope in a certain audience.
* Between now and December, we chose healthcare as a way of anchoring it.
* Apply it in the digital twin communities longer-term.
* The platform is a deliverable.
* Perhaps do a Turing Impact Story if we can get some use cases of it up and running, and could then get to do a short "2-3 case studies"
* ==Knowledge share events (open development of assurance cases, perhaps around project lifecycle stage) run by us at Turing==, paper written by York
* Try to get some of the same people at this these assurance development events as the UI/UX ones. Note Malvika has flagged that participation is voluntary [name=Cass] - need ot make this a really worth while thing for people to engage in.
* [name=Kalle] added some of this up on the top of this section as well

#### Platform development
* Technically a "tool" at the moment, aim for it to becaome a "platform"
*
* [Feature planning issue](https://github.com/alan-turing-institute/AssurancePlatform/issues/159)
* Add: modularity-side of things
* Assurance case repository for folks to download and learn from
* Needs a schema.
* Should include a comment feature
* UI/UX will be lead by expert agency
* [name=Kalle] I have actually been playing around with Figma to learn, with the assurance platform as a test
* Nice example: [Digital Public Goods Alliance website](https://digitalpublicgoods.net/)
* Standards
* Work to encourage users to employ FAIR, i.e. prompt for doi in evidence
* Note there are models other then GSN which we should consider in the design for interoperability.
* Collaboration
* Think about where the collaboration (e.g. a team developing a case) should happen - on the platfor or using github features. Chris flagged making sure this is accessible to non-git folks. [name=Cass] I'd prefer to keep this on github to keep it slightly more closed and keep people using that platform. We could scaffold how to do this.

##### Feedback on dev priorities
* Ibrahim - non-expert usage (there are already plenty of tools experts can use) ==can we get some examples of these?==
* tip pop-outs
* constraints on what can be connected, so people don't have to know the compliance
* template based instantiation, so please don't have to start from scratch - they are easy to understand but hard to start from scratch
* Zoe - MVP
* Shakir - [tool Shakir and team created for GSN] (https://www.cs.york.ac.uk/safedh/SMART.html)
* Lessons learned: user goes through safety argument. Insights to learn from this about user functionality.
* Get the infrastructure ready early on! Go to metamodels, and look into modularity. Make is extensible beyond GSN.
* Templates and videos -- help non-expert users.
* Priorities. Build these into a roadmap.
* Repo usability / completeness
* UI/UX
* Consisentency - make sure foundations are there

### Action points for follow up
* get into this project with Shakir https://www.cs.york.ac.uk/safedh/SMART.html. Looks to be some nice reporting on what they did
* Set up a miro space for some cellective ideation about stuff
*
*

### Next Meeting
*
*
*
*

### Information and Links
*
*
*
*