# Trustworthy and Ethical Assurance ![](https://raw.githubusercontent.com/alan-turing-institute/turing-commons/main/docs/assets/images/illustrations/assurance-alt.jpg) :::info ℹ️ **About this note** This note sets out tasks and some possible themes for Amanda, Ben, Chris, Geoff and Jason to discuss, following on from the conversation on 19th April 2024. Useful resources: - TEA Platform GitHub repository: https://github.com/alan-turing-institute/AssurancePlatform - TEA Platform documentation: https://alan-turing-institute.github.io/AssurancePlatform/ - TEA-DT Project Webpage: https://www.turing.ac.uk/research/research-projects/trustworthy-and-ethical-assurance-digital-twins-tea-dt ::: ## Tasks - [x] Chris to draft possible themes and intellectual tasks based on discussion - [ ] Chris to send over information to help development of a letter of support for BRAID programme - [ ] All to review and add comments - [ ] Chris and Jason to discuss and explore prototype of formal model - [ ] Amanda to arrange for Chris to give talk to relevant groups or teams at Google ## Themes ### 1. Developing best approach to formal modelling of an assurance case Having a rigorous approach to formally modelling an assurance case could enable several useful features for the platform: - Communicating uncertainty - Guiding user to improve argument - Improving adoption by communities with additional requirements (e.g. clinical safety or efficacy) To achieve this, it is first necessary to determine how an assurance case should be represented formally (e.g. DAG). ### 2. UI/UX improvements As Ben mentioned, the current tool still expects a high-level of expertise from the user. What improvements can be made to the platform to improve the onboarding process? Some planned features include: 1. Tooltips to explain key elements: (https://github.com/alan-turing-institute/AssurancePlatform/issues/170) 2. Implement an interactive tutorial for new users: https://github.com/alan-turing-institute/AssurancePlatform/issues/332 3. Develop self-directed curriculum to support skills and training: https://github.com/alan-turing-institute/AssurancePlatform/tree/tea-curriculum/site/docs/curriculum ### 3. Empirical study of teams adopting TEA platform There are numerous studies that could be carried out to determine/validate the utility of the platform and methodology. Co-designing a research project with the Moral Imagination team and University of York could be a valuable opportunity. Possible research questions could include: 1. Does the use of the TEA platform improve a project team's ability identify ethical hazards? 2. Are individuals willing to acknowledge and disclose ethical gaps? Does the structure of an assurance case help them to do this? 3. What barriers exist for project teams adopting this tool? 4. Does the tool complement existing approaches (e.g. model cards, algorithmic impact assessments)? Another option here is to conduct a study comparing multiple interventions, and evaluating results (e.g. use of TEA platform vs. expert facilitation vs. LLM support). ### 4. Co-design of argument patterns Argument patterns are templates that can be loaded into the platform to help teams assure common goals (e.g. fairness, explainability, safety, security). Developing these templates is a key priority to a) improve the usability of the platform, and b) enable some of the other research themes. ### 5. Build dataset of assurance cases for training LLMs If the TEA platform does go on to integrate LLMs as a form of decision support, it will be important to ensure there are reliable and user-tested assurance cases available (e.g. for one-shot inference or RAG). ### 6. Incorporating LLMs into platform to augment capabilities What is the roadmap for incorporating LLMs into the TEA platform (e.g. technical and infrastructural requirements, key evaluation steps)? What would a release schedule for this look like? ### 7. Stress testing capabilities of the TEA platform How could the TEA platform be stress tested further through exposure to new projects (e.g. beyond digital twins research)? What tests need to be designed and carried out to properly evaluate the TEA platform? ### 8. Exploring regulatory and governance gaps and capabilities While the TEA platform is intended to aid reflection and deliberation, it is also built around a technique and rocess that has traditionally been used for regulatory compliance or conformity assessment (e.g. safety cases). As policy-makers and regulators increasinlgy call for some form of ethical regulation, how can the TEA platform help project teams to meet these additional responsibilities?