# Explainability Argument Pattern :::info ℹī¸ **About this Document** This is a work-in-progress argument for the second health workshop for the TEA-DT project. Please read the instructions below before reviewing the content of the argument pattern. The `.JSON` file can be accessed here: https://raw.githubusercontent.com/alan-turing-institute/AssurancePlatform/main/examples/Explainability%20(TEA-DT%20Health%20Workshop)-2024-5-18T9-54-49.json ::: ## Instructions 1. Review the [goal and context](#goal-and-context) for this pattern. 2. Discuss the questions. If you answer "no" to either of the questions, please make adjustments as required. 3. Review the [strategies and property claims](#strategies-and-property-claims) for this pattern. 4. Discuss the questions and make revisions as required. 5. Optional: - Flag any evidence types (e.g. randomised control trial, risk assessment report) that you would use to support any claims. - Flag any claims that you are unsure how you would evidence. :::success 📝 **Cheat Sheet** A cheat sheet with descriptions of each of the core elements as well as some general tips for developing clear and concise assurance cases is available. 👉 [Access Cheat Sheet](https://hackmd.io/@tea-platform/BJyX12aSA) ::: ## Goal and Context The *goal claim* adopted for this pattern is as follows: > The outputs of the digital twin are explainable to patients. The context for this pattern is as follows: > A clinical use case (e.g. hospital or GP surgery), where the explanation will be provided by a trained healthcare professional (e.g. consultant). Some more specific context claims to consider: - **C1:** Description of the digital twin system and its purpose. - **C2:** Operational environment and intended use cases. - **C3:** Applicable regulations and standards for safety and security. - **C4:** Assumptions about external systems and user interactions. :::warning ❓ **Questions** 1. Is this goal claim appropriately specified? Is the goal claim clear? 2. Does the context statement capture all intended use cases? ::: ## Strategies and Property Claims ### Strategy 1: Argument Over Communication Method - **Property Claims**: 1. The outputs are explainable to patients through verbal communication during a consultation. 2. The outputs are explainable to patients through visual aids (e.g., diagrams, charts). 3. The outputs are explainable to patients through written reports. 4. The user interface supports timely delivery of explanations. 5. The user interface is intuitive and meets the needs of the healthcare professional. ### Strategy 2: Argument Over Patient Understanding - **Property Claims**: 1. The outputs are explainable to patients with a basic understanding of medical terms. - Assumption: Basic understanding of medical terms is based on the [NHS Service Manual's](https://service-manual.nhs.uk/content/health-literacy) notion of 'Health Literacy'. 2. The outputs are explainable to patients of varying demographics. 1. The outputs are explainable to patients of varying age groups. 2. The outputs are explainable to patients with different cultural backgrounds. 3. The outputs are explainable to patients with different levels of education. 3. Any uncertainty arising from the use of probabilistic techniques can be communicated appropriately to patients. 4. The features used by the system are meaningful and intelligible to patients. ### Strategy 3: Argument Over User Training - **Property Claims**: 1. Users are trained to explain the process behind the digital twin's development and deployment. 2. The model is sufficiently interpretable by relevant users. 3. The digital twin includes features that help users generate patient-friendly explanations. - The digital twin pipeline provides real-time support for users in explaining complex outputs. - The digital twin integrates with other tools that aid in patient communication (e.g., translation services, interactive visual aids). ### Strategy 4: Argument Over Adequate Scope of Explanations - **Property Claims**: 1. The {statistical model} used by the digital twin is interpretable. - All features used by the {statistical model} are clinically relevant and meaningful. - The {statistical model} is intrinsically interpretable (e.g., logistic regression, decision trees). - The {statistical model} uses post hoc techniques to ensure interpretability (e.g. feature importance for neural networks). - The user interface has clear explanations for how the input features influence the digital twin's outputs (e.g. recommendations, visualisations) 2. The process of how the digital twin was designed, developed, and deployed is explainable. - Information about the project's governance is accessible to relevant parties. - Information about the project's governance is sufficient to support appropriate explanations as needed. 3. Automated explanations (or explantion-supporting information) are accurate. - Methods for deriving explanations achieve a sufficient level of predictive consistency and reliability. - Methods for deriving explanations are sufficiently representative of the actual underlying architecture. ### Strategy 5: Argument Over Transparency - **Property Claims**: 1. Data governance processes have been clearly documented. - Assumption: processes for documentation are sufficient for context of use (e.g. details about pre-processing, data collection, metadata and annotations) 2. Patients can provide feedback on the explanations given, and this feedback is used to improve future explanations. 3. Mechanisms exist to support contestability of decisions by patients. 4. There is a mechanism to assess patient understanding after explanations are provided. 5. Explanations are refined based on patient feedback. - Assumption: Patient comprehension assessments are sufficient for determining relevant refinements. 6. Methods for scrutinising ongoing behaviour of system are available to appropriate auditors or assessors. 7. A clear audit trail will be established to ensure decision traceability. :::warning ❓ **Questions** 1. Do the strategies collectively support (and help operationalise) the goal claim? 2. Do the strategies help identify all necessary requirements for the project or system, which can be developed into property claims? - If you answered "no" to this question, which property claims are missing? - NB: when first approaching this pattern, you should assume that many relevant property claims are missing. 3. Are new strategies required? Do existing strategies need to be revised (e.g. merged or split)? 4. Do any of the current strategies or claims fall outside of the scope of your responsibilities (e.g. assurance for property X would be delivered by a third-party)? 5. (Optional) Do any of the property claims suggest a specific type of evidence (e.g. results of model testing, user evaluation report)? :::