# TEA-DT Workshop (Natural Environment)


:::success
⏱️ **Agenda**

**Times:** 13:00–16:50 (UK Time)
**Breaks:** we will take breaks every hour, but please feel free to step away from the computer whenever you need.

1. Introduction to project and workshops
2. Introductions from participants
3. Structured discussion around assurance needs and capabilities
4. Presentation 1: What is TEA?
5. Wrap-Up
:::

:::info
ℹ️ **About this Document**

Please use this document to answer each of the following questions. You can provide your answers anonymously. To do so, please ensure you are not signed in, and do not write anything that may reveal identifying information.
:::

## Questions

### 1) What does ‘assurance’ mean to you?

- The ability to state a given asset's function and process of operation is ==reliable== for adoption.
- Can I ==trust== a given asset's function and process?
- Accuracy = accurate for what? what's your error metrics? Assurance then is having suite of metrics that accounts for as many errors in your model as possible. (==Transparency==?)
- Does this depend on stakeholders that are using your model or who you may be communicating with?
- The model/approach is suitable, reliable and trustworthy for the stated purpose or application.
- Eliminating spurious information.
- It's important to state the initial purpose for the users not involved in the design/development process.
- How well is the thing, whatever that might be, that you're interested in, meeting an objective or a series of objectives beyond reasonable doubt?
- Someone using a DT who wasn't a designer and developer because it shows that other people can actually apply it.
- Also, are the statements made about the model/approach true.
- Metrics of assurance.
- Seeing is believing = Until they have things in their hands, users won't build confidence and trust.
-

### 2) What processes of assurance do you currently follow? (and Why?)

- Understanding black box nature of AI models.
- Validation - good replication of a past event/condition of interest.
- From modelling point of view: validation, calibration, uncertainty quantification, benchmarking, model intercomparison experiments, documentation, peer review publications, stakeholder engagement
- Evaluation and sign-off by operators (e.g. end users). Assessment of competence.
- Controlled adoption
-
1) Start "simple" with things we understand and can demonstrate.
2) Bring in expertise where it is needed
3) Implement frequent/regular scientific, coding, and technical review procedures
4) Caveats and assumptions should be made explicit.
5) Plan, do, check, act
-

### 3) Are there any guiding principles that you follow? [aspire to assure]

- Transparency (often the case in civil service) -- supporting co-creation of technical review documents, validation, testing from different teams. How to validate that is still a question... the ability for someone to review your approach.
- FAIRness (Findable, Accessible, Interoperable, Reusable)
- 3 hands raised for "who has encountered this principle in their work?"
- How do we communicate that to the end user?
- FAIR can also mean different things to different places
- There is an ISO standard for quality management, which is useful here: https://www.iso.org/standard/62085.html#:~:text=ISO%209001%20is%20a%20globally,demonstrate%20their%20commitment%20to%20quality
- Fit-for-purpose, utility, accuracy, performance.
- Where does responsibility for assurance start and stop?
- Model generalisability (e.g. applicability to new domains, accurate across various scales/areas)
- Answerability, auditability, accountability
-
-
-

### 4) What techniques or mechanisms do you use to assure these principles?

- Sensitivity analysis
- Internal and External Audit by an independent third party
- Plan, Check, Do, Act method
- Accreditation Frameworks (e.g. ISO9001 and so on) offer some fundamental structure
- Description of provenance for data pipelines within product / service descriptions
-
-

## Feedback

### 1. Do you have any feedback about the TEA platform?

- Answer 1
- Answer 2
- Answer 3

### 2. Do you have any feedback about the TEA-DT workshops?

- Answer 1
- Answer 2
- Answer 3

### 3. Do you have any feedback about the TEA-DT project, more generally?

- Answer 1
- Answer 2
- Answer 3