---
title: "Translational Machine Learning for Health: A software-carpentry style short course"
author: Steve Harris
breaks: false # False means NOT to render line breaks as hard line breaks.
created: 2023-06-05
version: 0.1
---

# Translational Machine Learning for Health:
A software-carpentry style short course

The Algorithm Stewardship curriculum instills the knowledge and skills required to design, build, and deploy machine learning solutions within the NHS using FlowEHR. The curriculum employs a real-world case study of predicting emergency department admissions, and encompasses key elements of the model deployment pipeline: understanding and manipulating features, building robust prediction models, managing model and data drift, identifying and addressing risks, creating model cards, and effectively communicating model uncertainties.

This curriculum is designed to be taught over 2-3 full days or four half-days, and culminates in a capstone challenge offering additional hands-on practice. The curriculum utilises a practical problem from the healthcare domain that does not demand any specific domain knowledge. However, attendees should have some experience with machine learning models, and familiarity with Python programming is highly advantageous. This could be gained, for instance, from attending a Software Carpentry, Data Carpentry: Ecology, or Data Carpentry: Social Sciences workshop.

## Proposed course outline

- **Title:** Algorithm Stewardship: Developing and Deploying Machine Learning for Health into the NHS with FlowEHR
- **Duration:** 2-3 Days
- **Objective:** This course equips professionals with the practical skills to work with FlowEHR, an open-source platform for iterative, safe, and reproducible development and deployment of data science solutions within the NHS. We focus on the application of these skills to real-world machine learning problems in healthcare settings.

### Lessons
- **Introduction to FlowEHR:** Gain an understanding of FlowEHR, its use, capabilities, and how it can enable multidisciplinary teams to work efficiently on data science projects within the NHS. Learn how to harness this tool to enhance patient outcomes and boost health system efficiency.
- **The case study via a Jupyter Notebook:** Using a predictive model for emergency department (ED) admissions as a case study, participants will delve into the intricacies of applying machine learning to a real-life healthcare problem. This problem-solving process will utilise a set of XGBoost classifiers applied to 100k+ ED visits, forecasting emergency admissions within specified time windows.
- **Feature Pipeline Development and Deployment:** Learn to build a testable feature pipeline that can both train the model and serve new predictions. We will cover feature building, testing, and orchestration of the pipeline for batch data using Apache Spark.
- **Monitoring for Data and Model Drift:** Build and deploy tools for monitoring data drift (e.g. using [Alibi-detect](https://docs.seldon.io/projects/alibi-detect/en/latest/)), a critical aspect of maintaining the accuracy and reliability of machine learning models in a healthcare setting. Understand the importance of model drift and how to detect and address it to ensure model effectiveness over time.
- **Risk Management and Hazard Identification:** Engage with risk management processes required for standards such as [DCB0129/160](https://digital.nhs.uk/data-and-information/information-standards/information-standards-and-data-collections-including-extractions/publications-and-notifications/standards-and-collections/dcb0129-clinical-risk-management-its-application-in-the-manufacture-of-health-it-systems) in machine learning deployment, including identifying potential hazards and managing a risk register. Gain insight into how to prevent, mitigate, and respond to potential risks in AI deployment.
- **Creation of Model Cards and Algorithmic Stewardship Dashboard:** Develop an understanding of model cards and how to create them for enhancing transparency and accountability. Learn to build a dashboard for algorithmic stewardship, a critical tool for monitoring and managing deployed models.
- **Communication of Model Uncertainty:** Learn strategies to effectively communicate model uncertainty to end-users and understand how this might change as model drifts and data inputs change over time.

**Outcome:** Participants will gain practical experience and a deep understanding of deploying machine learning models in the healthcare sector. They will be equipped to navigate the engage with the intricacies of algorithmic stewardship in healthcare, contributing to the development and maintenance of effective and safe AI solutions within the NHS.

## Plan of Work: Developing the 'Algorithm Stewardship' Short Course

- **Weeks 1-3:** Course Design and Initial Lesson Development\
Our ML-Ops and Software Carpentry experts will outline the course and begin crafting the lesson plans for each module. The key objective is to create a comprehensive roadmap that aligns with the detailed curriculum content.

- **Weeks 4-6:** Creation of Course Material\
The team will generate essential teaching resources, including slides, Python code examples, and relevant datasets. This phase will prioritise producing accessible, user-friendly material, consistent with Software Carpentry's hands-on approach.

- **Weeks 7-8:** Lesson Refinement and Pilot Testing
Course materials will be critically reviewed, revised, and pilot-tested to ensure pedagogical soundness and efficacy in achieving the targeted learning outcomes.

- **Week 9:** Initial Course Delivery
The course will be delivered for the first time, facilitating feedback gathering and providing an opportunity for assessing lesson effectiveness.

- **Weeks 10-11:** Revision and Enhancement
Feedback and observations from the initial delivery will guide the team in refining and improving course materials. This stage will involve necessary modifications to ensure content clarity and effective delivery.

- **Week 12:** Publishing and Expansion Planning
The final week will see the course materials finalised and readied for publication within the Software Carpentry community. Furthermore, a roadmap will be created for developing an MSc module at UCL, based on this material. This plan will ensure the course's longevity and continued accessibility for FlowEHR users.

---
[](https://hackmd.io/iUZAkwCbQxSNiUVDvmhVYQ)