# SALib: Sensitivity Analysis Tools for Biomedical Research

> Or just *Sensitivity Analysis Tools for Biomedical Research*?
>
> On the portal for the submission, they ask to list projects that would be involved. I list SALib as the first project and SciPy as the second one. Hopefully that would help our case. This is why I also added some wording around SciPy.
>
> Note that due to the word limit, we would not put full citations, this would be done on the full proposal

## Proposal Summary/Scope of Work

> Provide a short summary of the work being proposed (maximum of 500 words)

This proposal has 4 goals: (1) Develop a new coherent API, (2) Develop a framework for uncertainty visualization, (3) General maintenance, and (4) outreach.

1) Through a landscape analysis of how SALib and Sensitivity Analysis tools are used, we will develop a new API which will consolidate our findings. The new API will allow easier, faster and more interactive analysis. We also intend to reach out to the biomedical community to align with its needs. A side objective is to strengthen existing relation and interactions with the Python library SciPy (which is of paramount importance for the biomedical community and received CZI funding) as they started to implement Sensitivity Analysis features. SciPy's scipy.stats.sobol_indices function was a joint effort between SALib and SciPy maintainers. Pamphile Roy is a maintainer of both libraries and he added this function under the grant EOSS5-0000000176.

2) SALib mathematical tools allow to focus the analysis on the most important effects in a model but offer only simple plotting capabilities. To understand the nature of those effects, which are critical for decision-making, appropriate holistic visualizations are a must-have. We will provide a simple and consistent API which will work with all our methods. Depending on the analysis, we will use expert knowledge to make informed decisions and present end users with sensible defaults. We will also take advantage of new advances in the field and add state of the art techniques such as SimDec, functional boxplot and 3D-kiviat. We will also reach out to the Napari team to integrate sensitivity analysis tools directly into the workflow of Biomedical researchers.

3) The project requires essential maintenance work which will allow the project to be more reliable. Mainly, we will add Hypothesis testing support; support for Array and Dataframe API to be compatible with any array and Dataframe libraries; and extend our documentation by providing hands on tutorials and add interactive components thanks to the integration of jupyterlite.

4) We will do outreach activities towards the biomedical community to support them into their usage of SALib and also to raise awareness around sensitivity analysis and uncertainty quantification. We will present our work at CZI meetings, SciPy conference, local PyData conferences, Scientific Python summit and NumFOCUS events. We will also directly engage with biomedical researchers and host "office hours" to support them as needed. To that end, a space dedicated to biomedical researchers will be added to our Discord server.

## Value to Biomedical Users

> Described the expected value of the proposed work to the biomedical research community (maximum of 250 words)

Advanced senstivitity analysis methods have gained attention in recent years, from scientists and AI researchers to policymakers, assessment of uncertainties and sensitivity of numerical models is becoming a central preoccupation. Recently, the European Commission and some US institutions have recommended the use of sensitivity analysis methods. Despite this, many researchers are still predominently using standard statistical tools to analyse their models despite known shortcomings. One reason for this is the lack of usability and accessbility of available sensitivity analysis tools, although the situation is slowly improving.

Goal 1 aims to lay out a consistent and stable bedrock through a standardized API to better support rapid and interactive analyses.

Goal 2 will provide effective tools to not only help biomedical modelers to obtain further, more holistic, insights, but also uncover errors and unknown behaviours on their models. This will have a tremendous impact on the reliability of models and their decision-making power.

Goal 3 intends to improve consistency and future reliability of the tool.

Goal 4 longevity of the project by attracting new contributors as well as users through awareness.

Additionally, while SALib is not a direct dependency of SciPy, improving SALib would have a direct impact on SciPy itself and its sensitivity analysis capabilities. The team successfully presented Sobol' indices at the SciPy2023 conference and we had a lot of interest and interaction with biomedical researchers. Hence, improving SALib would have a direct impact on SciPy itself and its sensitivity analysis capabilities.

---

link to some Andrea article maybe?? -> This paper would be a great reference for the first sentence of 'Value to Biomedical Users' https://www.sciencedirect.com/science/article/pii/S1364815218302822

accountability/insights

JRC policy with SA

Take elements from https://arxiv.org/abs/2001.03965

## Landscape Analysis

> Briefly describe the other software tools (either proprietary or open source) that the audience for this proposal primarily uses. How do the software project(s) in this proposal compare to these other tools in terms of user base size, usage, and maturity? How do existing tools and the project(s) in this proposal interact? (maximum of 250 words)

Based on our review of several biomedical journals, a few open source solutions stands out. Most of them are written in Matlab (e.g. UQLab), R (e.g. sensobol) or lower-level languages (e.g. OpenTURNS, Dakota). Their impact is still limited, however, due to their comparatively small user base, usability issues particularly for non-experts, and restrictive open-source license terms (e.g., GPL, LGPL). SHAP, a newer project, has gained recent popularity but focuses on a specific analysis (Shapley effects). SHAP's success can arguably be attributed to its excellent visualization capabilities, easing result interpretation, and has found some use among biomedical researchers for genomic analysis.

In contrast, SALib is as a mature project with a substantial userbase and a permissive open-source license (MIT). It is written solely in Python, ensuring cross-platform compatibility, ease of contribution and maintenance and, most importantly, offers wide variety of state-of-the-art algorithms. It is a dependency of many open-source projects, such as agentpy (used for simulating virus spread in agent-based modeling) and Rhodium (a robust-decision making library).

Recent investigations indicate that use of SALib is gaining traction in the Biomedical fields. For example, analysis conducted in the 2022 paper on SALib found that a high number of citations from the biomedical (or adjacent) fields. Citations of the 2022 paper on SALib itself include a paper on characterizing therapeutic cell behavior in nerve tissue engineering and non-invasive blood pressure measurements.

Thus, further development of SALib, its adaptation for and popularization in the biomedical field stands out as a valuable and far-reaching initiative.

----

Take elements from https://arxiv.org/abs/2001.03965