Sensitivity Analysis in Python: scipy.stats.sobol_indices

Alt title:

More than error bars: Sensitivity Analysis in Python with scipy.stats.sobol_indices

Short Summary

The study of uncertainties can provide invaluable insights on a system or model. A central question is: what parameter of my system is the most important. This is sensitivity analysis and the Sobol' indices are helping answer this question.

Thanks to new developments in SciPy, there is now a function for that: scipy.stats.sobol_indices.

This method has been successfully used to analyse systems in various fields, enabling practicionners make critical decisions: Can I neglect this parameter? Is it more important than my other parameters? Can I give it a ranking? Can I explain my machine learning model's parameters?

Abstract

Uncertainties are omnipresent and when it comes to complex systems, referring to nominal values is too restrictive. Understanding the nature and the impact of these uncertainties has become an important aspect of engineering work. On a societal point of view, uncertainties play a role in terms of decision-making. From the European Commission through the Better Regulation Guideline, impact assessments are now advised to take uncertainties into account.

The field of Uncertainty Quantification (UQ) seeks to link a set of input perturbations on a system towards a quantity of interest. We focus specifically on determining which input of a system is the most influential. This problem is also refered to as: Sensitivity Analysis (SA).

The Sobol' indices is arguably the method which is the most used and provides a sensible interpretation: what is the contribution of each parameter to the variance of the quantities of interest? With this information, systems/models can be simplified, inputs prioritized and knowledge is gained.

This presentation will start with a short introduction on uncertainties and pivot towards sensitivity analysis. Then, it will offer some intuition on Sobol' indices. Finally, it will present scipy.stats.sobol_indices the new feature introduced in SciPy using a practical example–from the Biomedical field–and conclude with pitfalls to avoid and recommendations when dealing with Sobol' indices.

SciPy is commited to providing foundational tools that other libraries and applications can rely on. This presentation seeks to generate discussion, gather feedback to improve the library and call for new contributors.

Other information

Track: General
Keywords: SciPy, Uncertainty Quantification, Sensitivity Analysis
Type: Talk

Author 1:

First Name: Pamphile T.
Last Name: Roy
Email: [e-mail]
Country/Region: Austria
Organization: Quansight
Web page: https://github.com/tupui

Author 2:

First Name: Matt
Last Name: Haberland
Email: [e-mail]
Country/Region: USA
Organization: California Polytechnic State University, San Luis Obispo
Web page: https://brae.calpoly.edu/faculty-and-staff-haberland