Explainable AI (XAI)

# Explainable AI (XAI) Basics <br> <br> <small>Neil John D. Ortega</small><br> <small> <small>ML Engineer @ LINE Fukuoka</small><br> <small>2021/09/10</small> </small> --- ## Agenda - What is XAI? - Why XAI? - Taxonomy of methods - Example: Integrated Gradients (IG) - Recap --- ## Agenda - **What is XAI?** - Why XAI? - Taxonomy of methods - Example: Integrated Gradients (IG) - Recap ---- ## What is Explainable AI (XAI)? - A set of techniques that can be used to make ML models more __understandable__ to humans: - Allows to ML/DL experts to navigate the _learning phase_ of a model - Allows _knowledge discovery_ in certain problem domains - Increases _reliability and robustness_ of ML models against adversarial attacks ---- ## XAI Models - Should be __F.A.S.T.__: - __F__ air - __A__ ccountable - __S__ ecure - __T__ ransparent ---- ## Two Myths.. <span>__Myth #1__: _"Only DL models need explainability."_</span> <span>__Myth #2__: _"Unavoidable trade-off between performance and interpretability."_</span> <div> __Goal__: ![Learning performance vs explainability](https://i.imgur.com/66wxSxc.png) <small><strong>Fig. 1.</strong> Learning performance vs explainability. [2]</small> </div> ---- ## Interpretability vs Explainability <div> _"All explainable models are interpretable._ _But not all interpretable models are explainable."_ \- Albert Einstein </div> ##### Physics analogy: - Quantum mechanics - can be used to make predictions about the real (microscopic) world... - Interpretability is understanding the mechanics of ML/DL - As to __why__ the quantum world behaves that way, we don't yet have complete understanding - Explainability is having a global theory --- ## Agenda - What is XAI? - **Why XAI?** - Taxonomy of methods - Example: Integrated Gradients (IG) - Recap ---- ## Why XAI? - __Model validation__ - Is the model fair? Is it unbiased? Does it respect privacy? - Important with compliance to industry rules and regulations, increased adoption in other industries ---- ## Why XAI? - __Model debugging__ - Is the model reliable and robust? - Is the model transparent and interpretable enough? - Important when model is misbehaving ---- ## Why XAI? - __Knowledge discovery__ - Can we go beyond prediction? Can the model shed light in areas where explanations and causal relations are lacking? - Important with scientific applications/discovery ---- ## Do we always need XAI? #### It depends. ## :shrug: --- ## Agenda - What is XAI? - Why XAI? - **Taxonomy of methods** - Example: Integrated Gradients (IG) - Recap ---- ## Criteria [4] - __C__ omplete  - __A__ ccurate  - __M__ eaningful  - __C__ onsistent  ---- ## Intrinsic vs Post-hoc - __Intrinsic__ depends highly on the kind (i.e. complexity) of the model used - __Post-hoc__ refers to methods done _after_ the model is trained <div> ![Intrinsic vs Post-hoc methods](https://i.imgur.com/HbxaXoW.png) <small><strong>Fig. 2.</strong> Intrinsic vs post-hoc methods. [4]</small> </div> ---- ## Local vs Global - __Local__ focuses on subsets of model prediction space (can be individual sample) - __Global__ uses aggregated, ranked contributions over entire model prediction space - Precision vs recall <div> ![Local vs Global methods](https://i.imgur.com/fZogTYT.png) <small><strong>Fig. 3.</strong> Local vs global methods. [4]</small> </div> ---- ## Model-Specific vs Model-Agnostic - __Model-specific__ can only be used with specific models (e.g. gradient-based, etc.) - __Model-agnostic__ can be used with any model (e.g. SHAP, etc.) <div> ![Model-specific vs Model-agnostic methods](https://i.imgur.com/9Tv1pCE.png) <small><strong>Fig. 4.</strong> Model-specific vs Model-agnostic methods. [4]</small> </div> ---- ## Explanation Type [4] - Feature attribution - Visualization - Decision rules - Counterfactual datapoints - Approximate models, etc. --- ## Agenda - What is XAI? - Why XAI? - Taxonomy of methods - **Example: Integrated Gradients (IG)** - Recap ---- ## Integrated Gradients - Post-hoc - Model-specific - Local/global ---- ## [Integrated Gradients TF Collab](https://colab.research.google.com/github/tensorflow/docs/blob/master/site/en/tutorials/interpretability/integrated_gradients.ipynb) :computer: --- ## Agenda - What is XAI? - Why XAI? - Taxonomy of methods - Example: Integrated Gradients (IG) - **Recap** ---- <style> .reveal h1 {font-size: 2.0em !important;} .reveal h2 {font-size: 1.28em !important;} .reveal ul {font-size: 32px !important;} .reveal ol strong, .reveal ul strong { color: #E26A6A !important; } </style> ## Recap - XAI is a set of techniques aimed at making ML/DL models more transparent - XAI can be applied for model validation, model debugging, and knowledge discovery - XAI methods can be classified (loosely) into different categories to facilitate further study - Integrated Gradient is one method that can be used to interpret gradient-based methods --- # Thank you! :nerd_face: --- ## References  <style> .reveal p {font-size: 20px !important;} .reveal ul, .reveal ol { display: block !important; font-size: 18px !important; } .reveal li {line-height: 1.4 !important;} section[data-id="references"] p { text-align: center !important; } </style> - [1] Du, Mengnan, et al. “[Techniques for Interpretable Machine Learning](https://arxiv.org/abs/1808.00033).” Communications of the ACM, vol. 63, no. 1, 2019, pp. 68–77., doi:10.1145/3359786. - [2] Gianfagna, Leonida, and Antonio Di Cecco. Explainable AI with Python. Springer, 2021. - [3] “Integrated Gradients: Tensorflow Core.” TensorFlow, www.tensorflow.org/tutorials/interpretability/integrated_gradients. - [4] “Introduction to Explainable AI (ML Tech Talks).” YouTube, YouTube, 15 July 2021, www.youtube.com/watch?v=6xePkn3-LME. - [5] Sun, Yi, and Mukund Sundararajan. “[Axiomatic Attribution for Multilinear Functions](https://arxiv.org/abs/1703.01365).” Proceedings of the 12th ACM Conference on Electronic Commerce - EC '11, 2011, doi:10.1145/1993574.1993601.