Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
af7127e
chore: first commit
May 29, 2025
ee31017
docs: update myst
Jun 4, 2025
fb22152
abstract and more bibtex
Jun 4, 2025
ce4b265
improve introduction
Jun 5, 2025
d474515
first version of introduction
Jun 5, 2025
2fcdd78
starting core concepts section
Jun 6, 2025
1fd3153
complete core concepts
Jun 6, 2025
ce69ad4
start with SHAP for GBDTs
Jun 9, 2025
fdb9406
Add first plots in GBDT
Jun 10, 2025
100a066
explainations for gbdt shap
Jun 10, 2025
d70dfec
start with CNN analysis
Jun 10, 2025
426894d
more code snippets for gbdt
Jun 10, 2025
c154824
start with model definition
Jun 11, 2025
d1693e7
add cnn model class
Jun 11, 2025
7626690
details about model training
Jun 11, 2025
d222b22
add conclusion
Jun 12, 2025
6b5cc60
completed strengths and limitations
Jun 12, 2025
ec2aa53
add perf metrics for cnn
Jun 12, 2025
b6c19f0
add global and dep plots
Jun 13, 2025
2eb0f31
added all plots
Jun 13, 2025
d90c958
add content for global shap in cnn
Jun 13, 2025
470fa8e
revert back unwanted changes
Jun 13, 2025
c6f01fb
fix: add CRediT roles
ab93 Jun 16, 2025
3921ec4
fix: add doi
ab93 Jun 16, 2025
f778450
fix: ignore lightgbm doi
ab93 Jun 16, 2025
34696c3
fix: lightgbm citation
ab93 Jun 16, 2025
0fd6743
feat: add notebook for GBDT
Jun 25, 2025
991e1fa
fix: setminus error
Jun 25, 2025
928fb21
chore: reformat gbdt nb
Jun 25, 2025
7ae7d0f
feat: first version of cnn nb
Jun 27, 2025
03a5909
fix: affiliations
Jun 27, 2025
626914f
feat: improve reproducibility
Jun 27, 2025
71c566a
improve content for Dependency and Local plots
Jul 9, 2025
2f8c9da
comments: add UCIMLR citation
ab93 Aug 24, 2025
02ca7aa
comments: address expanding shap explanation section
ab93 Aug 24, 2025
fac0fde
comments: reason why categorical features need dependency plots
ab93 Aug 24, 2025
05ed180
comments: causality reason
ab93 Aug 24, 2025
fc0b2f2
fix: doi error
ab93 Aug 24, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added papers/avik_basu/banner.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added papers/avik_basu/beeswarm_Laying_cnn.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added papers/avik_basu/beeswarm_Sitting_cnn.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added papers/avik_basu/beeswarm_Standing_cnn.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added papers/avik_basu/beeswarm_Walking_cnn.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added papers/avik_basu/beeswarm_avg_cnn.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added papers/avik_basu/beeswarm_xgb.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added papers/avik_basu/dep_Laying_cnn.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added papers/avik_basu/dep_Sitting_cnn.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added papers/avik_basu/dep_Standing_cnn.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added papers/avik_basu/dep_Walking_Downstairs_cnn.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added papers/avik_basu/dep_Walking_Upstairs_cnn.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added papers/avik_basu/dep_Walking_cnn.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added papers/avik_basu/dep_categorical_xgb.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added papers/avik_basu/dep_contact_xgb.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added papers/avik_basu/dep_poutcome_xgb.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added papers/avik_basu/local_Laying_cnn.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added papers/avik_basu/local_Sitting_cnn.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added papers/avik_basu/local_Standing_cnn.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added papers/avik_basu/local_Walking_Downstairs_cnn.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added papers/avik_basu/local_Walking_cnn.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
962 changes: 962 additions & 0 deletions papers/avik_basu/main.md

Large diffs are not rendered by default.

296 changes: 296 additions & 0 deletions papers/avik_basu/mybib.bib
Original file line number Diff line number Diff line change
@@ -0,0 +1,296 @@
@inproceedings{jupyter,
abstract = {It is increasingly necessary for researchers in all fields to write computer code, and in order to reproduce research results, it is important that this code is published. We present Jupyter notebooks, a document format for publishing code, results and explanations in a form that is both readable and executable. We discuss various tools and use cases for notebook documents.},
author = {Kluyver, Thomas and Ragan-Kelley, Benjamin and Pérez, Fernando and Granger, Brian and Bussonnier, Matthias and Frederic, Jonathan and Kelley, Kyle and Hamrick, Jessica and Grout, Jason and Corlay, Sylvain and Ivanov, Paul and Avila, Damián and Abdalla, Safia and Willing, Carol and {Jupyter development team}},
editor = {Loizides, Fernando and Scmidt, Birgit},
location = {Netherlands},
publisher = {IOS Press},
url = {https://eprints.soton.ac.uk/403913/},
booktitle = {Positioning and Power in Academic Publishing: Players, Agents and Agendas},
year = {2016},
pages = {87--90},
title = {Jupyter Notebooks - a publishing format for reproducible computational workflows},
}

@article{matplotlib,
abstract = {Matplotlib is a 2D graphics package used for Python for application development, interactive scripting, and publication-quality image generation across user interfaces and operating systems.},
author = {Hunter, J. D.},
publisher = {IEEE COMPUTER SOC},
year = {2007},
doi = {https://doi.org/10.1109/MCSE.2007.55},
journal = {Computing in Science \& Engineering},
number = {3},
pages = {90--95},
title = {Matplotlib: A 2D graphics environment},
volume = {9},
}

@article{numpy,
author = {Harris, Charles R. and Millman, K. Jarrod and van der Walt, Stéfan J. and Gommers, Ralf and Virtanen, Pauli and Cournapeau, David and Wieser, Eric and Taylor, Julian and Berg, Sebastian and Smith, Nathaniel J. and Kern, Robert and Picus, Matti and Hoyer, Stephan and van Kerkwijk, Marten H. and Brett, Matthew and Haldane, Allan and del Río, Jaime Fernández and Wiebe, Mark and Peterson, Pearu and Gérard-Marchant, Pierre and Sheppard, Kevin and Reddy, Tyler and Weckesser, Warren and Abbasi, Hameer and Gohlke, Christoph and Oliphant, Travis E.},
publisher = {Springer Science and Business Media {LLC}},
doi = {https://doi.org/10.1038/s41586-020-2649-2},
date = {2020-09},
year = {2020},
journal = {Nature},
number = {7825},
pages = {357--362},
title = {Array programming with {NumPy}},
volume = {585},
}

@misc{pandas1,
author = {{The Pandas Development Team}},
title = {pandas-dev/pandas: Pandas},
month = feb,
year = {2020},
publisher = {Zenodo},
version = {latest},
url = {https://doi.org/10.5281/zenodo.3509134},
}

@inproceedings{pandas2,
author = {Wes McKinney},
title = {{D}ata {S}tructures for {S}tatistical {C}omputing in {P}ython},
booktitle = {{P}roceedings of the 9th {P}ython in {S}cience {C}onference},
pages = {56 - 61},
year = {2010},
editor = {{S}t\'efan van der {W}alt and {J}arrod {M}illman},
doi = {https://doi.org/10.25080/Majora-92bf1922-00a},
}

@article{scipy,
author = {Virtanen, Pauli and Gommers, Ralf and Oliphant, Travis E. and
Haberland, Matt and Reddy, Tyler and Cournapeau, David and
Burovski, Evgeni and Peterson, Pearu and Weckesser, Warren and
Bright, Jonathan and {van der Walt}, St{\'e}fan J. and
Brett, Matthew and Wilson, Joshua and Millman, K. Jarrod and
Mayorov, Nikolay and Nelson, Andrew R. J. and Jones, Eric and
Kern, Robert and Larson, Eric and Carey, C J and
Polat, {\.I}lhan and Feng, Yu and Moore, Eric W. and
{VanderPlas}, Jake and Laxalde, Denis and Perktold, Josef and
Cimrman, Robert and Henriksen, Ian and Quintero, E. A. and
Harris, Charles R. and Archibald, Anne M. and
Ribeiro, Ant{\^o}nio H. and Pedregosa, Fabian and
{van Mulbregt}, Paul and {SciPy 1.0 Contributors}},
title = {{{SciPy} 1.0: Fundamental Algorithms for Scientific
Computing in Python}},
journal = {Nature Methods},
year = {2020},
volume = {17},
pages = {261--272},
adsurl = {https://rdcu.be/b08Wh},
doi = {https://doi.org/10.1038/s41592-019-0686-2},
}

@article{sklearn1,
author = {Pedregosa, F. and Varoquaux, G. and Gramfort, A. and Michel, V. and Thirion, B. and Grisel, O. and Blondel, M. and Prettenhofer, P. and Weiss, R. and Dubourg, V. and Vanderplas, J. and Passos, A. and Cournapeau, D. and Brucher, M. and Perrot, M. and Duchesnay, E.},
year = {2011},
journal = {Journal of Machine Learning Research},
pages = {2825--2830},
title = {Scikit-learn: Machine Learning in {P}ython},
volume = {12},
}

@inproceedings{sklearn2,
author = {Buitinck, Lars and Louppe, Gilles and Blondel, Mathieu and Pedregosa, Fabian and Mueller, Andreas and Grisel, Olivier and Niculae, Vlad and Prettenhofer, Peter and Gramfort, Alexandre and Grobler, Jaques and Layton, Robert and VanderPlas, Jake and Joly, Arnaud and Holt, Brian and Varoquaux, Gaël},
booktitle = {ECML PKDD Workshop: Languages for Data Mining and Machine Learning},
year = {2013},
pages = {108--122},
title = {{API} design for machine learning software: experiences from the scikit-learn project},
}

@misc{human_activity_recognition_using_smartphones_240,
author = {Reyes-Ortiz, Jorge, Anguita, Davide, Ghio, Alessandro, Oneto, Luca, and Parra, Xavier},
title = {{Human Activity Recognition Using Smartphones}},
year = {2013},
howpublished = {UCI Machine Learning Repository},
doi = {https://doi.org/10.24432/C54S4K},
note = {{DOI}: https://doi.org/10.24432/C54S4K}
}

@misc{bank_marketing_222,
author = {Moro, S., Rita, P., and Cortez, P.},
title = {{Bank Marketing}},
year = {2014},
howpublished = {UCI Machine Learning Repository},
doi = {https://doi.org/10.24432/C5K306},
note = {{DOI}: https://doi.org/10.24432/C5K306}
}


@inproceedings{DBLP:journals/corr/ChenG16,
author = {Tianqi Chen and Carlos Guestrin},
title = {XGBoost: A Scalable Tree Boosting System},
booktitle = {Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining},
year = {2016},
pages = {785--794},
doi = {https://doi.org/10.1145/2939672.2939785},
}

@misc{paszke2019pytorchimperativestylehighperformance,
title={PyTorch: An Imperative Style, High-Performance Deep Learning Library},
author={Adam Paszke and Sam Gross and Francisco Massa and Adam Lerer and James Bradbury and Gregory Chanan and Trevor Killeen and Zeming Lin and Natalia Gimelshein and Luca Antiga and Alban Desmaison and Andreas Köpf and Edward Yang and Zach DeVito and Martin Raison and Alykhan Tejani and Sasank Chilamkurthy and Benoit Steiner and Lu Fang and Junjie Bai and Soumith Chintala},
year={2019},
eprint={1912.01703},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/1912.01703},
doi = {https://doi.org/10.48550/arXiv.1912.01703},
}

@article{DBLP:journals/corr/LundbergL17,
author = {Scott M. Lundberg and
Su{-}In Lee},
title = {A unified approach to interpreting model predictions},
journal = {CoRR},
volume = {abs/1705.07874},
year = {2017},
url = {http://arxiv.org/abs/1705.07874},
eprinttype = {arXiv},
eprint = {1705.07874},
timestamp = {Fri, 26 Nov 2021 16:33:36 +0100},
biburl = {https://dblp.org/rec/journals/corr/LundbergL17.bib},
bibsource = {dblp computer science bibliography, https://dblp.org},
doi = {https://doi.org/10.48550/arXiv.1705.07874},
}

@article{shapley1953value,
title={A value for n-person games},
author={Shapley, Lloyd S and others},
year={1953},
publisher={Princeton University Press Princeton},
doi = {https://doi.org/10.1515/9781400881970-018},
}

@book{molnar2025,
title={Interpretable Machine Learning},
subtitle={A Guide for Making Black Box Models Explainable},
author={Christoph Molnar},
year={2025},
edition={3},
isbn={978-3-911578-03-5},
url={https://christophm.github.io/interpretable-ml-book}
}

@article{DBLP:journals/corr/abs-1802-03888,
author = {Scott M. Lundberg and
Gabriel G. Erion and
Su{-}In Lee},
title = {Consistent Individualized Feature Attribution for Tree Ensembles},
journal = {CoRR},
volume = {abs/1802.03888},
year = {2018},
doi = {https://doi.org/10.48550/arXiv.1802.03888},
url = {http://arxiv.org/abs/1802.03888},
eprinttype = {arXiv},
eprint = {1802.03888},
timestamp = {Mon, 13 Aug 2018 16:46:31 +0200},
biburl = {https://dblp.org/rec/journals/corr/abs-1802-03888.bib},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

@article{lundberg2020local2global,
title={From local explanations to global understanding with explainable AI for trees},
author={Lundberg, Scott M. and Erion, Gabriel and Chen, Hugh and DeGrave, Alex and Prutkin, Jordan M. and Nair, Bala and Katz, Ronit and Himmelfarb, Jonathan and Bansal, Nisha and Lee, Su-In},
journal={Nature Machine Intelligence},
volume={2},
number={1},
pages={2522-5839},
year={2020},
doi = {https://doi.org/10.1038/s42256-019-0138-9},
publisher={Nature Publishing Group}
}


@article{DBLP:journals/corr/ShrikumarGK17,
author = {Avanti Shrikumar and
Peyton Greenside and
Anshul Kundaje},
title = {Learning Important Features Through Propagating Activation Differences},
journal = {CoRR},
volume = {abs/1704.02685},
year = {2017},
doi = {https://doi.org/10.48550/arXiv.1704.02685},
url = {http://arxiv.org/abs/1704.02685},
eprinttype = {arXiv},
eprint = {1704.02685},
timestamp = {Thu, 14 Oct 2021 09:14:21 +0200},
biburl = {https://dblp.org/rec/journals/corr/ShrikumarGK17.bib},
bibsource = {dblp computer science bibliography, https://dblp.org}
}


@article{friedman2001greedy,
title={Greedy function approximation: a gradient boosting machine},
author={Friedman, Jerome H},
journal={Annals of statistics},
pages={1189--1232},
year={2001},
doi = {https://doi.org/10.1214/aos/1013203451},
publisher={JSTOR}
}

@inproceedings{NIPS2017_6449f44a,
author = {Ke, Guolin and Meng, Qi and Finley, Thomas and Wang, Taifeng and Chen, Wei and Ma, Weidong and Ye, Qiwei and Liu, Tie-Yan},
booktitle = {Advances in Neural Information Processing Systems},
editor = {I. Guyon and U. Von Luxburg and S. Bengio and H. Wallach and R. Fergus and S. Vishwanathan and R. Garnett},
pages = {},
publisher = {Curran Associates, Inc.},
title = {LightGBM: A Highly Efficient Gradient Boosting Decision Tree},
url = {https://proceedings.neurips.cc/paper_files/paper/2017/file/6449f44a102fde848669bdd9eb6b76fa-Paper.pdf},
volume = {30},
year = {2017}
}

@article{Moro2014ADA,
author = {S{\'e}rgio Moro and P. Cortez and Paulo Rita},
title = {A Data-Driven Approach to Predict the Success of Bank Telemarketing},
journal = {Decision Support Systems},
volume = {62},
pages = {22--31},
year = {2014},
doi = {https://doi.org/10.1016/j.dss.2014.03.001},
}

@article{lecun1998gradient,
author = {LeCun, Yann and Bottou, Léon and Bengio, Yoshua and Haffner, Patrick},
title = {Gradient-based Learning Applied to Document Recognition},
journal = {Proceedings of the IEEE},
volume = {86},
number = {11},
pages = {2278--2324},
year = {1998},
doi = {https://doi.org/10.1109/5.726791}
}

@article{Krizhevsky_2017,
title={ImageNet classification with deep convolutional neural networks},
volume={60},
ISSN={1557-7317},
url={http://dx.doi.org/10.1145/3065386},
doi={https://doi.org/10.1145/3065386},
number={6},
journal={Communications of the ACM},
publisher={Association for Computing Machinery (ACM)},
author={Krizhevsky, Alex and Sutskever, Ilya and Hinton, Geoffrey E.},
year={2017},
month=may,
pages={84–90}
}

@misc{mitchell2022gputreeshapmassivelyparallelexact,
title = {GPUTreeShap: Massively Parallel Exact Calculation of SHAP Scores for Tree Ensembles},
author = {Mitchell, Rory and Frank, Eibe and Holmes, Geoffrey},
year = {2022},
doi = {https://doi.org/10.48550/arXiv.2010.13972},
eprint = {2010.13972},
archivePrefix= {arXiv},
primaryClass = {cs.LG},
url = {https://arxiv.org/abs/2010.13972},
}

@misc{UCI_ML_Repository,
author = {Kelly, Markelle and Longjohn, Rachel and Nottingham, Kolby},
title = {The UCI Machine Learning Repository},
url = {https://archive.ics.uci.edu},
institution = {University of California, Irvine}
}
51 changes: 51 additions & 0 deletions papers/avik_basu/myst.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
version: 1
extends: ../papers.yml
project:
# Update this to match `scipy-2025-<folder>` the folder should be `<firstname_surname>`
id: scipy-2025-avik_basu
# Ensure your title is the same as in your `main.md`
title: Explaining ML predictions with SHAP
subtitle: Intuition, use cases, and best practices
description: |
This article explores how SHAP (SHapley Additive exPlanations) can be used to interpret machine learning model
predictions by providing consistent and theoretically grounded feature attributions.
# Authors should have affiliations, emails and ORCIDs if available
authors:
- name: Avik Basu
email: avik_basu@intuit.com
orcid: 0009-0002-4489-378X
affiliations:
- Intuit, Inc.
- PyOpenSci
roles:
- Conceptualization
- Software
- Visualization
- Writing – original draft
keywords:
- explainable AI
- Shapley values
# Add the abbreviations that you use in your paper here
abbreviations:
MyST: Markedly Structured Text
SHAP: SHapley Additive exPlanations
LLM: Large Language Model
CNN: Convolutional Neural Network
GBDT: Gradient Boosted Decision Trees
# It is possible to explicitly ignore the `doi-exists` check for certain citation keys
error_rules:
- rule: doi-exists
severity: ignore
keys:
- jupyter
- sklearn1
- sklearn2
- NIPS2017_6449f44a
- molnar2025
- UCI_ML_Repository
exports:
- id: pdf
format: typst
template: https://github.com/curvenote-templates/scipy.git
article: main.md
output: full_text.pdf
Loading
Loading