Post-hoc explainability has a proliferation problem, not a scarcity problem. SHAP, LIME, Integrated Gradients, DeepLIFT, Grad-CAM: there's no shortage of methods for explaining why a model made a prediction. What's missing is a consistent way to tell whether any given explanation is actually good, or just plausible-looking.
xai_evals treats generation and evaluation as one package, not two separate research directions. It supports SHAP and LIME for scikit-learn, XGBoost, CatBoost, and LightGBM models, and PyTorch/TensorFlow attribution methods (Integrated Gradients, DeepLIFT, Saliency, Grad-CAM, Occlusion) for deep learning, including a DLBacktrace-based tabular explainer built in.
Every explanation can then be scored against the same metric suite:
- Tabular metrics: faithfulness, infidelity, sensitivity, comprehensiveness, sufficiency, monotonicity, complexity, sparseness.
- Image metrics: faithfulness correlation, sensitivity measures, MPRT and SmoothMPRT (via Quantus).
That gives you a quantifiable, apples-to-apples comparison: run SHAP and Integrated Gradients on the same model, score both against the same faithfulness metric, and see which one actually reflects the model's behavior rather than just looking clean in a plot.
Who it's for
- ML teams that need to justify a model's predictions, not just explain them, in a regulated or audited context.
- Researchers comparing explainability methods who need a shared, quantitative basis for the comparison.
- Anyone deploying tabular or deep learning models where "trust me, the SHAP plot looks reasonable" isn't good enough.
Links:
- arXiv: https://arxiv.org/abs/2502.03014
- Authors: Pratinav Seth, Yashwardhan Rathore, Neeraj Kumar Singh, Chintan Chitroda, Vinay Kumar Sankarapu