Explainability

prophys.explain attributes predictions of a surrogate to its inputs, and risk statistics of a whole model to its random inputs. Coalition arithmetic (exact Shapley values, interaction indices, the KernelSHAP solve, TreeSHAP) runs in the optimized native kernels.

Energy yield of a turbine whose power curve is a learned surrogate. The variance is almost entirely wind, but the tail average above the 90 percent quantile owes a visible share to availability, losses and to what the power curve surrogate does not know.

Local explanations

Every method accepts a surrogate, a tree ensemble, or any function of an [N, d] array, and returns ShapValues with attributions values [N, K, d] and the reference value base so that base + values.sum(-1) reproduces each prediction.

Function	Method	When to use
tree_shap(ensemble, x)	Exact path dependent TreeSHAP (Lundberg et al. 2020).	Tree ensembles, including imported XGBoost and LightGBM models.
shapley_values(model, x, background)	Exact interventional Shapley values over all coalitions, with pairwise Shapley interaction indices.	Up to sixteen inputs.
kernel_shap(model, x, background, n_samples)	KernelSHAP with paired coalitions from the Shapley kernel. Exact when the budget covers every coalition.	Many inputs, any model.
expected_gradients(model, x, background)	Integrated gradients averaged over background baselines, an approximation of interventional Shapley values.	Differentiable models with many inputs.
integrated_gradients(model, x, baseline)	Path integral of the gradient, complete up to quadrature error.	A single meaningful baseline.

from prophys import explain

sv = explain.tree_shap(trees, records[:200])
sv.importance()                      # mean absolute attribution per input
ex = explain.shapley_values(power, x[:10], x_background)
ex.interactions[:, 0]                # pairwise interaction indices

System level attribution

shapley_effects() attributes a statistic of a model attribute to the model’s random variables. The value of a coalition J is a statistic of the conditional mean E[Y | X_J], estimated by nested Monte Carlo on the compiled graph (outer draws of X_J, inner draws of the rest).

statistic	Value of a coalition	Meaning of the attribution
variance	Var E[Y | X_J] with the inner sampling bias removed	Shapley effects (Owen 2014, Song et al. 2016). They sum to the variance, split it fairly under dependence, and equal Sobol indices for additive models.
cvar	Tail average of E[Y | X_J] above the level quantile	How much each input contributes to the tail average exceeding the mean. A tail risk attribution.
exceedance	Variance of the indicator E[Y | X_J] > threshold	Which inputs drive the exceedance of a limit.

r = explain.shapley_effects(model, "energy", statistic="cvar", level=0.9,
                            n_outer=2000, n_inner=32)
print(r)            # players, values and shares
r.interactions      # pairwise interaction indices of the same game

groups maps a player name to several random variables, for inputs that are dependent (a copula) and should be attributed together. Every coalition is evaluated exactly for up to twelve players. The epistemic random variable of a surrogate (s.epistemic) is a player like any physical input, so the share of risk owed to the surrogate’s ignorance is reported next to the physics.

Accuracy

The inner sample size sets the precision of the conditional means. For the variance the resulting bias is removed. For the tail average it is not, and players with a small true share are overstated when n_inner is small, so use at least 32 inner draws for tail attribution.