Sensitivity Analysis ====================== Two questions, two functions. They answer different things and can disagree legitimately. .. list-table:: :header-rows: 1 :widths: 20 40 40 * - - :func:`~prophys.engine.local_sensitivity` - :func:`~prophys.engine.sobol_sensitivity` * - Question - How much does the answer move when *this parameter* moves, here? - How much of the answer's *variance* does *this uncertain input* account for, across its whole range? * - Method - Automatic differentiation of the model graph - Variance decomposition over Monte-Carlo draws * - Cost - One backward pass, whatever the parameter count - ``(k + 2) * n_samples`` model evaluations * - Valid where - Near the evaluation point - Over the entire input distribution * - Reports - Derivatives and elasticities - First-order and total-order indices, hence interactions A parameter with a large derivative that is nonetheless known precisely contributes little variance; an input whose effect is symmetric about its mean can have a first-order Sobol index of zero and a large local derivative. Local sensitivity ------------------ .. code-block:: python import prophys as prp report = prp.local_sensitivity(compiled, "power", params=result.raw_params) print(report) :: LocalSensitivity: mean[power] = 208.657 d/d efficiency = +521.6 (elasticity +1.000) d/d noise = +0 (elasticity +0.000) Two numbers per parameter: **Gradient** — :math:`\partial(\text{statistic}) / \partial(\text{parameter})` in the parameter's own units. Reported with respect to the *constrained* value, so the chain rule through ``Param.forward`` is already applied. **Elasticity** — :math:`\frac{\partial y}{\partial x}\cdot\frac{x}{y}`, the fractional change in the output per fractional change in the input. Being dimensionless, it ranks parameters measured in different units against each other; an elasticity of 1.0 means the output is proportional to that parameter. It is ``nan`` where either the parameter or the statistic is zero, and :meth:`~prophys.engine.LocalSensitivity.ranked` orders those last. Only statistics with a pathwise derivative are available — the mean and the CDF. A sample quantile or CVaR is a selection among draws and is piecewise-constant in the parameters, so its automatic derivative would be zero almost everywhere; use Sobol indices for those. Global (Sobol) sensitivity --------------------------- Sobol indices decompose the variance of a model output over the joint distribution of its uncertain inputs — the attribute's upstream :class:`~prophys.domain.RandomVariable` leaves. .. code-block:: python report = prp.sobol_sensitivity(compiled, "power", n_samples=20_000) print(report) :: SobolSensitivity: variance of power over 20000 samples wind: first 0.941, total 0.966 (interaction +0.025) direction: first 0.014, total 0.049 (interaction +0.035) unexplained by first-order effects: 0.045 * **First order** :math:`S_i = \operatorname{Var}(E[Y \mid X_i]) / \operatorname{Var}(Y)` — the share of output variance removed by learning :math:`X_i` alone. Estimated by Saltelli (2010). * **Total order** :math:`S_{Ti} = E[\operatorname{Var}(Y \mid X_{\sim i})] / \operatorname{Var}(Y)` — the share attributable to :math:`X_i` through any route. Estimated by Jansen (1999). * **Interaction** :math:`S_{Ti} - S_i` — the part of an input's influence that acts only jointly with others. Near zero for a purely additive model. * **Unexplained** :math:`1 - \sum_i S_i` — variance carried entirely by interactions. The design is the standard three-matrix one: two independent samples ``A`` and ``B`` drawn from each input's own declared distribution, plus one hybrid matrix per input. Inputs are sampled independently from their marginals, which is what the decomposition assumes; components of a :class:`~prophys.domain.JointRandomVariable` (a copula, a multivariate Gaussian) count as **one** factor, since resampling one component of a correlated pair independently would evaluate the model off its own input distribution. Scenario evaluation -------------------- The Sobol design is built on :meth:`CompiledModel.conditional_expectation `, which is directly useful on its own. Where :meth:`~prophys.engine.CompiledModel.expectation` marginalizes the uncertain inputs, this conditions on values you supply — a design scenario, a return-period event, a row of a sensitivity table: .. code-block:: python import numpy as np compiled.conditional_expectation("power", {"wind": np.array([4.0, 8.0, 12.0])}) # array([ 32., 256., 864.], dtype=float32) Every named leaf takes the values given for it, so passing arrays of length ``n`` evaluates ``n`` scenarios in one batched call. Every upstream random variable must be named; an unbound leaf raises rather than reverting to a sampled or default value. .. autofunction:: prophys.engine.local_sensitivity .. autoclass:: prophys.engine.LocalSensitivity :members: .. autofunction:: prophys.engine.sobol_sensitivity .. autoclass:: prophys.engine.SobolSensitivity :members: