Observations

calibrate and finetune accept a plain array of observations, which asserts that every value was measured exactly. Real measurement records routinely are not:

• a concentration below an instrument’s detection limit is known only to be somewhere below it;

• a specimen that survived the whole test is known only to have failed after the test ended;

• a reading logged to the nearest bin is known only to lie within it;

• a failed sensor produced no information at all.

Substituting a number for any of those — the limit, a half-limit, a bin midpoint, an interpolated fill — changes the likelihood the fit maximizes, and so changes the fitted parameters. Observations carries what the record actually says.

Kind	Contributes	Constructor
exact	\(\log f(x)\)	exact()
left-censored	\(\log F(\text{limit})\)	detection_limit()
right-censored	\(\log(1 - F(\text{limit}))\)	right_censored()
interval	\(\log(F(\text{upper}) - F(\text{lower}))\)	interval()
missing	\(0\)	from_arrays(..., missing=mask)

Pass an Observations anywhere an observation array was accepted; the engine dispatches on the type.

Non-detects

import prophys as prp

obs = prp.Observations.detection_limit(concentrations, limit=0.5)
result = prp.finetune(compiled, {"c": obs}, n_steps=3000)

Values at or below the limit become left-censored at it; the rest stay exact. Compared with substituting the limit for every non-detect, this recovers the generating parameters rather than pulling the mean up and the scale down.

Survival and reliability

obs = prp.Observations.right_censored(recorded, censored=still_running)

Specimens still unfailed when the test ended are known only to exceed their recorded time. Ignoring the censoring treats every truncated value as a failure, which understates the lifetime and overstates the hazard rate.

Binned readings

obs = prp.Observations.interval(lower_edges, lower_edges + width)

Every observation known only to lie between two bounds — readings rounded or binned at recording time.

Missing entries

obs = prp.Observations.from_arrays(values, missing=np.isnan(values))

Marked records contribute nothing, which gives the same fit as omitting them while keeping the record’s length and row order intact. Values at those positions may be nan in the caller’s array; they are replaced by a finite placeholder, so a nan cannot reach a gradient through an unselected branch.

Downstream effects

An Observations record changes more than the fit:

• information_criteria() counts only informative observations, so missing records do not inflate the sample-size penalty;

• parameter_uncertainty() reports the same count;

• calibration() and crps() use only the exactly-observed values, since a censored record carries no PIT value;

• RunManifest records the counts by kind, so an exported result says how much of its data was censored.

read_observations() builds one straight from a file — see Reading Data.

class prophys.domain.Observations(value, lower, upper, kind)

A set of observations of one attribute, each with its own kind.

Build one with the classmethod matching how the data were recorded (exact(), detection_limit(), right_censored(), interval(), from_arrays()) rather than through __init__, which takes the already-resolved arrays.

Parameters

value:

Observed values. Ignored for non-exact kinds, but must still be finite: a nan here propagates through jnp.where into the gradient even on branches that discard it, so missing and censored entries carry a placeholder instead.

lower, upper:

Interval bounds, used by the censored kinds.

kind:

Per-observation code from the module constants.

classmethod exact(values)

Every value measured exactly — the same likelihood a plain array gives, in Observations form.

Parameters:

values (Any)

Return type:

Observations

classmethod detection_limit(values, limit)

Values at or below limit are left-censored at it; the rest are exact.

This is the standard non-detect record from an analytical laboratory: the instrument reports “< limit”, and all that is known is that the true value lies somewhere below.

Parameters:

• values (Any)

• limit (Any)

Return type:

Observations

classmethod right_censored(values, censored)

Entries where censored is true are known only to exceed their recorded value — a specimen still unfailed when the test ended.

Parameters:

• values (Any)

• censored (Any)

Return type:

Observations

classmethod interval(lower, upper)

Every observation known only to lie between lower and upper — readings rounded or binned at recording time.

Parameters:

• lower (Any)

• upper (Any)

Return type:

Observations

classmethod from_arrays(values, *, lower=None, upper=None, kind=None, missing=None)

The general constructor: per-observation kinds and bounds.

missing is a boolean mask; entries it marks are recoded as MISSING regardless of kind and contribute nothing. Values at those positions are replaced by a finite placeholder, so a record stored with nan at its gaps can be passed in directly.

Parameters:

• values (Any)

• lower (Any)

• upper (Any)

• kind (Any)

• missing (Any)

Return type:

Observations

log_prob(compiled, attribute_name, raw_params)

Per-observation log-likelihood contributions under compiled.

Every branch is evaluated for every observation and selected with jnp.where, rather than gathered per kind: the selection has to survive jax.jit and jax.grad, where a data-dependent Python branch would either retrace per record or bake one kind’s rule into the compiled function.

Parameters:

attribute_name (str)

Return type:

Any

counts()

Number of observations of each kind, for a fit’s provenance record and for summary().

Return type:

dict[str, int]

property n_informative: int

Observations that contribute a likelihood term — everything not marked missing. This is the count an information criterion’s sample size should use.

Parameters:

• value (Any)

• lower (Any)

• upper (Any)

• kind (Any)

prophys.domain.as_observations(obs)

Coerce a plain array into fully-observed Observations.

Lets code that needs the observation model (an information criterion’s sample size, a provenance record) treat both input forms uniformly.

Parameters:

obs (Any)

Return type:

Observations