Prior Fitted Network

InContextRegressor is a transformer trained once on synthetic regression tasks drawn from the mechanism families of prophys models. Given a few dozen to a few hundred observed rows of an unknown mechanism it returns posterior predictive quantiles at new inputs in one forward pass, with no fitting (a prior fitted network, Muller et al. 2022, Hollmann et al. 2025).

Thirty noisy observations of a saturating dose response. The shipped network returns the median and a 90 percent predictive band directly.

The prior

Every training task draws inputs from uniform, normal or log normal distributions, a random subset of relevant inputs, and a mechanism from one of seven families that recur in industrial risk models:

Family	Shape
logistic	Dose response and fragility curves.
power	Hazard and damage power laws of a softplus index.
decay	Exponential decay with distance plus a linear trend.
piecewise	Piecewise linear responses with random knots.
seasonal	Periodic cycles with a trend.
smooth	Random smooth functions from random Fourier features.
interaction	Linear responses with a product interaction.

Observations carry Gaussian, heteroscedastic, heavy tailed (Student t with three degrees of freedom) or count (Poisson) noise, so the network learns calibrated predictive distributions for all of them.

Using it

from prophys import nn

pfn = nn.InContextRegressor.pretrained()
pfn.fit(doses, responses)                # stores the context, no training
q = pfn.predict_quantiles(new_doses)     # [n, 9] at pfn.levels
median = pfn.predict(new_doses)

InContextRegressor(head="bar") trains a bar-distribution head instead; see Probabilistic Training.

Inputs and targets are standardised with the context statistics. Contexts larger than the network’s context size are subsampled and smaller ones are repeated. Context rows attend to each other and query rows attend only to the context (PrefixAttention), so predictions for different queries are independent.

The shipped network takes up to four inputs and a context of 64 rows. On held out prior tasks its pinball loss is 35 percent below that of the context quantiles (0.133 against 0.204).

Training your own

from prophys import nn

pfn = nn.InContextRegressor(features=4, context=64, queries=32, width=64, depth=3)
nn.train_pfn(pfn, steps=6000, tasks=16, learning_rate=1e-3, log_every=250)
pfn.save("my_pfn.npz")

pfn = nn.InContextRegressor.load("my_pfn.npz")

train_pfn() draws fresh tasks at every step and trains with the pinball loss on the query rows. sample_prior_task exposes the prior for inspection or extension. Prefix attention is a licensed feature, so the network needs a license to train and to run.