virgil.fitting
fit finds maximum a posteriori parameters, taking the same arguments as
numpyro_model, plus optional regularizers. gauss_newton_mass turns a fit into a
mass matrix for numpyro's NUTS, for sampling any posterior with one peak:
images, and orbits and binaries with fitted error scales and gains.
gauss_newton_diag gives the per-coordinate scales that
fit(..., precondition="diag") divides by, which conditions a fit of an image or other badly scaled
parameters. kkt_report certifies convergence of an image fit by its KKT conditions, not a loss plateau. Angles with an AngleVector prior are fitted as 2-D vectors,
with no wrap boundary at 0°/360°.
Maximum a posteriori fits of models, including images.
fit takes the same arguments as
numpyro_model: a model (a template,
or a function of the parameters), a dict of numpyro priors whose keys are the
free parameters, and the data, plus optional regularizers (see
virgil.imaging). It finds the maximum a
posteriori parameters with Levenberg–Marquardt, L-BFGS or Adam, optimizing
each parameter in unconstrained coordinates through the bijection to its
prior's support, in float64 by default. A parameter whose prior is uniform in
some coordinate (a log-uniform scale, an isotropic inclination) is fitted in
that flat coordinate, where its prior adds nothing to the loss, so that
Levenberg–Marquardt works with the Jeffreys priors. To sample the same
posterior, pass the same arguments to numpyro_model; it samples in numpyro's
coordinates unless told flat_coordinates=True.
FitResult
dataclass
The result of fit.
Attributes:
| Name | Type | Description |
|---|---|---|
model |
SourceModel or list
|
The fitted model, or models (one per dataset) if the model function returned a list. |
values |
dict
|
The fitted parameter values, keyed by path, in the model's own
parameters (not the flat coordinates some are fitted in). An angle with an
|
info |
dict
|
|
model
instance-attribute
values
instance-attribute
info
instance-attribute
__init__(model, values, info)
GaussNewtonMass
Bases: dict
NUTS keyword arguments from gauss_newton_mass.
A dict that unpacks into NUTS(...) as usual, and records the
flat_coordinates of its matrix as an attribute for
chain_init_params. condition
is the condition number of the matrix's correlation form (None for
a diagonal mass).
flat_coordinates = False
class-attribute
instance-attribute
condition = None
class-attribute
instance-attribute
NoLeastSquaresForm
Bases: TypeError
The objective has no residual vector (use L-BFGS or Adam).
A TypeError for backward compatibility; fit picks L-BFGS on it
and variational falls back to a diagonal start, while other errors
propagate.
NoStartingValue
Bases: ValueError
A free parameter has no starting value (a function model needs one).
SingularCurvature
Bases: ValueError
The Gauss–Newton curvature is singular at the given values: some
direction is constrained by neither the data nor a proper prior (an
ImproperUniform or flat-in-coordinate parameter the data do not
fix), or the matrix is not positive definite in the working precision.
fit(model, priors, data, regularizers=(), *, noise=None, init=None, method=None, max_steps=None, gtol=0.0001, max_step_size=2.0, lbfgs_memory=None, learning_rate=0.01, cg_steps=50, dtype='float64', likelihoods=(), time_limit=None, progress=None, precondition=None, adam_eps=1e-08, ftol_abs=None, ftol_window=100, post_step=None)
Find the maximum a posteriori parameters of a model given data.
Arguments are shared with numpyro_model
(common arguments). Parameters under a prior uniform
in some coordinate are optimized in it; values are in the model's own.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model
|
SourceModel or callable
|
A template whose leaves at the paths in |
required |
priors
|
dict[str, Distribution]
|
A prior for each free parameter, keyed by its path. |
required |
data
|
OIData or sequence of OIData
|
The data, fitted jointly; |
required |
regularizers
|
sequence
|
Loss penalties, e.g. |
()
|
noise
|
dict
|
Priors on error terms; see common arguments. |
None
|
init
|
dict
|
Starting values by path (required for a function model). |
None
|
method
|
(lm, lbfgs, adam, lbfgsb)
|
|
"lm"
|
max_steps
|
int
|
Step limit (1000 LM, 20000 L-BFGS and L-BFGS-B, 2000 Adam); see fit stopping. |
None
|
gtol
|
float
|
Gradient tolerance; see fit stopping. |
0.0001
|
max_step_size
|
float
|
L-BFGS step cap; see fit stopping. |
2.0
|
lbfgs_memory
|
int
|
L-BFGS history (100, cut to keep it under 1 GiB; 10, uncut, for L-BFGS-B). |
None
|
cg_steps
|
int
|
CG steps per LM step (above 200 coordinates). |
50
|
learning_rate
|
float or callable
|
Adam's rate: a float or an optax schedule (a callable of the step). |
0.01
|
dtype
|
(float64, float32)
|
Precision; see common arguments. |
"float64"
|
likelihoods
|
sequence
|
Further terms, e.g. |
()
|
time_limit
|
float
|
Wall-clock budget (s). One that runs out on Adam's or LM's last chunk
reports |
None
|
progress
|
callable or True
|
Callback (or |
None
|
adam_eps
|
float
|
Adam only: |
1e-08
|
ftol_abs
|
float
|
Adam only: a plateau stop after |
None
|
ftol_window
|
int
|
Adam only: the window of |
100
|
post_step
|
callable
|
Adam only, not with |
None
|
precondition
|
(None, diag)
|
|
None
|
Returns:
| Type | Description |
|---|---|
FitResult
|
Model, values and diagnostics; warns if LM, L-BFGS or Adam did not converge
( |
gauss_newton_mass(model, priors, data, values, *, likelihoods=(), noise=None, flat_coordinates=False)
A NUTS mass matrix from the Gauss–Newton curvature at a fit.
Near the maximum a posteriori the posterior is close to a Gaussian with
precision F = JᵀJ + diag(P) in numpyro's unconstrained coordinates; its
inverse is the mass matrix. See the Gauss–Newton mass matrix
for the construction and when not to use it.
Use it at fixed field hyperparameters, with every sampled parameter in
priors. Above 4096 coordinates the mass is diagonal, with a UserWarning.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model
|
As for |
required | |
priors
|
As for |
required | |
data
|
As for |
required | |
likelihoods
|
As for |
required | |
noise
|
As for |
required | |
values
|
dict
|
Where to take the curvature, normally |
required |
flat_coordinates
|
bool
|
Pass the value given to |
False
|
Returns:
| Type | Description |
|---|---|
GaussNewtonMass
|
Keyword arguments for |
Raises:
| Type | Description |
|---|---|
SingularCurvature
|
If a direction is fixed by neither the data nor a proper prior, or the matrix is not positive definite. |
Examples:
>>> result = fit(scene, priors, data)
>>> kernel = NUTS(numpyro_model(result.model, priors, data),
... init_strategy=init_to_value(values=result.values),
... **gauss_newton_mass(scene, priors, data, result.values))
gauss_newton_diag(model, priors, data, values, *, regularizers=(), likelihoods=(), noise=None, n_probes=None, dtype='float64')
The diagonal of the Gauss–Newton curvature of a fit's loss.
Provisional
May change in any 0.4.x release without a deprecation cycle.
The curvature is F = JᵀJ + diag(P) + diag(R) in the unconstrained
coordinates fit optimizes: J is the Jacobian of the
whitened residuals, P the curvature of the priors and R the Hessian diagonal
of the regularizers, floored at zero. On the pixels of a regularizer with
metric_rows (maximum entropy) it is that metric alone, weight * b *
(1 - b) (the softmax's, unsmoothed pixels). The data JᵀJ and other terms
are dropped there, so it is a scale, not the curvature, at weak weights. It is the scale
fit(..., precondition="diag") divides each coordinate by; see
gauss_newton_mass for the dense matrix.
Source: probes, Bekas et al. (2007) and Hutchinson (1989); curvature, Skilling & Bryan (1984), Cornwell & Evans (1985), Martens (2010), Schraudolph (2002); small exact blocks, h-greer/hubble.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model
|
As for |
required | |
priors
|
As for |
required | |
data
|
As for |
required | |
regularizers
|
As for |
required | |
noise
|
As for |
required | |
likelihoods
|
As for |
required | |
dtype
|
As for |
required | |
values
|
dict
|
Parameter values at which to take the curvature (e.g. |
required |
n_probes
|
int
|
Up to 1024 coordinates the diagonal is exact, one Jacobian-vector
product per coordinate. Above, it is estimated: JᵀJ from the mean of
|
None
|
Returns:
| Type | Description |
|---|---|
dict
|
The diagonal by coordinate name, with each coordinate's shape. |
kkt_report(result, priors, data, regularizers=(), *, noise=None, likelihoods=(), start=None, weight=None)
Convergence certificates of a regularized image fit, not a loss plateau.
Provisional
May change in any 0.4.x release without a deprecation cycle.
Call with a FitResult and the priors,
data, regularizers, noise and likelihoods it was fitted
with. start is the starting values (fit's init): the
gradient scales and pg_rel are then relative to those at the start as
well, which a fit with no penalty needs (its gradient is zero at the
optimum). Without it they use the result alone: the larger of the data and
penalty gradients.
"images" holds, by path:
Image(softmax):kkt_w, the rms over pixels above 1e-3 of the peak ofg_I - μ, the gradient per pixel flux less the multiplier, in units ofw / F(regulariser weight over flux), which gatesconvergedatKKT_W_TOL.kkt_rel, the rms ofg - μover pixels above 1e-3 of the peak, withg = ∂J/∂bfor unit-sumbandμ = Σ b g, over|μ|plus the rms gradient scale;want_grow, the fraction of pixels below 1e-6 of the peak whose objective would raise them.LinearImage"sqrt": the same withg = ∂J/∂fluxandμ = 0. A pixel exactly zero has zero gradient and is not seen to want growth.LinearImage"linear":kkt_w, the rms projected gradient over pixels above 1e-3 of the peak or at a bound withg < 0, in units ofw / F0withF0the pixel sum;pg_rel, the ∞-norm of the projected gradient over its reference; andzero_violation, the fraction of zero pixels withg < 0. Active bounds are not violations.- A lone
MaxEntropy:band_limit, the windowed power ofln bbeyond the data's largest spatial frequency over that of its hard low-pass; about 1 at an optimum, a larger value meaning structure the data cannot constrain.
"nuisances" holds the projected-gradient ∞-norm of every other
parameter in its fitted coordinate, and grad_rel relative to the start
gradient. "converged" is true when every image has kkt_w below
KKT_W_TOL (kkt_rel or pg_rel below KKT_TOL if there is no
weight), want_grow and zero_violation zero, and band_limit
below BAND_TOL; and, given start, every nuisance grad_rel is
below NUISANCE_TOL. Only a plain fitted or fixed image flux is
supported; a tied or derived one raises NotImplementedError.
Source: Nocedal & Wright (2006), ch. 12 and §16.7.