virgil.svi
variational fits a guide (an approximate posterior) to the posterior of
numpyro_model with numpyro's SVI, taking the same
arguments as fit and starting from a fit and its
Gauss–Newton covariance. See Variational inference.
Variational inference with numpyro's SVI, started from a fit.
Provisional
May change in any 0.4.x release without a deprecation cycle.
variational takes the same arguments as
fit and
numpyro_model, and fits a guide (an
approximate posterior) to the posterior by maximizing the evidence lower
bound (ELBO) with numpyro's SVI. The guide works in the unconstrained
coordinates NUTS samples, and starts at the Laplace approximation: its
centre at a FitResult and its width from
gauss_newton_mass. The default guide
is a normalizing flow that bends that Gaussian to follow curved ridges and
skewed scales; a Gaussian guide is close to the Laplace approximation, for
which see laplace_cov. Every guide covers
one mode.
See Variational inference for the choice of guide and how to check the result. References: ADVI (Kucukelbir et al. 2017, https://arxiv.org/abs/1603.00788), the block neural autoregressive flow (De Cao, Titov & Aziz 2019, https://arxiv.org/abs/1904.04676), the inverse autoregressive flow (Kingma et al. 2016, https://arxiv.org/abs/1606.04934) and Pareto-smoothed importance sampling (Vehtari et al. 2024, https://arxiv.org/abs/1507.02646). The related geometric variational inference (Frank, Leike & Enßlin 2021, https://arxiv.org/abs/2105.10470) fits a Gaussian in nonlinearly transformed coordinates; it is not implemented here.
VariationalResult
dataclass
The result of variational.
Attributes:
| Name | Type | Description |
|---|---|---|
samples |
dict
|
Draws from the guide, |
losses |
ndarray
|
The loss (the negative ELBO, a Monte Carlo estimate) at each step. |
guide |
AutoGuide
|
The fitted guide; |
params |
dict
|
The guide's fitted parameters. |
converged |
bool
|
Whether the loss stopped falling: the mean loss over the last |
info |
dict
|
|
samples
instance-attribute
losses
instance-attribute
guide
instance-attribute
params
instance-attribute
converged
instance-attribute
info
instance-attribute
__init__(samples, losses, guide, params, converged, info)
variational(model, priors, data, regularizers=(), *, noise=None, likelihoods=(), start=None, guide='bnaf', steps=3000, optimizer=None, learning_rate=0.003, num_particles=8, num_samples=2000, key=None, flat_coordinates=False, dense_start=True, init_scale=0.1, window=None, psis=True, dtype='float64', **options)
Approximate the posterior with a guide fitted by numpyro's SVI.
Maximizes the ELBO with Adam on the posterior of
numpyro_model. The guide starts at
start with the Gauss–Newton width, or width init_scale given
regularizers, dense_start=False or start=None (then at the
priors' medians). Guide choice and checks:
Variational inference.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model
|
Scene, priors and data as for
|
required | |
priors
|
Scene, priors and data as for
|
required | |
data
|
Scene, priors and data as for
|
required | |
regularizers
|
sequence
|
Regularizers; see Common arguments. |
()
|
noise
|
dict
|
Noise priors; see Common arguments. |
None
|
likelihoods
|
sequence
|
Extra likelihoods; see Common arguments. |
()
|
start
|
FitResult or dict
|
Where the guide starts: a fit or its |
None
|
guide
|
str or callable
|
|
'bnaf'
|
steps
|
int
|
Steps (3000). |
3000
|
optimizer
|
optax optimizer
|
Default |
None
|
learning_rate
|
float
|
The optimizer's step size (3e-3). |
0.003
|
num_particles
|
int
|
Draws per ELBO estimate (8). |
8
|
num_samples
|
int
|
Draws returned (2000). |
2000
|
key
|
PRNGKey
|
Default |
None
|
flat_coordinates
|
bool
|
As for |
False
|
dense_start
|
bool
|
Start the width at the Gauss–Newton covariance (default |
True
|
init_scale
|
float
|
The width (default 0.1) of the coordinates the Gauss–Newton start does not cover, or of all of them without it. |
0.1
|
window
|
int
|
Convergence-test window (default |
None
|
psis
|
bool
|
Whether to compute the Pareto k̂. |
True
|
dtype
|
(float64, float32)
|
The precision (default |
"float64"
|
**options
|
Fixed error terms and |
{}
|
Returns:
| Type | Description |
|---|---|
VariationalResult
|
|
Examples:
>>> vi = variational(scene, priors, data, start=result)