virgil.mode_search
Mode search results turned into an evidence and posterior draws: a defensive
mixture of Student-t components at the modes of a multimodal target (and the
images of its exact symmetries), importance sampling for ln Z, a sentinel
for a missed dominant mode, and an HMC resample-move for posterior draws. The
position-level fit of virgil.orbit_cells uses it; any
target with a known list of modes can.
Mode search, defensive importance sampling and HMC resample-move.
Provisional
May change in any 0.4.x release without a deprecation cycle.
For a target that is a needle in its prior (a posterior mass of order
e⁻⁴⁰ of the prior), tempering from the prior fails. Here a search finds
the modes of the target; DefensiveMixture
puts a Student-t on each (with the images of exact symmetries of the target)
beside a share of the prior; importance_evidence
estimates ln Z with an error from the effective sample size and flags a
search that missed the dominant mode; resample_move
turns the weighted draws into posterior draws. The target is a function
log_target(x, theta) of a flat vector x and a pytree theta of traced
parameters, so one compilation serves every cell of a system.
Source: Hesterberg (1995), Hoffman & Gelman (2014), BlackJAX (Cabezas et al. 2024), Gilks & Berzuini (2001), Neal (2011). The sentinel and the symmetric distinct-mode rule are not from these: they are provisional.
Mode
dataclass
A mode of the label-summed target.
cov is the inverse of minus the Hessian of the search objective at
x; log_target is the log of the objective the search maximized, at x
(the log target for a target whose Laplace ln Z needs no correction).
x
instance-attribute
log_target
instance-attribute
cov
instance-attribute
n_inliers
instance-attribute
__init__(x, log_target, cov, n_inliers)
Evidence
dataclass
Importance-sampling estimate of ln Z from a defensive mixture.
log_z_error is the delta-method √(1/ESS - 1/n), a first-order
lower bound (it misses heavy tails and mass the proposal missed; ln Ẑ
also carries a bias of -1/(2 ESS)). at_floor
is the sentinel: most of the importance weight sits on draws with fewer
than m_min inliers (the all-outlier floor) although a mode with at
least m_min inliers has a Laplace ln Z W nats above it;
log_z must then not be reported. undersampled is ess < ESS_TARGET
(information only). n_inliers is the weighted mean of inliers_fn;
log_q is the proposal's log density at the draws (None if unset).
log_z
instance-attribute
log_z_error
instance-attribute
ess
instance-attribute
x
instance-attribute
log_w
instance-attribute
at_floor
instance-attribute
undersampled
instance-attribute
n_inliers
instance-attribute
n_nonfinite
instance-attribute
log_q = None
class-attribute
instance-attribute
__init__(log_z, log_z_error, ess, x, log_w, at_floor, undersampled, n_inliers, n_nonfinite, log_q=None)
DefensiveMixture
q = α prior + (1 - α) Σ_m w_m t_DF(g x_m, g cov_m gᵀ).
w_m is proportional to the Laplace evidence of mode m, split
equally over the len(images) + 1 components of its images g (exact
linear symmetries of the target; the identity is implicit). The component
arrays are padded to a power of two (at least M_PAD) with log-weight
-inf, so the shapes (and a compiled log_q) do not depend on the
number of modes. With no modes q is the prior (α = 1).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
modes
|
list of Mode
|
Distinct modes. |
required |
log_prior
|
callable
|
|
required |
sample_prior
|
callable
|
|
required |
images
|
sequence of (d, d) arrays
|
Non-identity symmetry maps. |
()
|
dim
|
int
|
Dimension of |
None
|
modes = list(modes)
instance-attribute
d = d if dim is None else dim
instance-attribute
log_w = np.asarray(logw + [-onp.inf] * pad, dt)
instance-attribute
mu = np.asarray(mu, dt)
instance-attribute
chol = np.asarray(chol, dt)
instance-attribute
linv = np.asarray(onp.linalg.inv(chol), dt)
instance-attribute
logdet = np.asarray(2.0 * onp.log(onp.diagonal(chol, axis1=1, axis2=2)).sum(-1), dt)
instance-attribute
log_alpha = np.asarray(math.log(alpha), dt)
instance-attribute
log_beta = np.asarray(math.log1p(-alpha) if alpha < 1.0 else -onp.inf, dt)
instance-attribute
__init__(modes, log_prior, sample_prior, images=(), alpha=ALPHA, df=DF, dtype=None, dim=None)
tree_flatten()
tree_unflatten(aux, children)
classmethod
log_q(x)
ln q(x) of one vector.
sample(key, n)
n exact draws from q.
distinct_modes(modes)
The modes, best first, dropping any within MAHAL of a kept one.
A mode is the same as a kept one when its squared Mahalanobis distance is
below MAHAL under both covariances: a broad partial-outlier mode
never hides a needle (the one-sided rule lost 62 nats).
laplace_log_evidence(mode)
Laplace ln Z of a mode: log_target + d/2 ln 2π + ½ ln |cov|.
importance_evidence(log_target, theta, mixture, key, n=N_IS, *, inliers_fn, log_floor, m_min=M_MIN)
ln Z of log_target by importance sampling from mixture.
The sentinel at_floor fires when more than half of the importance
weight is on draws with fewer than m_min inliers (inliers_fn
rounded to an integer; the weighted mean would fire on a healthy mode of
exactly m_min inliers as soon as one draw loses one) while a mode
with at least m_min inliers has a Laplace ln Z above
log_floor + W;
log_floor is the ln Z of the all-outlier target (a mode's peak
height says nothing of its mass: a ridge fitted by m_min nights is
high and thin).
Returns:
| Type | Description |
|---|---|
Evidence
|
The weighted draws are exact posterior draws once resampled. |
resample_move(log_target, theta, ev, key, n_draws=N_POST, n_moves=N_HMC, cov=None)
The posterior draws of ev after resampling and moving them.
Systematic resampling of ev.x by ev.log_w, then n_moves BlackJAX
HMC transitions per draw: step size by dual averaging to ACCEPT over
N_WARM warm-up transitions on a pilot of the resampled set, jittered by
±JITTER, trajectory ⌈π/(2ε)⌉ capped at L_MAX, mass matrix
cov (the dominant mode's, else the weighted sample covariance). The
weighted draws are already exact: the moves only rejuvenate duplicates.
Source: Gilks & Berzuini (2001), Hoffman & Gelman (2014), Neal (2011).
logmeanexp(a)
ln mean exp a (-inf if every entry is).