Skip to content

virgil.mode_search

Mode search results turned into an evidence and posterior draws: a defensive mixture of Student-t components at the modes of a multimodal target (and the images of its exact symmetries), importance sampling for ln Z, a sentinel for a missed dominant mode, and an HMC resample-move for posterior draws. The position-level fit of virgil.orbit_cells uses it; any target with a known list of modes can.

Mode search, defensive importance sampling and HMC resample-move.

Provisional

May change in any 0.4.x release without a deprecation cycle.

For a target that is a needle in its prior (a posterior mass of order e⁻⁴⁰ of the prior), tempering from the prior fails. Here a search finds the modes of the target; DefensiveMixture puts a Student-t on each (with the images of exact symmetries of the target) beside a share of the prior; importance_evidence estimates ln Z with an error from the effective sample size and flags a search that missed the dominant mode; resample_move turns the weighted draws into posterior draws. The target is a function log_target(x, theta) of a flat vector x and a pytree theta of traced parameters, so one compilation serves every cell of a system.

Source: Hesterberg (1995), Hoffman & Gelman (2014), BlackJAX (Cabezas et al. 2024), Gilks & Berzuini (2001), Neal (2011). The sentinel and the symmetric distinct-mode rule are not from these: they are provisional.

Mode dataclass

A mode of the label-summed target.

cov is the inverse of minus the Hessian of the search objective at x; log_target is the log of the objective the search maximized, at x (the log target for a target whose Laplace ln Z needs no correction).

x instance-attribute

log_target instance-attribute

cov instance-attribute

n_inliers instance-attribute

__init__(x, log_target, cov, n_inliers)

Evidence dataclass

Importance-sampling estimate of ln Z from a defensive mixture.

log_z_error is the delta-method √(1/ESS - 1/n), a first-order lower bound (it misses heavy tails and mass the proposal missed; ln Ẑ also carries a bias of -1/(2 ESS)). at_floor is the sentinel: most of the importance weight sits on draws with fewer than m_min inliers (the all-outlier floor) although a mode with at least m_min inliers has a Laplace ln Z W nats above it; log_z must then not be reported. undersampled is ess < ESS_TARGET (information only). n_inliers is the weighted mean of inliers_fn; log_q is the proposal's log density at the draws (None if unset).

log_z instance-attribute

log_z_error instance-attribute

ess instance-attribute

x instance-attribute

log_w instance-attribute

at_floor instance-attribute

undersampled instance-attribute

n_inliers instance-attribute

n_nonfinite instance-attribute

log_q = None class-attribute instance-attribute

__init__(log_z, log_z_error, ess, x, log_w, at_floor, undersampled, n_inliers, n_nonfinite, log_q=None)

DefensiveMixture

q = α prior + (1 - α) Σ_m w_m t_DF(g x_m, g cov_m gᵀ).

w_m is proportional to the Laplace evidence of mode m, split equally over the len(images) + 1 components of its images g (exact linear symmetries of the target; the identity is implicit). The component arrays are padded to a power of two (at least M_PAD) with log-weight -inf, so the shapes (and a compiled log_q) do not depend on the number of modes. With no modes q is the prior (α = 1).

Parameters:

Name Type Description Default
modes list of Mode

Distinct modes.

required
log_prior callable

log_prior(x), a function with a stable identity (it is static under jit).

required
sample_prior callable

sample_prior(key, n) draws (n, d) from the prior.

required
images sequence of (d, d) arrays

Non-identity symmetry maps.

()
dim int

Dimension of x; fixes the padded shapes of an empty (prior-only) mixture so that it shares the compiled kernels of the other cells.

None

modes = list(modes) instance-attribute

d = d if dim is None else dim instance-attribute

log_w = np.asarray(logw + [-onp.inf] * pad, dt) instance-attribute

mu = np.asarray(mu, dt) instance-attribute

chol = np.asarray(chol, dt) instance-attribute

linv = np.asarray(onp.linalg.inv(chol), dt) instance-attribute

logdet = np.asarray(2.0 * onp.log(onp.diagonal(chol, axis1=1, axis2=2)).sum(-1), dt) instance-attribute

log_alpha = np.asarray(math.log(alpha), dt) instance-attribute

log_beta = np.asarray(math.log1p(-alpha) if alpha < 1.0 else -onp.inf, dt) instance-attribute

__init__(modes, log_prior, sample_prior, images=(), alpha=ALPHA, df=DF, dtype=None, dim=None)

tree_flatten()

tree_unflatten(aux, children) classmethod

log_q(x)

ln q(x) of one vector.

sample(key, n)

n exact draws from q.

distinct_modes(modes)

The modes, best first, dropping any within MAHAL of a kept one.

A mode is the same as a kept one when its squared Mahalanobis distance is below MAHAL under both covariances: a broad partial-outlier mode never hides a needle (the one-sided rule lost 62 nats).

laplace_log_evidence(mode)

Laplace ln Z of a mode: log_target + d/2 ln 2π + ½ ln |cov|.

importance_evidence(log_target, theta, mixture, key, n=N_IS, *, inliers_fn, log_floor, m_min=M_MIN)

ln Z of log_target by importance sampling from mixture.

The sentinel at_floor fires when more than half of the importance weight is on draws with fewer than m_min inliers (inliers_fn rounded to an integer; the weighted mean would fire on a healthy mode of exactly m_min inliers as soon as one draw loses one) while a mode with at least m_min inliers has a Laplace ln Z above log_floor + W; log_floor is the ln Z of the all-outlier target (a mode's peak height says nothing of its mass: a ridge fitted by m_min nights is high and thin).

Returns:

Type Description
Evidence

The weighted draws are exact posterior draws once resampled.

resample_move(log_target, theta, ev, key, n_draws=N_POST, n_moves=N_HMC, cov=None)

The posterior draws of ev after resampling and moving them.

Systematic resampling of ev.x by ev.log_w, then n_moves BlackJAX HMC transitions per draw: step size by dual averaging to ACCEPT over N_WARM warm-up transitions on a pilot of the resampled set, jittered by ±JITTER, trajectory ⌈π/(2ε)⌉ capped at L_MAX, mass matrix cov (the dominant mode's, else the weighted sample covariance). The weighted draws are already exact: the moves only rejuvenate duplicates.

Source: Gilks & Berzuini (2001), Hoffman & Gelman (2014), Neal (2011).

logmeanexp(a)

ln mean exp a (-inf if every entry is).