Skip to content

virgil.ensemble

Ensembles of randomized image reconstructions in the manner of PYRA and MYTHRA (Drevon et al. 2025): draw regularizer families, weights, pixel sizes, fields and starting images; fit each group as an L-curve; select the members that fit the data and average them into a mean image with a per-pixel spread. See the tutorial Imaging, part 7.

Ensembles of randomized image reconstructions, averaged into one image.

A single regularized reconstruction depends on choices the data do not fix: the regularizer and its weight, the pixel size, the field and the starting image. Drevon et al. (2025, arXiv:2609.15365), who won the 2024 interferometric imaging contest, run many reconstructions with these choices drawn at random (their PYRA), keep those that fit the data, and average them while the average still fits (their MYTHRA). The mean is less sensitive to any one choice than a single reconstruction, and the spread of the members is a map of how much the image depends on those choices. This module does the same with virgil's own fits; it is written from the paper's description, not from their code.

The work is split so that a cluster can run it in parallel:

  1. draw_groups draws the reconstruction settings. Each group has one geometry (pixel size and number of pixels), one regularizer family, one starting image and several weights.
  2. run_group fits one group, as an l_curve over its weights. The weights are traced, so a group compiles once, and groups that share a geometry and a family share the compilation.
  3. combine selects the members and averages them into an Ensemble.

ensemble runs all three in turn. On a cluster, run one group per array task (run_group(data, draw_groups(data, n, key, spec)[task])), save the Groups and combine them in one more job.

The ensemble's standard deviation is not a posterior uncertainty. It measures how much the image changes between reasonable reconstruction choices, not the noise in the data; for the posterior, sample an image (see the tutorial "Imaging, part 5").

FAMILIES = {'tv': TV, 'tsv': TSV, 'maxent': MaxEntropy, 'starlet': StarletL1} module-attribute

EnsembleSpec dataclass

What an ensemble draws at random, and how it selects members.

Attributes:

Name Type Description
families tuple of str

Regularizer families to draw from, uniformly: "tv" (TV), "tsv" (TSV), "maxent" (MaxEntropy) and "starlet" (StarletL1).

weight_ranges dict

For each family, the range (low, high) of the weight per data point (the weight divided by the total number of independent data). Weights are drawn log-uniformly in it, the invariant prior for a scale, one in each of n_weights equal bins of log w so that every sweep spans the range. The defaults are wide enough to hold the corners of the contest bench's datasets with a window's width to spare below them; if the L-curve corners fall at the edges, move the range.

n_weights int

Weights per group, at least three (the L-curve's corner needs them).

oversample tuple of float

Pixels per Nyquist pixel (nyquist_pixel_scale), drawn uniformly.

field_factors tuple of float

The field is field_of_view(data) times one of these, drawn uniformly. A field larger than the interferometric field of view lets the image hold flux the shortest baselines resolve out, which on the contest bench gave better images than a field of 1 or less (field 4 was the best of its arms). On a uv lattice (AMI, data with a uv_grid) a field above 1 aliases, so there the factors are capped at 1.

starts tuple of str

Starting images, drawn uniformly: "moments" and "dirty" (see starting_image; a dirty start needs data with phases) and "flat".

max_npix int

Largest number of pixels on a side. A field that would need more keeps its size and takes coarser pixels instead, so its pixels per Nyquist pixel fall below oversample, but never below one: if even Nyquist pixels cannot cover it, the field is cropped.

window_dex float

Width of the window of weights kept in each group, in dex, from corner / 10**window_dex up to the L-curve's corner: as in MYTHRA, the weights just before the turnover, where the fit to the data has stopped improving but the image is not yet over-smoothed. Stronger weights, past the corner, trade fit for smoothness fast.

max_chi2_red float

Drop a member if the raw χ² per data point of any dataset exceeds this.

chi2_ratio float

Drop a member if, on any dataset, its χ² per data point exceeds this multiple of the best member's on that dataset. With miscalibrated errors no member reaches χ²/N ≈ 1, so a relative threshold is the one that bites.

mad_cut float

Then drop members whose total χ² per data point lies more than this many robust standard deviations (1.4826 times the median absolute deviation) above the median.

max_shift_mas float or None

Without a star, the members are recentred on the best one, searching shifts up to this (default: the beam's major axis). With a star, the star fixes the position and they are not shifted.

mean_rtol float or None

A member joins the mean if, on every dataset, the mean's χ² stays within this fraction of the best member's. The default, None, is each dataset's χ²/N noise, √(2/N): a mean that fits as well as the best member, up to the noise. 0 keeps the mean's χ² at or below the best member's, which on data the best member fits to the noise often keeps that member alone (and the spread is then zero); on the contest bench it kept 1 of 72 members on most datasets.

min_kept int

combine warns if fewer members than this are kept.

families = ('tv', 'tsv', 'maxent', 'starlet') class-attribute instance-attribute

weight_ranges = dataclasses.field(default_factory=_default_weight_ranges) class-attribute instance-attribute

n_weights = 8 class-attribute instance-attribute

oversample = (2.0, 3.0, 4.0) class-attribute instance-attribute

field_factors = (1.0, 2.0, 4.0) class-attribute instance-attribute

starts = ('moments', 'flat') class-attribute instance-attribute

max_npix = 128 class-attribute instance-attribute

window_dex = 1.0 class-attribute instance-attribute

max_chi2_red = onp.inf class-attribute instance-attribute

chi2_ratio = 2.0 class-attribute instance-attribute

mad_cut = 5.0 class-attribute instance-attribute

max_shift_mas = None class-attribute instance-attribute

mean_rtol = None class-attribute instance-attribute

min_kept = 3 class-attribute instance-attribute

__init__(families=('tv', 'tsv', 'maxent', 'starlet'), weight_ranges=_default_weight_ranges(), n_weights=8, oversample=(2.0, 3.0, 4.0), field_factors=(1.0, 2.0, 4.0), starts=('moments', 'flat'), max_npix=128, window_dex=1.0, max_chi2_red=onp.inf, chi2_ratio=2.0, mad_cut=5.0, max_shift_mas=None, mean_rtol=None, min_kept=3)

__post_init__()

Draw dataclass

The settings of one group of reconstructions.

Attributes:

Name Type Description
index int

Position of the group in the ensemble.

family str

The regularizer family.

npix int

Pixels on a side.

pixel_scale_mas float

Pixel size in mas.

start str

The starting image.

weights tuple of float

The regularizer weights, largest first.

index instance-attribute

family instance-attribute

npix instance-attribute

pixel_scale_mas instance-attribute

start instance-attribute

weights instance-attribute

geometry property

(family, npix, pixel_scale_mas): groups sharing it share a compilation.

__init__(index, family, npix, pixel_scale_mas, start, weights)

Group dataclass

The result of run_group.

Attributes:

Name Type Description
draw Draw

The group's settings.

curve LCurve

Its fits, one per weight, largest weight first.

draw instance-attribute

curve instance-attribute

__init__(draw, curve)

Member dataclass

One reconstruction in an ensemble.

Attributes:

Name Type Description
draw Draw

The settings of its group.

weight float

Its regularizer weight.

result FitResult

The fit.

chi2_red tuple of float

Raw χ² per data point of each dataset (with the quoted errors).

kept bool

Whether it is in the mean image.

reason str or None

Why it was left out: "window" (outside its group's L-curve window), "diverged" (a non-finite χ²: the fit failed), "chi2" (a dataset fitted too badly), "outlier" (total χ² far above the others) or "mean" (adding it made the mean fit a dataset worse). None if kept.

draw instance-attribute

weight instance-attribute

result instance-attribute

chi2_red instance-attribute

kept = False class-attribute instance-attribute

reason = None class-attribute instance-attribute

total_chi2_red property

χ² per data point over all datasets.

__init__(draw, weight, result, chi2_red, kept=False, reason=None)

Ensemble dataclass

The result of combine.

Attributes:

Name Type Description
model System

The mean scene, exactly: the kept members' images on their own grids (member0, member1, ...), each carrying its share of the flux, and with a star a star component. chi2_red and trace are of this model.

mean Image

The mean extended emission, on the common grid (the finest pixels and the largest field of the kept members). Its brightness sums to one; with a star its flux is the mean flux relative to the star. Resampling smooths the members' images a little, so this image is for display and scoring; it fits the data less well than model.

std (array, shape(npix, npix))

Standard deviation across the kept members of the same quantity as mean.brightness, pixel by pixel.

chi2_red tuple of float

Raw χ² per data point of the mean scene on each dataset.

trace list of tuple

chi2_red of the running mean after each accepted member, from the best member alone; on every dataset it stays within spec.mean_rtol of the first entry.

members list of Member

Every reconstruction, kept or not.

groups list of Group

The groups, with their L-curves.

model instance-attribute

mean instance-attribute

std instance-attribute

chi2_red instance-attribute

trace instance-attribute

members instance-attribute

groups instance-attribute

kept property

The members in the mean.

__init__(model, mean, std, chi2_red, trace, members, groups)

summary()

A short table: the groups, what was kept, and the mean's fit.

draw_groups(data, n_groups, key, spec=None)

Draw the settings of n_groups groups of reconstructions.

Each group draws a regularizer family, a pixel size (the Nyquist scale over one of spec.oversample), a field (field_of_view(data) times one of spec.field_factors, at most 1 on a uv lattice; beyond spec.max_npix pixels the pixels are coarsened to fit it, down to the Nyquist scale), a starting image and spec.n_weights log-uniform weights, one in each equal bin of the family's range in log w. The draws depend only on key and spec, so every task of a cluster array can draw them all and run its own.

Parameters:

Name Type Description Default
data OIData or sequence of OIData

The data.

required
n_groups int

Number of groups.

required
key PRNGKey

The random key.

required
spec EnsembleSpec

What to draw (default EnsembleSpec()).

None

Returns:

Type Description
list of Draw

Sorted by geometry, so that groups sharing a compilation run one after the other.

reference_starts(data, star=True, starts=('moments', 'flat'))

The starting images that each group resamples onto its own grid.

Each is made once by starting_image, which fits a star plus a Gaussian envelope: "moments" is that Gaussian, "dirty" the positive part of the dirty image, and "flat" a uniform image with the fitted flux.

Returns:

Type Description
dict

Start name to an Image.

run_group(data, draw, *, star=True, starts=None, **fit_options)

Fit one group of an ensemble: an L-curve over its weights.

Parameters:

Name Type Description Default
data OIData or sequence of OIData

The data.

required
draw Draw

The group, from draw_groups.

required
star bool

Whether the scene is an unresolved star at the origin plus the image (default), or the image alone, centred by a Centroid prior one pixel wide.

True
starts dict

From reference_starts; computed here if not given (pass it to avoid refitting the envelope for every group).

None
**fit_options

Passed to l_curve and so to fit, e.g. method and max_steps.

{}

Returns:

Type Description
Group

combine(data, groups, *, spec=None, star=True)

Select the members of an ensemble and average them.

The selection follows MYTHRA (Drevon et al. 2025):

  1. In each group, keep the weights from spec.window_dex below the L-curve's corner up to the corner: the weights just before the turnover.
  2. Drop members whose fit diverged (a non-finite χ²), then keep members whose raw χ² per data point is below spec.max_chi2_red and within spec.chi2_ratio of the best member's on every dataset, then drop total-χ² outliers by their median absolute deviation (spec.mad_cut).
  3. Resample the survivors' images to a common grid, the finest pixels and the largest field among them, conserving flux; without a star, recentre each on the best member (align).
  4. In order of total χ², add members to a running mean one at a time, keeping each only if the mean's χ² on every dataset stays within spec.mean_rtol (by default the χ²/N noise, √(2/N)) of the best member's (so visibilities and closure phases, given as separate datasets, are judged separately). Judging against the best member, not the running mean, keeps the tolerance from compounding. The running mean is judged as the mixture of the members' images on their own grids, which is exact; resampling to the common grid smooths them, which on precise data can raise χ² several-fold and so let worse members through.

With a star, the mean is of the whole normalized sky, star included: the star's fraction of the flux is the members' mean, and the image's pixels the mean of their fluxes.

Parameters:

Name Type Description Default
data OIData or sequence of OIData

The data the groups were fitted to.

required
groups sequence of Group

From run_group.

required
spec EnsembleSpec

The selection settings (default EnsembleSpec()).

None
star bool

As for run_group.

True

Returns:

Type Description
Ensemble

ensemble(data, n_groups, key, *, spec=None, star=True, **fit_options)

Run, select and average an ensemble of randomized reconstructions.

draw_groups, then run_group for each group in turn, then combine. Each group is one L-curve, so the ensemble has n_groups * spec.n_weights members, and compiles once per distinct (family, npix, pixel_scale_mas): a handful, since each is drawn from a short list.

Parameters:

Name Type Description Default
data OIData or sequence of OIData

The data. Give visibilities and closure phases as separate datasets for the mean to be judged on each.

required
n_groups int

Number of groups.

required
key PRNGKey

The random key.

required
spec EnsembleSpec

What to draw and how to select (default EnsembleSpec()).

None
star bool

Whether the scene has an unresolved star at the origin (default).

True
**fit_options

Passed to fit.

{}

Returns:

Type Description
Ensemble