Skip to content

virgil.radio

Provisional

May change in any 0.4.x release without a deprecation cycle.

Complex visibilities from radio interferometers (CASA measurement sets), with per-observation calibration nuisances and a likelihood term for fit. See the radio interferometry guide.

Radio-interferometric (u, v, complex visibility) data and calibration.

Provisional

May change in any 0.4.x release without a deprecation cycle.

ComplexVisData holds calibrated complex visibilities at arbitrary (u, v) points, read from a CASA measurement set or built from arrays. Its term is a likelihood term for fit (and numpyro_model) whose model is any SourceModel, multiplied by per-observation nuisances (CalibrationNuisance): a flux scale, relative gains, phase-centre offsets, error scales per spectral window and a spectral index.

Conventions (pinned by tests, see the Radio interferometry guide): the visibility is V(u, v) = Σ I(x, y) exp(-2πi (u x + v y)) with x towards East and y towards North and u, v in wavelengths, as everywhere in virgil; a weight is 1 / σ² for each of the real and imaginary parts.

Source: the RML objective of Zawadzki et al. (2023); the reference-observation gauge of Cornwell & Wilkinson (1981); the profiled error scale of Pawitan (2001).

ComplexVisData

Bases: Module

Complex visibilities at (u, v) points, with calibration labels.

Build one with from_arrays or read_ms.

Attributes:

Name Type Description
u, v (N,) float64

Spatial frequencies in wavelengths (UVW * ν / c per channel).

vis (N,) complex

Calibrated visibilities (Jy).

weight (N,) float

1 / σ² of each of the real and imaginary parts (Jy⁻²).

freq (N,) float64

Channel frequency (Hz).

ddid, obs (N,) int32

Labels relabelled 0..n-1: obs in sorted order of the original OBSERVATION_ID (obs_ids); ddid numbers the sorted (DATA_DESC_ID, observation) pairs, so a ddid shared by several observations of a concatenated set gets one label (and one error scale) per observation. ddid_ids holds each label's original DATA_DESC_ID.

ddid_obs tuple of int

The (relabelled) observation of each ddid label.

time, ant1, ant2, chan, scan optional arrays

MJD seconds, antenna indices, channel index and scan number (SCAN_NUMBER), when known. time, ant1, ant2 and scan are what sigma_scale needs.

u instance-attribute

v instance-attribute

vis instance-attribute

weight instance-attribute

freq instance-attribute

ddid instance-attribute

obs instance-attribute

ddid_ids = eqx.field(static=True) class-attribute instance-attribute

obs_ids = eqx.field(static=True) class-attribute instance-attribute

ddid_obs = eqx.field(static=True) class-attribute instance-attribute

time = None class-attribute instance-attribute

ant1 = None class-attribute instance-attribute

ant2 = None class-attribute instance-attribute

chan = None class-attribute instance-attribute

scan = None class-attribute instance-attribute

from_arrays(u, v, vis, weight, freq, ddid=0, obs=0, *, time=None, ant1=None, ant2=None, chan=None, scan=None) classmethod

Build from arrays; scalars broadcast over the points.

ddid and obs are the original labels, relabelled 0..n-1 in sorted order. Each ddid must belong to one observation.

read_ms(path, ddids=None, stride=1, datacolumn='DATA', field=None, conjugate=None, u_sign=None) classmethod

Read a CASA measurement set (needs the radio extra).

Cross-correlations only; one sample per (row, channel), kept if the row and both parallel-hand polarizations (XX, YY or RR, LL) are unflagged and both weights positive. The two polarizations are averaged with their weights (Stokes I for equal weights) and the weight is their sum. WEIGHT_SPECTRUM is used when present and non-empty, else WEIGHT. u = UVW[0] ν / c, v = UVW[1] ν / c per channel. One ddid is read at a time, so memory is that of the largest ddid at the given stride.

Parameters:

Name Type Description Default
path str

The measurement set.

required
ddids sequence of int

DATA_DESC_ID values to read (default all).

None
field int

FIELD_ID to read. By default the selection must hold one field only; a mosaic (several fields) raises.

None
stride int

Keep every stride-th row of each ddid.

1
datacolumn str

"DATA" or "CORRECTED_DATA".

'DATA'
conjugate optional

Override the module's pinned convention (MS_CONJUGATE, MS_U_SIGN) for comparisons.

None
u_sign optional

Override the module's pinned convention (MS_CONJUGATE, MS_U_SIGN) for comparisons.

None

__len__()

subset(mask)

The points where boolean mask is true, labels unchanged.

Labels are kept as they are (not relabelled), so a CalibrationNuisance built for the full data still fits.

select(ddids=None, obs=None)

Points of the given ddids and/or observations.

ddids and obs are the original ids (ddid_ids, obs_ids). Labels are kept; see subset.

model(scene, values, nuisance)

The complex model M_k at every point (all at once).

M = F g_e (ν/ν₀)^α exp(-2πi(δα_e u + δδ_e v)) T(u, v) V_scene(u, v); see CalibrationNuisance. scene is a SourceModel (normalized to unit flux; cal.flux sets the scale), evaluated at wavelength c / ν of each point so that chromatic scenes work.

term(scene, nuisance, mask=None, chunk_size=10000000, sigma=None, chi2='auto')

A likelihood term for fit's likelihoods.

Parameters:

Name Type Description Default
scene callable

Maps the fitted values (a dict by site name) to a SourceModel; a function of the values, not a model instance.

required
nuisance CalibrationNuisance

The calibration model; its sites are keys of the fit's priors.

required
mask (N,) bool

Points to use (default all); masked points change neither χ² nor the normalization.

None
chunk_size int

Points evaluated at a time, bounding memory.

10000000
sigma (n_ddid,) array

With sigmas="profile", fixed error scales instead of the ŝ of this term's own points. Score held-out data at the training fit's ŝ (fit(...).info["derived"]["cal.sigma"]): profiling its own ŝ is not a predictive score.

None
chi2 ('auto', 'direct', 'toeplitz')

How fit evaluates χ². "auto" uses the dirty-beam form of toeplitz when it is exact (one fixed Image, no antenna_gains, fitted alpha or offsets, float64, method="lbfgs" or "adam", enough points per pixel and ddid, see set_ft_defaults); "toeplitz" requires it (ValueError names what fails; fitted offsets then use offsets="spread"); "direct" never uses it. Each fit rebuilds the precompute: reuse toeplitz(...) for repeated fits. numpyro_model and cross_validate stay direct (it re-profiles each fold: use sigmas=True).

"auto"
Notes

In your own jitted code pass the term as an argument with eqx.filter_jit; a jax.jit closure bakes the data in as constants (see the radio interferometry page).

toeplitz(image, nuisance, mask=None, offsets='none', eps=1e-14, max_offset_mas=None, chunk_size=65536)

Precompute the dirty beams and images of a one-Image fit.

Returns a ToeplitzVisData, whose term gives the χ² of term for a scene that is one Image with the layout of image, at a cost independent of the number of points. Provisional.

Parameters:

Name Type Description Default
image Image

The layout (shape, pixel_scale_mas, rotation_deg, smooth_fwhm_mas) and a fixed dra, ddec (folded into the visibilities); its pixels are not used.

required
nuisance CalibrationNuisance

Its taper, nu0_hz and fixed alpha are folded in; antenna gains and a fitted alpha are refused.

required
mask (N,) bool

Points to use (default all).

None
offsets ('none', 'spread')

"none" (default) needs CalibrationNuisance(offsets=False). "spread" fits the offsets by evaluating the stored spread grids at shifted positions (an extension: see the guide), and refuses baselines past its Nyquist margin.

"none"
eps float

Tolerance of the spreading kernel.

1e-14
max_offset_mas float

Largest offset of any observation, in mas (default the bound of nuisance.priors()).

None
chunk_size int

Points spread at a time, bounding memory.

65536

sigma_hat(scene, values, nuisance, mask=None)

The error scales s_d per ddid at values: the profiled ŝ of CalibrationNuisance(sigmas="profile"), the fitted cal.sigma with sigmas=True, ones otherwise. Provisional, as the module.

chi2_by_ddid(scene, values, nuisance, mask=None, chunk_size=10000000)

χ² of the data with the σ scales applied, per ddid.

Returns (chi2, n): float64 arrays of length n_ddid, n the number of unmasked points of each ddid (χ² sums two terms per point). chunk_size is that of term.

sigma_scale(by='ddid')

Model-free estimate of the error scale from time-adjacent samples.

Samples of one baseline, channel and scan (and ddid label) are paired in time order without overlap; ŝ² is the mean over pairs of ΔRe² / (1/w_i + 1/w_j), likewise for Im, and ŝ averages the two.

This is an upper bound. Sky, gain or phase change within a pair adds to |ΔV|², so ŝ is biased high for rapidly varying data (long gaps, atmospheric phase), exact only if the signal is constant. Samples without a partner are not used. Needs the time, ant1, ant2 and scan fields filled by read_ms; with stride > 1 the rows read are no longer adjacent.

Parameters:

Name Type Description Default
by ('ddid', 'obs', None)

Grouping of the result: per DATA_DESC_ID, per observation id, or pooled (None).

"ddid"

Returns:

Type Description
SigmaScale

Group labels, pair counts, and s, s_re, s_im with standard errors. lag_ratio well above 1 means the sky varies (ŝ is conservative); near 1 means constant data or a time order that does not follow the data (a scrambled time column): check ŝ.

dirty_beam(npix, pixel_scale_mas, weighting='natural', robust=0.5, values=None, nuisance=None, uv_cell=None, chunk_size=1000000, uvtaper=None)

The dirty beam Σ w cos(2π(u x + v y)) / Σ w on the pixel grid.

The response to a unit point source at the phase centre, normalized to 1 there. See dirty_image for the parameters; values and nuisance together divide the weights by the fitted s_d² (as use_sigma=True). On an even npix the phase centre lies between four pixels (the convention of Image), so the sampled peak is slightly below 1; use an odd npix for a pixel centred on it.

dirty_image(npix, pixel_scale_mas, weighting='natural', robust=0.5, scene=None, values=None, nuisance=None, use_sigma=False, uv_cell=None, chunk_size=1000000, uvtaper=None)

Dirty image Σ w Re[V e^{+2πi(u x + v y)}] / Σ w in Jy/beam.

A display diagnostic, never part of the likelihood: a type-1 NUFFT at the raw (u, v), in the orientation of Image (East left, North up). Details: Dirty and residual images.

Parameters:

Name Type Description Default
npix int
required
pixel_scale_mas float
required
weighting ('natural', 'uniform', 'briggs')

Natural w (1/σ²), uniform w / W_c for the summed weight W_c of the point's uv cell, or Briggs with robust.

"natural"
robust float

Briggs R, about −2 (uniform) to 2 (natural).

0.5
scene callable or SourceModel

With values and nuisance, image the residual V − M, M including the nuisances.

None
values optional

Fitted values and the CalibrationNuisance.

None
nuisance optional

Fitted values and the CalibrationNuisance.

None
use_sigma bool

Divide the weights by the fitted s_d² (needs values and nuisance).

False
uv_cell float

Cell of the density grid in wavelengths; default 1/FoV (npix · pixel_scale_mas in radians).

None
chunk_size int

Points transformed at a time, bounding memory.

1000000
uvtaper float or (bmaj_mas, bmin_mas, pa_deg)

CASA's uvtaper: a Gaussian FWHM in mas (a float is circular; pa_deg is East of North), applied after the density weights. Provisional, as the module.

None

Returns:

Type Description
(npix, npix) float64 array

CalibrationNuisance

Per-observation calibration nuisances of a radio dataset.

Static configuration: which nuisances are fitted. The sites (prefix="cal") are keys of the fit's priors; a function model must accept them. See Radio interferometry.

=============== ================ ========================== ============= Site Shape Default prior Not fitted =============== ================ ========================== ============= cal.flux () LogUniform(1e-4, 1e2) always fitted cal.gain (n_obs-1,) Jeffreys 1/g 1 cal.offset (n_obs-1, 2) Uniform(-θ_k, θ_k) mas 0 cal.sigma (n_ddid,) Jeffreys 1/s 1 cal.alpha () Uniform(0, 5) 0 (or alpha) =============== ================ ========================== =============

fit_gain_widths adds scalar sites cal.gain_amp_width and cal.gain_phase_width (LogUniform(1e-3, 10)). The reference observation has gain 1 and zero offset; θ_k is offset_beams beams of observation k (fit(...).info["at_bound"] reports a hit).

Parameters:

Name Type Description Default
data ComplexVisData
required
ref_obs int

Index of the reference observation.

0
gains bool (``alpha`` also a float)

Which nuisances are fitted (the flux always is); gains and offsets need more than one observation; a nuisance not fitted takes the table's last column. sigmas="profile" sets the error scales at every evaluation (see sigma_hat). A float alpha fixes the spectral index (a non-zero integer is refused: alpha=1 used to mean "fit").

True
offsets bool (``alpha`` also a float)

Which nuisances are fitted (the flux always is); gains and offsets need more than one observation; a nuisance not fitted takes the table's last column. sigmas="profile" sets the error scales at every evaluation (see sigma_hat). A float alpha fixes the spectral index (a non-zero integer is refused: alpha=1 used to mean "fit").

True
sigmas bool (``alpha`` also a float)

Which nuisances are fitted (the flux always is); gains and offsets need more than one observation; a nuisance not fitted takes the table's last column. sigmas="profile" sets the error scales at every evaluation (see sigma_hat). A float alpha fixes the spectral index (a non-zero integer is refused: alpha=1 used to mean "fit").

True
alpha bool (``alpha`` also a float)

Which nuisances are fitted (the flux always is); gains and offsets need more than one observation; a nuisance not fitted takes the table's last column. sigmas="profile" sets the error scales at every evaluation (see sigma_hat). A float alpha fixes the spectral index (a non-zero integer is refused: alpha=1 used to mean "fit").

True
offset_beams float

Half-width of the default offset prior, in beams; default 3. Provisional, as the module.

3.0
antenna_gains bool

Marginalize a complex gain per antenna and solution interval (antenna gains); needs time, ant1, ant2; no least-squares form.

False
solint ('int', 'scan')

Solution interval: an integration, a scan or seconds (within a scan).

"int"
gain_amp_width float

Widths τ_a and τ_φ (rad) of the Gaussian gain priors (choices).

0.1
gain_phase_width float

Widths τ_a and τ_φ (rad) of the Gaussian gain priors (choices).

0.1
fit_gain_widths bool or {'amp', 'phase'}

Fit the widths (log-uniform priors; see priors).

False
gain_steps int

Gauss–Newton steps of the gain solution.

5
nu0_hz float

Reference frequency of the spectral index; default the weight-averaged frequency of data.

None
taper_fwhm_mas float

FWHM of a Gaussian taper (a convolution), in mas.

None
prefix str

Site-name prefix.

'cal'

n_obs = len(data.obs_ids) instance-attribute

n_ddid = len(data.ddid_ids) instance-attribute

ref_obs = int(ref_obs) instance-attribute

gains = bool(gains) and self.n_obs > 1 instance-attribute

offsets = bool(offsets) and self.n_obs > 1 instance-attribute

sigmas = 'profile' instance-attribute

nu0_hz = float(nu0_hz) instance-attribute

flux_start = float(total) if total > 0 else 0.1 instance-attribute

offset_bound_mas = float(rad2mas / (2.0 * q_min)) instance-attribute

offset_unit_mas = rad2mas / (2.0 * onp.pi * q_max) instance-attribute

offset_beams = float(offset_beams) instance-attribute

taper_fwhm_mas = None if taper_fwhm_mas is None else float(taper_fwhm_mas) instance-attribute

prefix = prefix instance-attribute

antenna_gains = bool(antenna_gains) instance-attribute

solint = solint instance-attribute

gain_widths = (float(gain_amp_width), float(gain_phase_width)) instance-attribute

fit_widths = (self.antenna_gains and fit_gain_widths in (True, 'amp'), self.antenna_gains and fit_gain_widths in (True, 'phase')) instance-attribute

gain_steps = int(gain_steps) instance-attribute

sites property

Names of the fitted sites.

__init__(data, ref_obs=0, gains=True, offsets=True, sigmas=True, alpha=True, nu0_hz=None, taper_fwhm_mas=None, prefix='cal', offset_beams=3.0, antenna_gains=False, solint='scan', gain_amp_width=0.1, gain_phase_width=0.5, fit_gain_widths=False, gain_steps=5)

priors(**bounds)

Default priors of the fitted sites, a dict for fit.

bounds override the ends (lo, hi) of a site's prior, keyed by flux, gain, offset, sigma, alpha, gain_amp_width or gain_phase_width (the last two are LogUniform(1e-3, 10) by default). Without a bound, the gains and σ scales are improper Jeffreys priors (flat in ln x, no box) and the offsets are uniform over ±offset_beams beams of their observation (2π b_k, see the class docstring). offset=(-1/(2 q_min), 1/(2 q_min)) in mas, with q_min the shortest baseline (offset_bound_mas), is the old default: a shift of the phase centre beyond that is less than half a turn of phase on every baseline.

defaults()

Starting values of the fitted sites.

SigmaScale dataclass

Result of ComplexVisData.sigma_scale.

Arrays have one entry per group, in the order of labels (the original DATA_DESC_ID or observation ids, or None when pooled). s is the combined error scale and s_se its standard error; s_re and s_im are the real and imaginary estimates; n_pairs the pairs used; lag_ratio the far-pair to adjacent-pair mean-square difference. All are upper bounds on the true scale (sky change inflates them).

labels instance-attribute

n_pairs instance-attribute

s instance-attribute

s_se instance-attribute

s_re instance-attribute

s_re_se instance-attribute

s_im instance-attribute

s_im_se instance-attribute

lag_ratio instance-attribute

__init__(labels, n_pairs, s, s_se, s_re, s_re_se, s_im, s_im_se, lag_ratio)

primary_beam(npix, pixel_scale_mas, freq_hz, dish_m, factor)

Log primary-beam attenuation -4 ln 2 r² / FWHM² on the pixel grid.

FWHM = factor c / (ν D) and r is the distance from the image centre (factor is about 1.0–1.3 depending on the illumination: for example 12 m dishes with factor = 1.13, as ALMA's, or 22 m dishes for ATCA; use the telescope's documented values). The pixel grid is that of pixel_offsets). Add it to an Image's log-brightness so that the image is the sky times the beam.

dartboard_folds(data, k=5, n_q=12, n_phi=16, seed=0)

Cross-validation folds from uv cells (a "dartboard").

The uv plane is cut into n_q log-spaced annuli between the smallest and largest q = √(u² + v²) and n_phi azimuth sectors of φ = arctan2(v, u) mod π, so a point and its Hermitian partner share a cell. The occupied cells are shuffled into k folds with numpy.random.default_rng(seed).

Returns:

Type Description
fold, annulus : (N,) int arrays

The fold (0..k-1) and the annulus index of every point.