virgil.detection
Detection statistics of a companion search over a grid: the profile
likelihood ratio Δχ², the grid-marginalized log Bayes factor and the best
flux SNR. They are traceable in the data, so they can be computed for many
simulated observations under jax.lax.map with one compilation, which is
what empirical false-alarm probabilities and ROC curves need.
The Monte Carlo: gaussian_null and bootstrap_null simulate the null
hypothesis (Gaussian noise from the errors, or a residual bootstrap of the
data), rescale_errors first brings the null to χ²_r = 1,
injection_grid lays out companions to inject, and injection_recovery
runs the search on null and injected simulations with one compiled kernel.
Its DetectionMC result gives empirical false-alarm probabilities,
thresholds, ROC curves, completeness maps and contrast curves, and saves,
loads and concatenates for array jobs.
Detection statistics for companion searches, for ROC curves.
detection_statistics reduces a
companion search over a grid to three numbers that measure how strongly the
data prefer a companion to none:
delta_chi2, the profile likelihood ratio 2 [max log L − log L₀] over the grid, with the companion flux constrained to be non-negative;log_bayes_factor, the log evidence ratio of "a companion somewhere on the grid" to "no companion", marginalized over the grid;max_snr, the largest best-fit flux over its Laplace uncertainty, the significance map of the composition tutorial.
The null hypothesis is the same model with the companion flux set to zero and every other parameter as given, so fixed known components (a resolved star, a disk, a companion already found) stay in both hypotheses.
The function is traceable in the data: inside jax.jit or
jax.lax.map over simulated observations it compiles once, which is what
Monte Carlo estimates of false-alarm rates and completeness need.
local_nsigma converts delta_chi2 to
Wilks's Gaussian-equivalent significance at a single position. It ignores
the look-elsewhere effect of searching a grid, so it overstates the
significance of the best of many positions; calibrate with simulations.
The simulations:
gaussian_nullandbootstrap_nullbuild simulators,simulate(key, scene=None) -> OIData: Gaussian noise from a template's errors, or a residual bootstrap of real data about the null scene.rescale_errorsfirst scales errors so that the null scene has χ²_r = 1.injection_gridlays out companions to inject, andinjection_recoveryruns the statistics on null and injected simulations with one compiled kernel.- Its result, a
DetectionMC, holds plain NumPy arrays and gives empirical false-alarm probabilities, thresholds, ROC curves, completeness maps and contrast curves; it saves, loads and concatenates, so that array jobs can be merged.
DetectionMC
dataclass
Statistics of simulated null and injected companion searches.
Made by injection_recovery;
plain NumPy, with no JAX inside.
Attributes:
| Name | Type | Description |
|---|---|---|
null |
dict[str, ndarray]
|
Per null draw: |
injected |
dict[str, ndarray]
|
The same per injected draw, plus the injected values under their
names (e.g. |
meta |
dict
|
JSON-serializable: the grid, fingerprints (hashes of every field,
static or not) of the model, the null scene and the template, the
noise model,
|
Notes
Conventions: the false-alarm probability of a value is the fraction of
null draws at or above it; at a threshold t a draw is detected
when its statistic exceeds t (and, with match_radius, its best
position matches the injection). Separations are hypot(dra, ddec)
of the injections, or their sep for angular grids (sep in mas,
pa in degrees), in mas; fluxes are relative to the primary, as in
absil_limits.
The single-position null of delta_chi2 is ½δ₀ + ½χ²₁, so a local
3σ (local_nsigma = 3, delta_chi2 = 9) is a one-sided FAP of
0.135% (scipy.stats.norm.sf(3)), not the two-sided 0.27%; over a
grid the empirical FAP of that value is larger.
null
instance-attribute
injected
instance-attribute
meta
instance-attribute
n_null
property
Number of null draws.
n_injected
property
Number of injected draws.
__init__(null, injected, meta)
__repr__()
separations()
Separations of the injections in mas.
hypot(dra, ddec) for Cartesian grids, or the injected sep
for angular ones (sep, pa in degrees).
matched()
Whether each injection's best position is within match_radius.
All True when meta["match_radius"] is None. Angular positions
(sep, pa in degrees) are compared on the sky.
false_alarm_probability(stat, value, *, confidence=0.95)
Empirical false-alarm probability of value, with an interval.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
stat
|
str
|
|
required |
value
|
float or array - like
|
Observed statistic(s). |
required |
confidence
|
float
|
Confidence of the binomial interval. |
0.95
|
Returns:
| Type | Description |
|---|---|
fap, lower, upper : numpy.ndarray
|
|
threshold(stat, fap, *, n_boot=200, seed=0)
Threshold of stat at false-alarm probability fap.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
stat
|
str
|
The statistic. |
required |
fap
|
float
|
False-alarm probability (e.g. 1.35e-3, the one-sided Gaussian tail at 3σ). |
required |
n_boot
|
int
|
Bootstrap resamples of the null draws, for the error. |
200
|
seed
|
int
|
Seed of the bootstrap. |
0
|
Returns:
| Type | Description |
|---|---|
threshold, error : float
|
The |
Warns:
| Type | Description |
|---|---|
RuntimeWarning
|
If fewer than one null draw is expected above the threshold
( |
Notes
The bootstrap runs in batches of resamples, holding about 2²²
values at a time, so its memory does not grow with
n_boot × n.
detected(stat, fap)
Whether each injection is detected at false-alarm probability fap.
Its statistic exceeds :meth:threshold and, with match_radius,
its best position matches the injection.
roc(stat, flux=None, sep_bin=None)
ROC curve: true- against false-positive rate over thresholds.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
stat
|
str
|
The statistic. |
required |
flux
|
float or (float, float)
|
Only the injections of this flux (to relative precision 1e-6),
or with |
None
|
sep_bin
|
(float, float)
|
Only the injections with |
None
|
Returns:
| Type | Description |
|---|---|
fpr, tpr, thresholds : numpy.ndarray
|
For each threshold |
auc(stat, flux=None, sep_bin=None)
Area under the :meth:roc curve (0.5 for no discrimination).
completeness(stat, fap, sep_bins=None, flux_bins=None)
Detection fraction of the injections by separation and flux.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
stat
|
str
|
The statistic. |
required |
fap
|
float
|
False-alarm probability of the detection threshold. |
required |
sep_bins
|
array - like
|
Bin edges (mas, and flux). By default each distinct injected
separation or flux (to six significant figures) is its own
bin, which suits an
|
None
|
flux_bins
|
array - like
|
Bin edges (mas, and flux). By default each distinct injected
separation or flux (to six significant figures) is its own
bin, which suits an
|
None
|
Returns:
| Type | Description |
|---|---|
dict
|
|
contrast_curve(stat, fap, completeness=0.5, sep_bins=None, flux_bins=None)
Flux detected with a given completeness, against separation.
In each separation bin of :meth:completeness, the detection
fraction is made non-decreasing in flux (a running maximum) and
interpolated linearly in log flux to where it first reaches
completeness.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
stat
|
str
|
The statistic. |
required |
fap
|
float
|
False-alarm probability of the detection threshold. |
required |
completeness
|
float
|
Target detection fraction, e.g. 0.5 or 0.9. |
0.5
|
sep_bins
|
array - like
|
As for :meth: |
None
|
flux_bins
|
array - like
|
As for :meth: |
None
|
Returns:
| Type | Description |
|---|---|
sep, flux : numpy.ndarray
|
Separations (mas) and the companion flux relative to the
primary, the units of
|
save(path)
Write to path, an .npz with the metadata as JSON.
load(path)
classmethod
Read a file written by :meth:save.
concatenate(results)
classmethod
Merge results of one experiment run with different seeds.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
results
|
sequence of DetectionMC
|
E.g. one per array job. They must agree on the grid, the model,
the null scene, the template, the noise model and
|
required |
Returns:
| Type | Description |
|---|---|
DetectionMC
|
All the null and injected draws, with every seed in |
Raises:
| Type | Description |
|---|---|
ValueError
|
If the metadata are incompatible or a seed repeats. |
detection_statistics(model, data, grid, *, flux_param=None, batch_size=None)
Detection statistics of a companion search over a grid.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model
|
SourceModel or class
|
Companion model, as for
|
required |
data
|
OIData
|
Data to search. It may be traced (e.g. a simulation built inside
|
required |
grid
|
dict[str, array - like]
|
Grid axes, as for
|
required |
flux_param
|
str
|
The key of |
None
|
batch_size
|
int
|
Number of grid points evaluated at once, as for
|
None
|
Returns:
| Type | Description |
|---|---|
dict[str, Array]
|
Scalars:
|
Warns:
| Type | Description |
|---|---|
RuntimeWarning
|
With concrete (untraced) data only: if the flux axis does not
resolve the likelihood peak ( |
Notes
Every statistic uses the full likelihood grid, evaluated once. The flux
axis must resolve the likelihood peak in flux for log_bayes_factor
to approximate the evidence integral, and the coordinate spacing must
resolve it in position (not checked); a coarse grid still gives a
valid test statistic when its null distribution is simulated, but not
an accurate evidence. A strong detection has a narrow peak, so a
log-spaced axis needs many points per decade: the peak's relative width
(FWHM over flux) is about 2.4/SNR, so two steps across it at SNR 10
need a step of 0.12 in ln(flux), about 20 points per decade.
The trapezoid weights make log_bayes_factor a trapezoid-rule
integral over a prior with fixed bounds (the axes' ends), so it
converges as the grid is refined. Equal weights per point would put
the prior's edges half a step beyond the axes' ends, and, since L / L₀
stays near 1 far from a companion, would change the result at first
order in the step.
delta_chi2 at one fixed position follows ½δ₀ + ½χ²₁ under the null
(Chernoff 1954, the flux being on its boundary at 0); searching a grid
makes it larger (the look-elsewhere effect). See
local_nsigma.
Examples:
Statistics for many simulated null observations, compiled once:
>>> keys = jax.random.split(jax.random.PRNGKey(0), 100)
>>> stats = jax.lax.map(
... lambda key: detection_statistics(
... model, template.with_model(null_scene, key=key), grid
... ),
... keys,
... )
local_nsigma(delta_chi2)
Local (Wilks) significance of delta_chi2, in Gaussian sigma.
The significance a delta_chi2 would have at one position fixed in
advance: under the null hypothesis it then follows ½δ₀ + ½χ²₁ (one
flux, bounded at zero), whose upper tail at delta_chi2 is the
one-sided Gaussian tail at sqrt(delta_chi2). This is
nsigma with one degree of freedom.
It is local: it does not account for the look-elsewhere effect of taking the best of many grid positions, so for a search it overstates the significance. The global false-alarm probability needs simulations of the null over the same grid.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
delta_chi2
|
float or array - like
|
Profile likelihood ratio from
|
required |
Returns:
| Type | Description |
|---|---|
array - like
|
Local significance in Gaussian sigma ( |
gaussian_null(null_scene, template, *, error_scale=1.0)
A simulator of the null hypothesis with Gaussian noise.
Each draw is template.with_model(scene, key=key)
(OIData.with_model): the scene's
prediction plus Gaussian noise from the template's errors, closure
phases correlated through cp_noise, and draws of the template's
gains and closure-phase offsets if it has them.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
null_scene
|
SourceModel
|
The scene without a companion. |
required |
template
|
OIData
|
Sampling and errors to simulate. Its values are not used. |
required |
error_scale
|
float
|
The true noise as a multiple of the template's errors. The draws
still carry the template's errors, as real data analysed with
mis-estimated errors would, so |
1.0
|
Returns:
| Type | Description |
|---|---|
callable
|
|
Examples:
>>> simulate = gaussian_null(BinaryModelCartesian(0, 0, 0), template)
>>> data = simulate(jax.random.PRNGKey(0))
bootstrap_null(null_scene, data, *, method='sign_flip')
A simulator of the null hypothesis by residual bootstrap.
The residuals of data about null_scene's prediction are
whitened, sign-flipped ("sign_flip", a wild bootstrap) or resampled
with replacement ("resample"), re-coloured and added back to the
prediction of the scene being simulated:
- visibilities are independent per sample: whitened as r / σ;
- unprojected phases are wrapped into [−π, π) first, as in
OIData.residuals. Uncorrelated phases are whitened as Δ / σ, which equals the likelihood's chord 2 sin(Δ/2) / σ to O(Δ³); both are odd in Δ, so a sign flip flips the likelihood's whitened residual exactly; - closure phases from four or more telescopes are whitened with
data.cp_noise(ClosureNoise.whiten) into independent combinations, which are flipped or resampled and re-coloured withClosureNoise.colour, so the triangles' correlations survive; - projected phases (kernel or DISCO phases) are linear and not wrapped.
Sign flipping keeps each whitened residual's magnitude, so a heteroscedastic or mis-estimated error survives into every draw, and it removes any mean offset; it is the default. Resampling mixes the residuals of different samples (within the visibilities, and within the phases).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
null_scene
|
SourceModel
|
The scene without a companion, fitted to |
required |
data
|
OIData
|
The real data, concrete. Data with gains, closure-phase offsets or extra observables are not supported. |
required |
method
|
(sign_flip, resample)
|
How to draw new whitened residuals. |
"sign_flip"
|
Returns:
| Type | Description |
|---|---|
callable
|
|
Notes
Caveats:
- It needs a null that fits: residuals from a poor fit carry the misfit into every draw.
- A real companion in the data is part of the residuals, so it appears, scrambled, in every null draw and biases the null distribution upward (and the injections' too).
- Only the closure-phase covariance model is kept: correlations between visibilities, between baselines beyond the closure triangles, or between frames are lost.
- That covariance model is Kammerer et al.'s equal-baseline-noise
approximation (see
virgil._closure): with unequal errors within a group, the part of a real residual outside its column space is not recovered, and the re-coloured residuals are not exact closures of baseline phases.
rescale_errors(null_scene, data)
Scale the errors so that the null scene has χ²_r = 1.
The visibility errors and the phase errors are scaled separately, each
by s = sqrt(χ² / n): χ² is the sum of squares of that block of
whitened_residuals of
null_scene (including the periodic penalty rows of correlated
closure phases), and n its independent observables, as in
n_independent (the
independent closure-phase combinations, not the triangles). Without
gains, the whitened residuals scale as 1 / s, so afterwards each block
has χ² / n = 1 exactly; with gains, approximately.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
null_scene
|
SourceModel
|
The scene without a companion, fitted to |
required |
data
|
OIData
|
Data to rescale, concrete (not traced). |
required |
Returns:
| Name | Type | Description |
|---|---|---|
data |
OIData
|
A copy with |
factors |
dict[str, float]
|
The scale factors, |
Notes
n does not subtract the parameters fitted to obtain
null_scene; with few data and several fitted parameters, the
factors are biased low by about p / (2n).
injection_grid(separations, fluxes, n_pa, key)
Companions to inject: every separation and flux at random PAs.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
separations
|
array - like
|
Separations in mas (non-negative). |
required |
fluxes
|
array - like
|
Companion fluxes relative to the primary (non-negative; 0 gives null draws among the injections). |
required |
n_pa
|
int
|
Number of position angles per (separation, flux), each drawn uniformly in [0°, 360°). |
required |
key
|
Array or int
|
Random key (or integer seed) for the position angles. |
required |
Returns:
| Type | Description |
|---|---|
dict[str, ndarray]
|
|
injection_recovery(model, null_scene, template, grid, key, *, n_null, injections=None, noise='gaussian', match_radius=None, flux_param=None, draw_batch=1, chunk_size=None, batch_size=None, progress=True)
Detection statistics of simulated null and injected observations.
Every draw simulates an observation (null, or with a companion
injected), runs the companion search of
detection_statistics on it,
and keeps the statistics and the best position. One compiled kernel,
(key, injection) -> statistics, serves every draw, null and
injected, mapped with jax.lax.map(..., batch_size=draw_batch), so
memory stays bounded (there is no vmap over grid × draws). The default
grid batch_size is divided by draw_batch, and draw_batch is
capped at that default, so the working set stays within the usual
budget of one search whatever draw_batch is.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model
|
SourceModel or class
|
Companion model, as for
|
required |
null_scene
|
SourceModel
|
The scene without a companion. It must predict the same data as
|
required |
template
|
OIData
|
For |
required |
grid
|
dict[str, array - like]
|
The search grid. |
required |
key
|
Array or int
|
Random key, or an integer seed. Draw |
required |
n_null
|
int
|
Number of null draws (may be 0). |
required |
injections
|
dict[str, array - like]
|
Companions to inject, e.g. from
|
None
|
noise
|
(gaussian, bootstrap)
|
Null noise model: |
"gaussian"
|
match_radius
|
float
|
In mas, finite and non-negative. When set, an injection counts as
detected only if its best position lies within |
None
|
flux_param
|
str
|
The flux key of |
None
|
draw_batch
|
int
|
Draws evaluated together (vectorized) within |
1
|
chunk_size
|
int
|
Draws per call of the compiled kernel, rounded up to a multiple of
|
None
|
batch_size
|
int
|
Grid points evaluated at once within each search, as for
|
None
|
progress
|
bool
|
Show a |
True
|
Returns:
| Type | Description |
|---|---|
DetectionMC
|
The statistics of every draw, the injections, and metadata. |
Examples:
>>> grid = {"dra": np.linspace(-150, 150, 16),
... "ddec": np.linspace(-150, 150, 16),
... "flux": np.geomspace(1e-4, 0.03, 32)}
>>> injections = injection_grid([60, 100], [1e-3, 3e-3], 100, 1)
>>> mc = injection_recovery(
... BinaryModelCartesian, BinaryModelCartesian(0, 0, 0), template,
... grid, 0, n_null=1000, injections=injections,
... )
>>> mc.threshold("delta_chi2", 1.35e-3)