SURFLIB_DIFF_DESPIKE

Despike data scan by detecting points more than nsigma from running 2-point mean

Description:

This routine removes spikes from SCAN/MAP observations. The scan map differential despiking algorithm uses 2 criteria to decide which points are spikes.

First, for each bolometer used a pass is made through each scan calculating for each point:-

diff(i) = point(i) (point(i 1) + point(i + 1)) 2.0 (3)

Values of ’diff’ for the first and last points in the scan are calculated in a similar way but subtracting the mean of points 2 and 3 and points n-1 and n-2 respectively.

The mean and standard deviation of ’diff’ are calculated by coadding the 10 points at each end of the scan where, hopefully, there is no source emission. Spikes in these regions are handled by removing points from the coadd that lie further than 3 sigma from the mean, then redoing the calculation recursively until no further points need be removed.

The first criterion for a spike is that it’s ’diff’ value should be further from the mean of ’diff’ by NSIGMA times the sigma derived from the endpoints.

The problem with this simple approach is that bright sources in the scan themselves lead to excursions in ’diff’ that can be wrongly identified as spikes. To prevent this happening a second criterion is used. In this the scan values are convolved with a 3 sample wide box so that each ’box’ point is the average of the point itself and the points on either side of it. ’Box’ is expected to increase faster for real sources than for spikes because in them the increase will be spread over all 3 averaged points rather than just 1.

The second criterion for a spike is met, therefore, if a point’s ’diff’ is further from the ’diff’ mean than the value of ’box’ at that point.

Fixed-up values for points that have identified as spikes are calculated by interpolating between the closest healthy points on either side.

The second spike criterion also means unfortunately that the technique is less sensitive to spikes on bright sources than elsewhere. In addition, it is still possible to clip bright sources if too low a value for NSIGMA is used. It is recommended to run despike several times with different values of NSIGMA. Begin with NSIGMA=5, look at the result to see how effective despiking has been, then repeat the process with NSIGMA=4.5, 4.0 etc. until you start to clip source information.

Invocation

CALL SURFLIB_DIFF_DESPIKE (N_EXPOSURES, N_INTEGRATIONS, N_MEASUREMENTS, DEMOD_POINTER, N_BOL, N_POS, IN_DATA, IN_VARIANCE, IN_QUALITY, BADBIT, NSIGMA, OUT_DATA, OUT_VARIANCE, OUT_QUALITY, NUM_SPIKES, STATUS)

Arguments

N_EXPOSURES = INTEGER (Given)
maximum number of exposures per integration
N_INTEGRATIONS = INTEGER (Given)
number of integrations in the observation
N_MEASUREMENTS = INTEGER (Given)
number of measurements in the observation
DEMOD_POINTER (N_EXPOSURES, N_INTEGRATIONS, N_MEASUREMENTS)
array pointing to start and finish of scans in IN_DATA
N_BOL = INTEGER (Given)
the number of bolometers for which data was taken
N_POS = INTEGER (Given)
the number of positions measured in the scan
IN_DATA (N_BOL, N_POS) = REAL (Given)
the measured data
IN_VARIANCE (N_BOL, N_POS) = REAL (Given)
the variance on IN_DATA
IN_QUALITY (N_BOL, N_POS) = BYTE (Given)
the quality on IN_DATA
BADBIT = BYTE (Given)
bad bit mask
NSIGMA = REAL (Given)
cut-off sigma for spikes
OUT_DATA (N_BOL, N_POS) = REAL (Returned)
the deconvolved data
OUT_VARIANCE (N_BOL, N_POS) = REAL (Returned)
the variance on OUT_DATA
OUT_QUALITY (N_BOL, N_POS) = BYTE (Returned)
the quality on OUT_DATA
NUM_SPIKES = INTEGER (Returned)
Total number of spikes detected
STATUS = INTEGER (Given and returned)
global status