Chapter 7
Examples of Different Reductions

 7.1 Deep point-source maps
  7.1.1 Example 1 – The simple reduction
  7.1.2 Example 2 – Advanced pipeline method
 7.2 Extended galactic sources

7.1 Deep point-source maps

The science goal of many extra-galactic SCUBA-2 observations is to detect unresolved point sources. In the examples below we work through the reduction of just such an extra-galactic field, A1835.

Most extra-galactic objects are on average only slightly brighter than the confusion limit—the fluctuations of the background sky brightness due to multiple super-imposed, unresolved sources within the telescope beam, below which individual sources cannot be detected. It is likely that any sources in the map will be at best, only a few standard deviations brighter than the noise in the map (caused by a combination of instrumental noise and source confusion).

7.1.1 Example 1 – The simple reduction

The basic reduction method for maps like these follow two main steps—running the data through the map-maker using the dimmconfig_blank_field.lis configuration file (see Section 3.7). Then applying the Picard SCUBA2_MATCHED_FILTER recipe (see Section 8.7).

Step 1: Run the map-maker
In this example the raw data are stored locally in a directory called data. We have three observations (#13, #18, #21) of the field which we will reduced independently.

  % makemap data/s8*00013_00\*.sdf cosmo1 \
  % makemap data/s8*00018_00\*.sdf cosmo2 \
  % makemap data/s8*00021_00\*.sdf cosmo3 \

Step 2: Combine the maps
These three maps are then combined using the Picard recipe MOSAIC_JCMT_IMAGES. In this case we accept the default of wcsmosaic mosaicking and nearest-neighbour pixel spreading and so do not supply a parameter file.

  % picard MOSAIC_JCMT_IMAGES cosmo*.sdf

The output map, cosmo3_mos.sdf (named for the last input file appended by _mos), is shown in the left-hand panel of Figure 7.1. The advantage of using the Picard recipe over standalone Kappa commands is that the exposure time is also propagated correctly to the output mosaic (it is stored in the MORE.SMURF.EXP_TIME extension).

pict pict
Figure 7.1: Reduced pong maps of cosmology field A1835. Left: Map reduced with dimmconfig_blank_field.lis. Right: Map on the left after the matched filter has been applied and it has been cropped.

Step 3: Apply the matched filter
In order to optimally find sources that are the size of the telescope beam, we apply the matched filter recipe, namely SCUBA2_MATCHED_FILTER. We create a simple parameter file called smooth.ini:


where SMOOTH_FWHM = 15 indicates that the background should be estimated by first smoothing the map and PSF with a 15-arcsec FWHM Gaussian. Next, the recipe is executed as follows:

  % picard -recpars smooth.ini SCUBA2_MATCHED_FILTER cosmo2_mos.sdf

The output of this operation is a smoothed image called cosmo3_mos_mf.sdf and a cropped version is shown in the right-hand panel of Figure 7.1. You can immediately see the contrast to the left-hand panel which is the output from the map-maker. A number of signal peaks now emerge as possible sources.

Step 4: Crop the map
Next we shall crop the map to remove the noisy edges, in this case to a 900-arcsec radius circle. The output file will be named cosmo2_mos_mf_crop.sdf.

  % picard CROP_JCMT_IMAGES cosmo3_mos_mf.sdf


Figure 7.2: Signal-to-noise map made using the Kappa command makesnr. The map has been scaled from 0 to +3.

Step 5: Make an S/N map
Finally, we need to find sources. The filtered map contains a VARIANCE component, so it is easy to produce a S/N map using the Kappa task makesnr:

  % makesnr cosmo2_mos_mf_crop cosmo2_mos_mf_crop_snr

The resulting map, cosmo2_mos_mf_snr, is shown in Figure 7.2. Compared with the matched filter map the edges no longer appear as noisy because they have been down-weighted by the larger noise values where there were less data.

Step 6: Identify sources
The basic procedure for identifying sources would be to locate peaks above some threshold S/N. The S/N image above shows peaks that are likely to be real sources. For a start, a source appears where expected at the 0,0 position.

But how can we check if these sources are real?

7.1.2 Example 2 – Advanced pipeline method

Although this method is considerably simpler to execute, the products have undergone more advanced processing than the manual method just given. The pipeline is particularly recommended for this recipe due to its extra analysis steps.

Step 1: Create input file
Create an file with the names of all the files you wish to process (e.g. myfiles.lis)

Step 2: Run the pipeline
The pipeline must first be initiated for the wavelength you are working on. In the case below this is 850 μm. Note that the date does not have to be specified when initialising the pipeline. The pipeline is run using the REDUCE_SCAN_FAINT_POINT_SOURCES_JACKKNIFE recipe; this uses dimmconfig_blank_field.lis as the configuration file. If you wish to provide an alternative file you will need to put the name of the new configuration file in a recipe parameter file. See Section 4 for details.

  % oracdr_scuba2_850 -cwd YYYYMMDD
  % oracdr -loop file -files myfiles.lis -nodisplay \

You substitute the required date for YYYYMMDD. The pipeline will write out a large number of files with the following suffices.

sYYYYMMDD*_fmos The map for each observation
sYYYYMMDD*_mappsf The map for each observation with an artificial point source added at the map centre
gsYYYYMMD*_wmos The co-add of all the _fmos files
gsYYYYMMD*_whiten The whitened version of _wmos
gsYYYYMMD*_cal The calibrated version of _whiten
gsYYYYMMD*_mf The matched-filtered version of _cal

FAINT_POINT_SOURCES_JACKKNIFE is a recipe designed to process blank field/extra-galactic data. The recipe uses a jack-knife method to remove low-spatial frequency noise and generate a matched filter output map.

The recipe processes each observation twice, a standard reduction first, then a re-run with a fake point source added to the time series. This produces a co-added signal map (_wmos) and a coadded PSF map (_mappsf).


After the map-maker has completed, the recipe will call SCUBA2_JACKKNIFE. This routine divides the observations into two groups (odd and even) which are co-added and then subtracted to create a jack-knife map. This map contains only noise with no contribution from astronomical signal. The angular power spectrum of this map is then used to estimate and remove the residual 1/f noise from the signal map and the PSF map; this is the whitening step. The whitened jack-knife map is run through SCUBA2_MATCHED_FILTER using the whitened PSF map as the PSF input. It is this matched filter map which will be of most interest to users.

See SUN/264 for more information on REDUCE_SCAN_FAINT_POINT_SOURCES_JACKKNIFE and all other pipeline recipes.

Step 3: (Optional) Re-run SCUBA2_JACKKNIFE
You may wish to run the SCUBA2_JACKKNIFE step again independently from the pipeline. If your final map does not look as expected you might first examine the individual mosaics from the pipeline (_fmos), one of these observations might show visible artefacts that you wish to exclude from the co-add. The size of the region in the jack-knife image which is used to do the whitening step is determined automatically, but the method may fail if the box is too small.

If you decide to re-run this step you first co-add all the _mappsf files to create a coadded PSF using the Picard recipe MOSAIC_JCMT_IMAGES.

  % picard MOSAIC_JCMT_IMAGES *_mappsf

Next create a parameter file (recpars.lis) for the jack-knife recipe (SCUBA2_JACKKNIFE) containing the following lines.

  PSF_MATCHFILTER = <name_of_above_coadded_PSF>.sdf

Another option for this parameter file is WHITEN_BOX to set the size of the region used to calculate the angular power spectrum. Finally run SCUBA2_JACKKNIFE.

  % picard -log sf -nodisplay -recpars recpars.lis SCUBA2_JACKKNIFE *fmos.sdf

This will create files beginning with pgYYYMMDD that should have the same suffices as above: _wmos, _whiten, _cal, and _mf.

7.2 Extended galactic sources

This example is concerned with recovering bright extended emission. The signal from extended emission varies slowly as seen by the array passing over it. It thus appears at lower frequencies in the power spectrum and complicates the high-pass filter selection. Too harsh a filter will make flat maps but any extended emission will have been removed in doing so.

Step 1: Running the map-maker
We run the map-maker using dimmconfig_bright_extended.lis; we have also specified a couple of overrides on the command line—maptol = 0.04 is slightly more stringent than default and ast.zero_snr = 3.5 constrains everything below 3.5 σ to zero.

In this example we give the map-maker a file containing a list of the input files (filelist.txt) and dimmconfig_bright_extended.lis is in the local directory.

  % makemap in=^filelist.txt 850galactic \

The resulting map is shown in Figure 7.3.


Figure 7.3: The output from the map-maker using dimmconfig_bright_extended.lis.

Step 2: Generating an external mask
Next we create an external mask from the output of makemap. Here we follow the steps outlined in Section 6.6.

  % makesnr 850map 850map_snr

This S/N map is thresholded to set everything below 3 σ to 0 and everything above to 1.

  % thresh 850map_snr 850map_mask thrlo=3 newlo=0 thrhi=3 newhi=1

The thresholded map is shown in the left-hand panel of Figure 7.4. The next step is to smooth this map by convolving it with a Gaussian of 16 arcsec. For this we use a factor of 4 for the FWHM parameter.

  % gausmooth 850map_mask 850map_mask_sm fwhm=4

We threshold the map again to produce our mask. In this case all values below our threshold are set to ‘bad’. The smoothed map now has values scaled between 0 and 1, we set our threshold at 0.02 to include more of the emission beyond the 3 σ edge.

  % thresh 850map_mask_sm 850map_mask_zm thrlo=0.02 newlo=bad thrhi=0.02 newhi=1

The final mask is shown in the right-panel of Figure 7.4. Note how it encompasses more emission and has softer edges than the first threshold map.

pict pict
Figure 7.4: (Left) The initial mask created by thresholding 850map_snr to 3σ. (Right) Second mask made by thresholding the smoothed map to 0.02.

Step 3: Re-running the map-maker with an external mask supplied
As a last step the map is re-made with this mask supplied as an external file. For this run we apply the additional parameters in a personalised configuration file, mydimmconfig.lis.

  % makemap in=^filelist.txt 850galactic \
            config=^mydimmconfig.lis ref=850map_mask_zm

The configuration file, mydimmconfig.lis, has the following format—note how it is based on dimmconfig _bright_extended.lis. It has decreased the convergence parameter to maptol = 0.03 but increased the number of iterations to compensate as 40 is unlikely to be sufficient.

  numiter = -100
  noisecliphigh = 10.0
  maptol = 0.03
  ast.zero_snr = 0

Step 4: Cropping the map
We now crop the map to remove the noisy edges using the Picard recipe CROP_JCMT_IMAGES. To determine what to trim we can look at the exposure-time image with Gaia. See Figure 7.5.


Figure 7.5: The exposure-time image of the science map from Figure 7.3. You can right-click and drag the mouse between two points to measure the distance. Here we see the exposure time dropping off sharply at a radius of 30 arcmin. A non-default colour scale has been chosen to illustrate the morphology.

The exposure-time map shows a sharp drop off at a radius of 30 arcmin. We can thus specify a parameter file like below.

  MAP_RADIUS = 1800
  % picard CROP_JCMT_IMAGES 850galactic.sdf

The final cropped map is shown in Figure 7.6. Compared with the first map out of the map-maker (Figure 7.3), slightly more of the faint extended emission is apparent.

One of the challenges facing this type of reduction is the need to account for both faint extended structure and very bright sources in the same map. You may find some degree of bowling remains around the brightest sources.

There are areas you may wish to experiment with. One is to adjust the filtering. Another option is to supply an external mask from a different dataset, e.g. a Herschel map. See Chapter 6 for further discussion.


Figure 7.6: The final cropped, reduced map from the map-maker run with an external mask supplied.