Chapter 6
POL-2 – Advanced Data Reduction

 6.1 Adding new observations
 6.2 Experimenting with pixel sizes
 6.3 Investigating systematic error in IP
 6.4 Adding WCS information back into a vector catalogue

The pol2map tool for reducing POL-2 data was released to the science community for the start of 17B observing. As with all newly commissioned instrumentation the “ideal” reduction has yet to be finalised. This advanced section of the POL-2 data reduction documentation aims to provide you with tools for expanding and examining the POL-2 reduction process further and in more detail.

For further ideas, see Section 3.6.

6.1 Adding new observations

This section describes the six-step process of combining data for one or more new POL-2 observations into existing I, Q and U maps and vector catalogue created by an earlier run of pol2map.

Create a text file listing all the existing auto-masked I maps for individual observations stored in the directory specified by Parameter MAPDIR, and then add in the raw data files for the new observations. The auto-masked I maps have names that end in _imap.sdf.
  % ls maps/*imap.sdf > infiles.list
  % ls rawdata/*.sdf  >> infiles.list
Create a new auto-masked, co-added I map including the new observation. The calcqu and makemap commands will be run on the new data and the resulting maps combined with the existing maps derived from the older observations to create the new map.
  % pol2map in=^infiles iout=iauto_new qout=! uout=! mapdir=maps \
A decision needs to be taken whether to re-create all the externally masked maps using external masks defined by the new auto-masked map. This will be the case if the auto-masked map has been changed significantly by the addition of the new observation. To do this, it is necessary to compare the old and new masks. The old masks should have been created earlier using the MASKOUT1 and MASKOUT2 parameters (see Step 3 in Section 3). To create the new masks that would be generated from the new auto-masked map, use this command.
  % pol2map  in=^infiles iout=! qout=! uout=! mapdir=maps mask=iauto_new \
       maskout1=astmask_new  maskout2=pcamask_new
Decide if the addition of the new data has changed the masks significantly. This involves comparing astmask.sdf and astmask_new.sdf (and also pcamask.sdf and pcamask_new.sdf).
If the mask has changed significantly and all observations need to be reprocessed using the new mask, remove the existing externally-masked maps so that they will be re-created by the next invocation of pol2map. Note – this will increase the length of time taken by Step 6 enormously.

Ensure the new auto-masked co-add is used in place of the old one to define any new masks needed in future.

  % rm mapdir/*Qmap.sdf mapdir/*Umap.sdf mapdir/*Imap.sdf
  % mv iauto.sdf iauto_old.sdf
  % mv iauto_new.sdf iauto.sdf
Re-create the necessary externally masked maps and co-adds, and then create the new vector catalogue.
  % pol2map in=qudata/\* iout=iext_new qout=! uout=! mapdir=maps \
  % pol2map in=qudata/\* iout=! qout=qext_new uout=uext_new mapdir=maps \
       mask=iauto ipref=iext_new cat=mycat_new debias=yes

6.2 Experimenting with pixel sizes

Currently,the default map pixel size is 4′′ at both 450 and 450 µm. The pixel size is controlled by the PIXSIZE parameter in the Smurf pol2map command:

  % pol2map pixsize=12

The following four-step example shows how to investigate the impact of changing pixel size. In this example, we compare 12′′ pixels and 7′′ pixels.

Begin with an auto-masked total-intensity map from the raw data. For instance:
  % pol2map in=^myfiles.list iout=iauto12 pixsize=12 qout=! uout=! \
       mapdir=maps12 qudir=qudata
Create AST and PCA masks with 12′′ pixels from the iauto12.sdf file.
  % pol2map in=qudata/\* iout=! qout=! uout=! mapdir=maps12 mask=iauto12 \
       maskout1=astmask12 maskout2=pcamask12
Create masks with 7′′ pixels by resampling the 12′′ masks created at Step 2. This is done using the Kappa sqorst command:
  % sqorst  mode=pixelscale pixscale=\’7,7,7E-05\’ in=astmask12 out=astmask7
  % sqorst  mode=pixelscale pixscale=\’7,7,7E-05\’ in=pcamask12 out=pcamask7
Create the 7′′ externally masked I, Q and U maps using the above 7′′ masks (note the mask parameter value is enclosed in single and double quotes).
  % pol2map in=qudata/\* iout=iext7 qout=qext7 uout=uext7 masktype=mask \
                    mask="’astmask7,pcamask7’" mapdir=maps7 ipref=iext7  \
                    cat=cat7 debias=yes

Using larger pixels usually produces slower convergence, so the above process will take longer than usual – be patient!

Using larger pixels can sometimes encourage smooth blobs and other artificial features to appear in the map. The iauto12.sdf file should be examined to check that it does not have such artificial features.

Check the masks (astmask12.sdf and pcamask12.sdf) to make sure they look reasonable.

It is usually advisable to leave PIXSIZE at its default value and instead use the BINSIZE parameter to control the bin size in the vector catalogue - see Section 4.6).

6.3 Investigating systematic error in IP

The error on the IP is reported to be of the order of 0.5%. It is possible to investigate the effects of the systematic error in IP by creating maps using the upper and lower limits on the IP value. The makemap configuration parameter called ipoffset can be used to do such an investigation. To use it, run pol2map twice as follows:

  % pol2map config="ipoffset=-0.25"
  % pol2map config="ipoffset=0.25"

to produce maps using the upper and lower IP limits (a range of 0.5%). If pol2map has already been run on POL-2 data then a file will already exist that was created using the mean IP (the mean IP is used if ipoffset is omitted from the configuration value, or the configuration parameter itself is omitted).

6.4 Adding WCS information back into a vector catalogue

Vector catalogues produced by pol2map contain information about World Coordinate Systems (WCS) in two different forms:

The catalogue contains “RA” and “Dec” columns that hold the sky position (FK5, J2000) of each vector, in radians.
The catalogue header contains a Starlink “WCS FrameSet” which defines (amongst other things) the projection from pixel coordinates within the I, Q and U mosaics, to RA and Dec. This FrameSet is used by Starlink software, together with the pixels coordinates stored in the “X” and “Y” columns, to determine the RA and Dec of each vector. The WCS FrameSet also defines the polarimetric reference direction used by the Q, U and ANG values. See “Using World Co-ordinate Systems” within SUN/95 (the Kappa manual) for more information on the ways in which Starlink software handles WCS information.

Starlink software such as Polpack, Kappa and Gaia rely on the WCS FrameSet for all WCS-related operations (drawing annotated axes, aligning data sets, etc). Thus problems are likely if the WCS FrameSet is removed from the vector catalogue. This could happen for instance if you use inappropriate software to process an existing catalogue, creating a new output catalogue – the WCS FrameSet may not be copied to the output catalogue, causing subsequent WCS-related operations to fail. It is safe to use Polpack, Kappa, Gaia and Cursa) as all these packages copy the WCS FrameSet to any new output catalogues. Unfortunately, the popular Topcat catalogue browser (see and the STILTS package ( upon which it is based, do not copy the WCS FrameSet to any output catalogues.

For this reason, Polpack contains a command that can be used to copy the WCS FrameSet from one catalogue to another. Say for instance you create catalogue mycat.FIT using pol2map, and then use Topcat to remove low signal-to-noise vectors, saving the results to a new catalogue called selcat.FIT. The WCS FrameSet will be missing from selcat.FIT, and so we need to copy it back again from the original catalogue mycat.FIT. To do this we use the “polwcscopy” command:

  % polwcscopy in=selcat ref=mycat out=selcat2

This creates a third catalogue selcat2.FIT, which is a copy of selcat.FIT but with WCS inherited from mycat.FIT.