Chapter 1
Introduction

 1.1 This cookbook
 1.2 Before you start: computing resources
 1.3 Before you start: software
  1.3.1 Data formats
  1.3.2 Initialising Starlink
  1.3.3 KAPPA and SMURF for data processing
  1.3.4 GAIA for viewing your images and vector maps
  1.3.5 How to get help

1.1 This cookbook

This guide is designed to instruct POL-2 users on the best ways to reduce and visualise their data using Starlink packages: Smurf[5], Kappa, Polpack and Gaia.

This guide covers the following topics.

Throughout this document, a percent sign (%) is used to represent the Unix shell prompt. What follows each % will be the text that you should type to initiate the described action.

1.2 Before you start: computing resources

Compared with SCUBA-2 observations, POL-2 observations are far less memory-intensive to reduce. POL-2 time-series data is down-sampled to 2 Hz as a part of the reduction process. Assuming a typical 35-minute POL-2 observation, the reduction requires 35 GB of memory (in comparison with SCUBA-2 maps that may require up to 96 GB of memory).

The main consideration for POL-2 reductions is processing power. PCA calculations in makemap can be lengthy so fast processors with lots of cores are advised.

1.3 Before you start: software

This manual uses software from Starlink packages: Smurf [5], Kappa [8], Polpack[9] and Gaia [12]. Starlink software must be installed on your system, and Starlink aliases and environment variables must be defined before attempting to reduce any SCUBA-2 data (see Section 1.3.2).

1.3.1 Data formats

Data files for POL-2 are structurally the same as for SCUBA-2, and use the Starlink N-dimensional Data Format (NDF, see Jenness et al. 2014[16]), a hierarchical format which allows additional data and metadata to be stored within a single file. Kappa contains many commands for examining and manipulating NDF structures. The introductory sections of the Kappa document (SUN/95) contain much useful information on the contents of an NDF structure and how to manipulate them.

A single NDF structure describes a single data array with associated meta-data. NDFs are usually stored within files of type .sdf. In most cases (but not all), a single .sdf file will contain just one top-level NDF structure, and the NDF can be referred to simply by giving the name of the file (with or without the .sdf suffix). In many cases, a top-level NDF containing JCMT data will contain other ‘extension’ NDFs buried inside them at a lower level. For instance, raw files contain a number of NDF components, which store observation-specific data necessary for subsequent processing. The contents of these (and other NDF) files may be listed with Hdstrace. Each file holding raw JCMT data on disk is also known as a ‘sub-scan’.

The main components of any NDF structure are:

The Starlink Convert package contains commands fits2ndf and ndf2fits that allow interchange between FITS and NDF format.

1.3.2 Initialising Starlink

The commands and environment variables needed to start up the required Starlink packages (Smurf[5], Kappa, etc.) must first be defined. For C shells (csh, tcsh), the commands are:

  % setenv STARLINK_DIR <path to the starlink installation>
  % source $STARLINK_DIR/etc/login
  % source $STARLINK_DIR/etc/cshrc

before using any Starlink commands. For Bourne shells (sh, bash, zsh), the commands are as follows.

  % export STARLINK_DIR=<path to the starlink installation>
  % source $STARLINK_DIR/etc/profile

1.3.3 KAPPA and SMURF for data processing

The Starlink Sub-Millimetre User Reduction Facility package, or Smurf, contains the Dynamic Iterative Map-Maker, which will process SCUBA-2 time-series data into images (see SUN/258). Kappa, meanwhile, is an application package comprising general-purpose commands mostly for manipulating and visualising NDF data (see SUN/95). Before starting any data reduction it is necessary to initiate both Smurf and Kappa.

  % smurf
  % kappa

After entering the above commands, the help information for the two packages can be accessed by typing smurfhelp or kaphelp respectively in a terminal, or by using the showme facility to access the hypertext documentation. See Section 1.3.5 for more information.

Tip:
The .sdf extension on file names need not be specified when running most Starlink commands (the exception is Picard).

1.3.4 GAIA for viewing your images and vector maps

Images and vector maps can be displayed and analysed using Gaia (see SUN/214) – an interactive GUI-driven tool that incorporates facilities such as vector selection, vector binning, source detection, photometry and the ability to query and overlay on-line or local catalogue data.

  % gaia map.sdf

Alternatively, the Kappa package includes many visualisation commands that can be run from the shell command-line or incorporated easily into your own scripts—see Appendix “Classified KAPPA commands” in SUN/95.

1.3.5 How to get help





Help
command

Description

Usage




showme

If you know the name of the Starlink document you want to view, use showme. When run, it launches a new web page or tab displaying the hypertext version of the document.

% showme sun95




findme

findme searches Starlink documents for a keyword. When run, it launches a new web page or tab listing the results.

% findme kappa




docfind

docfind searches the internal list files for keywords. It then searches the document titles. The result is displayed using the Unix more command.

% docfind kappa




Run routines with prompts

You can run any routine with the option prompt after the command. This will prompt for every parameter available. If you then want a further description of any parameter, type ? at the relevant prompt.

% makemap prompt 
%̃ REF - Ref. NDF /!/ > ?




Google

A simple Google search such as “starlink kappa fitslist” will usually return links to the appropriate documents. However, the results may include links to out-of-date versions of the document hosted at non-Starlink sites. You should always look for results in "www.starlink.ac.uk/docs (or "www.starlink.ac.uk/devdocs for the current development version of the document).