B Specdre

 B.1 Introduction
 B.2 Specdre’s use of parameters
 B.3 Graphics
 B.4 Slicing data sets
 B.5 Cube manipulation
 B.6 Spectral fits
 B.7 Arc spectrum axis calibration
 B.8 References

B.1 Introduction

This appendix contains information on version 1.1 of the Specdre package (for SPEctroscopy Data Reduction) which has been added to Figaro. Sections B.6 and B.7 give examples of using several applications together.

Specdre is a package for spectroscopy data reduction and analysis. Some of the general features of the package are:

The topics addressed by the applications are mainly:

Specdre uses the NDF data access library, which allows you to specify sections rather than the whole data set. Also, for the special requirements of spectroscopy data reduction and analysis, an extension to the NDF format is used which stores additional information with the data, thus allowing much enhanced communication between Specdre applications.

B.2 Specdre’s use of parameters

Some parameters used by Specdre are common to several commands. The device parameter is sometimes associated with the global parameter GRAPHICS_DEVICE. When it is, it usually defaults. And these parameters are really global, in the sense that other packages may use and change them, too.

Where in and/or out are NDFs, they are mostly associated with the global DATA_ARRAY. The effect is that the default input is usually the output of the previous command.

info and dialog are always associated with SPECDRE_INFO and SPECDRE_DIALOG. These parameters control the amount of information and user interaction of many applications. Once info is switched off all applications will become quiet until the parameters are set true again.

varuse is a defaulted parameter to many applications, but not associated with a global parameter. By default it is true. Sometimes it has to be set false in order to ignore variance information in the input data.

Other parameters like start, step, end occur naturally in several applications. In some instances they may be scalars, in others vectors. Often their defaults are set by the application with knowledge of the data set at hand.

B.3 Graphics

The management of graphics output closely follows that of KAPPA. To make full use of the graphics capabilities, you will need to use some KAPPA commands; and you will find Pongo extremely useful to add almost anything to your plots. For normal use you can get along without KAPPA or Pongo. The actual plots are achieved through a combination of AGI, SGS, PGPLOT, and GKS. The graphics is done with PGPLOT, but the device is handled via AGI and SGS, and GKS is the low level package underlying them all. AGI stores information about the graphs produced in a data base. This data base tells co-operating applications where and on which device a plot was made and what its world coordinates were. Other packages use and update the same data base so that a consistent display administration can be achieved, even when different packages are used in turn. The overlay options in Specdre applications use this information and allow you to plot with Specdre in the right place on top of, for example, an image displayed in grey scale or colour with KAPPA.

Usually a command whose main task is to produce a plot, has a parameter device which is not prompted for and which is associated with the global parameter GRAPHICS_DEVICE. The value of this global parameter will be the name of a graphics device, and the named device will be used for the display. But you can always specify the device parameter on the command line, thus overriding the global parameter:

  ICL> specplot device=graphon
  ICL> specplot device=xw
  ICL> specplot device=ps_l

This will also change the global parameter so that next time you use the same device automatically.

There are other Specdre commands that may or may not use a graphics device. These will prompt for the device parameter and offer the null value as default. This can be accepted to avoid using a graphics device. When specifying parameters on the command line, device=! may not always work, but including the accept keyword will have the desired effect.

Some applications not only display graphics, but use a graphics display plus mouse and keyboard to conduct a dialogue with the user. This is usually optional and controlled by the dialog parameter. dialog is a character parameter. It is always allowed to be one of the letters {Y,N,T,F,y,n,t,f}. Sometimes it may also be G or g for graphics dialogue. y,t may or may not mean the same as g.

If you specify a printer or PostScript device, this may fail for the graphics dialogue case. But otherwise all plots can be sent to files that you print later. There is one important difference, you have only one screen, but the printer has many sheets of paper. Your plot may be in a number of printer files and each printout starts on a new page. If you have done overlay plots on the screen and want the same on the printer, then you can use as graphics device an Encapsulated PostScript device like epsf_l. You still get a number of files, but they can be merged into one (without form feed) using psmerge (cf. SUN/164). Usually the output is a complete PostScript file ready to be printed.

B.4 Slicing data sets

Specdre uses the NDF routines to access data in Starlink Data Files (SDF). This allows the user to specify sections of NDFs rather than whole (i.e. base) NDFs. Some applications need one-dimensional input data, but by themselves offer no means to take a subset of a larger or multi-dimensional data set. The desired effect can, however, be achieved by NDF. As an example, fitgauss will fit a spectrum only and rejects two-dimensional input. You still can use a 2-D data set by specifying a section thereof as input to fitgauss:

  ICL> fitgauss in=myndf(,5)               { 5-th row of 2-D input
  ICL> fitgauss in=myndf(5,)               { 5-th column
  ICL> fitgauss in=myndf(20:30,5)          { pixels 20 to 30 of 5-th row
  ICL> fitgauss in=myndf(6540.0:,5)        { pixels beyond coordinate 6540.0
  ICL> fitgauss in=myndf(6450.0~10.0,5)    { coordinate range 6540.0 +- 5.0

(For fitgauss the sub-setting within the row is not necessary, because it does its own masking of the given 1-D data.) It should be mentioned here that the NDF fed into an application need not be at the top level of its container file. Once your NDF got a Specdre Extension (Section 3.2.7) with fit results in it, you can e.g. plot the second fit parameter versus the row number:

  ICL> specplot in=myndf.more.specdre.results(2,1,)

B.5 Cube manipulation

A spectrum may be thought of as a one-dimensional data set. But spectroscopists are also aware of the two-dimensional space of sky position and might use time as an axis in data sets. So the data handling aspects of spectroscopy are more demanding than image processing – Specdre applications may encounter data with any dimensionality. Two-dimensional detectors are often used to take spectra and where observations are not very time-consuming three-dimensional data sets may be quite common.

Specdre can work on data with any dimensionality. The limit is 7-D due to the HDS data access routines. In practice the limit may be 6-D since one structure in the Specdre Extension has one dimension more than the main data array.

However, Specdre is about spectra. An N-D cube is taken as a set of spectra, each spectrum is a row or a column in the cube. A row extends along the first axis of the cube, while a column extends along any axis of the cube. In any cube Specdre assumes that there is exactly one spectroscopic axis, by default this is the first axis. The Specdre Extension specifies which axis is the spectroscopic one.

Specdre’s handling of N-D data falls into three categories.

subset is very similar to KAPPA’s ndfcopy or to taking an “on the fly” section as input to an application. The differences are that subset also takes subsets of NDFs in the Specdre Extension (v. 0.7) consistent with the subset of the main NDF, and that it removes “degenerate” axes. Consider the command

  ICL> subset image(5,1:10) row

When subset gets the input it is still an image of size 1 by 10. But in the output the degenerate axis has been removed so that it is also officially 1-D.

Where subset may remove axes, grow deliberately adds new axes – degenerate or genuine ones. So we could reverse the command above with

  ICL> grow row expand=[1,0] stapix=[1,0] endpix=[1,0] size=[1,0] ~
         new=y out=image

The zeros as second vector elements just show that the second axis of image matches the axis in row. expand shows which of the output axes are and are not in the input. Normally those new axes will of course not be degenerate, so size might be [5,0]. In that case row could be copied into any of the output columns, into one of them or repeatedly into a whole range of columns. The main idea of grow is that you assemble rows into images, images into cubes etc. So when new=n then you will copy into an existing file. The following puts two spectra and one image into a common cube. Whatever part of the cube is not copied to, remains filled with bad values.

  ICL> grow row5_1 expand=[0,1,1] stapix=[0,5,1] endpix=[0,5,1] ~
         size=[0,5,3] new=y out=cube_x_by_5_by_3
  ICL> grow plane2 expand=[0,0,1] stapix=[0,0,2] endpix=[0,0,2] ~
         new=n out=cube_x_by_5_by_3
  ICL> grow row3_to_5_3 expand=[0,1,1] stapix=[0,3,3] endpix=[0,5,3] ~
         new=n out=cube_x_by_5_by_3

If grow creates and expands new axes, xtract collapses existing axes. This is done by averaging the pixels along each collapsed axis. Note that this is an average and not a sum, which makes a big difference for the use of input variances and the meaning of output variances. Basically an average assumes that all values entering the mean are equal and scatter at random.

grow copies input into output of higher dimensionality, the common dimensions must match. fillcube is different. It copies input into output of the same dimensionality. Dimensions need not match, the copy is positioned in output by matching centre coordinates. Indeed the copy may not be contiguous in output. The output is an existing file, so you can fill it successively from different input files. This is mosaicing in N-D.

resample, too, plays a rôle in cube manipulation, since it can homogenise and linearise the coordinates along the spectroscopic axis. When used in mode=Cube it re-samples each row of a cube individually. Afterwards all rows have the same linear coordinate grid as expressed by the new vector of centres for the spectroscopic axis. Any spectroscopic values in the Specdre Extension are thus obsolete and removed. Sometimes it is necessary for other applications that grids are linear or that there is no array of spectroscopic values in the Specdre Extension.

B.6 Spectral fits

Specdre has a number of applications to fit analytical functions to spectral features. Two are line fit routines for Gauss and triangular profiles. These can fit up to six components at a time. The lines can be blended and the line parameters can be free, fixed, or tied to the corresponding (free) parameters of another component. A similar routine fits up to six diluted Planck curves. Finally, a polynomial fit can be performed, the order can be up to 7.

The fit routines can run with a (non-graphic) dialogue or not, they can display data and fit graphically at different stages of the fitting process (masking, guessing, fit residuals).

These fit routines work on one-dimensional data only. But you can pass an NDF section that is (part of) a row or column in an image or cube. For the fit only data inside the union of up to six masking intervals are used. The fit results go first of all to the standard output (the terminal), but can also be recorded in (appended to) an ASCII file. In addition fit results will always be stored in a results’ NDF in the Specdre Extension. Those results can be used to generate a model data set. You could then subtract that from the original data.

Here is an outline of a complex example showing how you might proceed with a long-slit spectrum (but note that the example will not work as presented since some parameters are omitted for brevity). Also, the telluric correction may not be the correct way of doing things. The purpose of the example is to illustrate how you can inter-operate Specdre and KAPPA applications.

  ICL> fitpoly  image(,1) mask1=[1,3,5] mask2=[2,4,6] comp=1             { 1)
  ICL> fitpoly  image(,2) mask1=[1,3,5] mask2=[2,4,6] comp=1
  ICL> fitpoly  image(,9) mask1=[1,3,5] mask2=[2,4,6] comp=1
  ICL> evalfit  image conti comp=1                                       { 2)
  ICL> div      image conti normal                                       { 3)
  ICL> fitgauss normal(,1) mask1=[2,4] mask2=[3,5] ncomp=3 comp=[2,3,4]  { 4)
  ICL> fitgauss normal(,2) mask1=[2,4] mask2=[3,5] ncomp=3 comp=[2,3,4]
  ICL> fitgauss normal(,9) mask1=[2,4] mask2=[3,5] ncomp=3 comp=[2,3,4]
  ICL> evalfit  normal tellur1 comp=3                                    { 5)
  ICL> sub      normal tellur1 stellar1
  ICL> cadd     tellur1 1.0 tellur2                                      { 6)
  ICL> div      normal tellur2 stellar2

B.7 Arc spectrum axis calibration

There are four applications to do a wavelength calibration, or at least a calibration into spectroscopic values anyway. You can use frequencies or photon energies if you feel like it. You can also use nanometre, micrometre or Ångström, as you please.

At the heart of this axis calibration is an algorithm written by Mills (1992) to automatically identify features in an arc spectrum. For this to work, you must have a data base of arc feature identifications rather than just a simple line list. You can use arcgendb to convert a line list (as distributed with Figaro) into a “feature data base”. Unfortunately the data base takes some time to build and is also rather big, 10 to 100 times bigger than the simple list. So there may be a point in taking only the relevant wavelength range from long line lists like Th-Ar and converting it into a feature data base.

With such a data base at your disposal, you still cannot run the auto-identifier. This is because the calibration procedure as performed by Figaro’s arc is broken up into three steps:

arclocat, arcident and arcdisp in general work not on a spectrum, but on an image or cube that has spectra in its rows (rows, not columns). arclocat dialog=g allows you to switch from one row to another as you please, arclocat dialog=n will scan through all rows in sequence. arcident will do an independent auto-identification on each spectrum, so it is suitable for a collapsed échellogram where successive extracted orders are in the rows of an image.

arcdisp dialog=g will work on each row in sequence, you can work on one row and proceed to the next in your own time. You can quit at any time; this will leave the file without spectroscopic values, but any improved feature identifications are kept. If you step through all rows, then the spectroscopic values will be kept as well. (You can have spectroscopic values in the Specdre Extension either for all rows or none at all!) You may want to set the label and unit for the spectroscopic values after arcdisp with KAPPA’s setlabel and setunits.

So far you have only succeeded in producing an array of calibrated spectroscopic values in the Specdre Extension of the arc spectrum. You will want to copy that array into the celestial observation you are actually interested in. This can be done with ndfcopy, provided the sky spectrum does have a Specdre Extension and does not have a SPECVALS component in it. That status can be achieved by two editext commands. Finally you may want to re-sample each spectrum (row in a cube) so that all use the same linear grid of spectroscopic values. To that end you use resamp mode=cube. Here is the whole procedure. (The commands are not complete; parameters are missing. Also there may be no Th-Ar line lists available for photon energies in the MeV range.)

  ICL> arcgendb thar.arc thar_arc
  ICL> arclocat dialog=n arc1
  ICL> arclocat dialog=g arc1
  ICL> arcident arc1 arc2 thar_arc
  ICL> arcdisp arc2
  ICL> setlabel arc2.more.specdre.specvals "particle energy"
  ICL> setunits arc2.more.specdre.specvals "log10(10**9*eV)"
  ICL> editext "create" sky1
  ICL> editext "delete specvals" sky1
  ICL> ndfcopy arc2.more.specdre.specvals sky1.more.specdre.specvals
  ICL> resamp cube sky1 sky2

B.8 References