4 Data Formats

 4.1 World coordinate systems
 4.2 Catalogue formats

GAIA can access images in various different data formats. The ones that you are mostly likely to encounter are FITS images and NDF files, though there are numerous other possibilities, including the IRAF format and old Figaro DST files.

In GAIA (and to anticipate the recipes of Part II) you can check which formats are currently available by clicking on the File Type: button in the file-picker window (see the recipe in Section 8). Be careful that you do not inadvertently set the button to specify just one of the formats; it is usually best left set to ‘any’.

The basic GAIA image display facilities read FITS files. The more astronomical functions usually require files in Starlink’s NDF format. However, any necessary format conversions are performed automatically and invisibly ‘on-the-fly’ by applications in the CONVERT package (see SUN/55[4]), and you will not normally be aware that they are happening. It is possible to configure the range of formats currently available to CONVERT. You are unlikely to need to make such a configuration, but if you choose to do so then SSN/20[22] gives the requisite details. However, you should ensure that FITS continues to be one of the formats available (because the basic image display functions read FITS files directly).

A full description of the NDF and FITS formats is beyond the scope of this cookbook. The NDF (n-dimensional Data Format; see SUN/33[19]) is the native Starlink format. The FITS4 (Flexible Image Transport System) format is in widespread use in astronomy. There is a brief introduction to the FITS format in SC/5[7].

In both the NDF and FITS formats, and indeed other common astronomical formats, the information stored in the file includes more than just the array of values for the two-dimensional image. Various other header or auxiliary information describing and annotating the image is also included. Typical auxiliary information might include: the instrument and telescope used, the date and time of observation, details of the instrumental set up etc. In the jargon of computer science such header information is often called ‘metadata’, though this term is rarely used in astronomy. Sometimes you may wish to examine this auxiliary information.

In GAIA (and again anticipating Part II) you can display the headers of a FITS image by clicking on the View menu (on the menu-bar along the top of the main window) and choosing the Fits Header… item (see the recipe in Section 8). Some alternative ways of listing the header information for the NDF, FITS and a couple of other formats are given in Appendix B.2 of SC/6[16].

Data files usually contain just a single image. However, both the FITS and NDF formats allow files to contain more than one image, and occasionally you might encounter such a file. Section 5.2 includes some notes on how to access a given image inside a file in this case.

The format of a data file is often indicated by specifying a ‘file-type’ at the end of the file-name. NDF files have file-type ‘.sdf’. When accessing NDF files with GAIA the file-type may optionally be omitted. FITS files usually have a file-type of ‘.fit’ or ‘.fits’.

4.1 World coordinate systems

Astronomical images often contain a so-called World Coordinate System (WCS). The WCS is a prescription for converting pixel position indices (that is, indices into the two-dimensional array representing the image) into physical units. In practice for direct images of a region of sky the WCS will usually transform pixel indices into some celestial coordinate system, such as the Right Ascension and Declination for some equinox and epoch. However, the underlying concept of a WCS is much more general. For example, it is possible to have a WCS which transforms pixel indices into x, y positions in micron on the face of the CCD chip which detected the image. Alternatively, for spectroscopic data one index might be transformed into a wavelength in Ångström. The WCS is stored with the other auxiliary information for the image, such as the instrument and telescope used, the date and time of observation, etc.

GAIA handles all the details of manipulating the WCS automatically (ultimately by using the Starlink AST library). All that you really need to know is that an image might or might not contain a WCS, and an image with a suitable WCS can be annotated and examined in terms of celestial coordinates rather than pixel indices.

The actual way in which the WCS details are stored in the auxiliary information for an image is rather arcane. There are several conventions in use for FITS files, none of which are standard. There are proposals for FITS WCS standards, but these are still under discussion at the time of writing. If you are interested to find out more, the AST library is documented in SUN/210[20] and SUN/211[21] and there are three papers describing the FITS WCS proposals (papers I5, II6 and III7). You should be aware, however, that all of these documents contain far more detail than you need to know in order to use GAIA, and, moreover, they are not for the faint-hearted.

4.2 Catalogue formats

The native format in which GAIA reads and writes the catalogues and tables which it plots as image overlays is the so-called Tab-Separated Table (TST) format, which is described in SSN/75[6]. GAIA can also read and write catalogues in the FITS tables, STL and ASCII_HEAD formats. GAIA differentiates between these different formats by using the file-type at the end of the file-name. Brief notes on the individual formats and their required file-types follow.

FITS tables
a variant of the FITS format for storing catalogues and tables. Accepted file types: .FIT .fit .FITS .fits .GSC .gsc
the Small Text List format used by CURSA (see SUN/190[5]). Accepted file types: .TXT .txt
the ‘ASCII Header’ format used by the EXTRACTOR package (see SUN/226[3]). Accepted file types: .asc .ASC .lis .LIS

Mixed capitalisations, such as ‘.Fits’ are also recognised. GAIA interprets all other file formats as indicating TST format catalogues (but note that CURSA requires TST format catalogues to have a file-type of .TAB or .tab).

4The original FITS format was proposed by Wells et al.[23] in 1981. However, it has been developed and enhanced over the years. The FITS standard is now maintained and documented by the FITS Support Office of the Astrophysics Data Facility at the NASA Goddard Space Flight Center (see URL: http://fits.gsfc.nasa.gov/fits_home.html). Though FITS is basically an astronomical format it is sometimes mentioned in books about standard image formats. See, for example, Graphics File Formats by Kay and Levine[12].