16 Storing AST Objects in FITS Headers (FitsChans)

 16.1 The Native FITS Encoding
 16.2 The FitsChan Model
 16.3 Creating a FitsChan
 16.4 Addressing Cards in a FitsChan
 16.5 Writing Native Objects to a FitsChan
 16.6 Extracting Individual Cards from a FitsChan
 16.7 The Native FitsChan Output Format
 16.8 Adding Individual Cards to a FitsChan
 16.9 Adding Concatenated Cards to a FitsChan
 16.10 Reading Native Objects From a FitsChan
 16.11 Saving and Restoring Multiple Objects in a FitsChan
 16.12 Mixing Native Objects with Other FITS Cards
 16.13 Finding and Changing Cards in a FitsChan
 16.14 Source and Sink Routines for FitsChans

A FITS header is a sequence of 80-character strings, formatted according to particular rules defined by the Flexible Image Transport ?? (FITS). FITS27 is a widely-used standard for data interchange in astronomy and has also been adopted as a data processing format in some astronomical data reduction systems. The individual 80-character strings in a FITS header are usually called cards or header cards (for entirely anachronistic reasons).

A sequence of FITS cards appears as a header at the start of every FITS data file, and sometimes also at other points within it, and is used to provide ancillary information which qualifies or describes the main array of data stored in the file. As such, FITS headers are prime territory for storing information about the coordinate systems associated with data held in FITS files.

In this section, we will examine how to store information in FITS headers directly in the form of AST Objects—a process which is supported by a specialised class of ?? called a ??. Our discussion here will turn out to be a transitional step that emphasises the similarities between a FitsChan and a Channel (§15). At the same time, it will prepare us for the next section (§17), where we will examine how to use a FitsChan to tackle some of the more difficult problems that FITS headers can present.

16.1 The Native FITS Encoding

As it turns out, we are not the first to have thought of storing WCS information in FITS headers. In fact, the original FITS standard (1981 vintage) defined a set of header keywords for this purpose which have been widely used, although they have proved too limited for many practical purposes.

At the time of writing, a number of different ways of using FITS headers for storing WCS information are in use, most (although not all) based on the original standard. We will refer to these alternative ways of storing the information as FITS encodings but will defer a discussion of their advantages and limitations until the next section (§17).

Here, we will examine how to store AST Objects directly in FITS headers. In effect, this defines a new encoding, which we will term the native encoding. This is a special kind of encoding, because not only does it allow us to associate conventional WCS calibration information with FITS data, but it also allows any other information that can be expressed in terms of AST Objects to be stored as well. In fact, the native encoding provides us with facilities roughly analogous to those of the ?? (§15)—i.e. a lossless way of transferring AST Objects from program to program—but based on FITS headers instead of free-format text.

16.2 The FitsChan Model

I/O between AST Objects and FITS headers is supported by a specialised form of ?? called a ??. A FitsChan contains a buffer which may hold any number, including zero, of FITS header cards. This buffer forms a workspace in which you can assemble FITS cards and manipulate them before writing them out to a file.

By default, when a FitsChan is first created, it contains no cards and there are five ways of inserting cards into it:

(1)
You may add cards yourself, one at a time, using ?? (§16.8).
(2)
You may add cards yourself, supplying all cards concatenated into a single string, using ??. (§16.9).
(3)
You may write an AST ?? to the FitsChan (using ??), which will have the effect of creating new cards within the FitsChan which describe the Object (§16.5).
(4)
You may assign a value to the ?? attribute of the FitsChan. The value should be the path to a text file holding a set of FITS header cards, one per line. When the SourceFile value is set (using AST_SETC or ??). the file is opened and the headers copied from it into the FitsChan. The file is then immediately closed.
(5)
You may specify a source routine which reads data from some external store of FITS cards, just like the source associated with a basic Channel (§15.13). If you supply a source routine, it will be called when the FitsChan is created in order to fill it with an initial set of cards (§16.14).

There are also four ways of removing cards from a FitsChan:

(1)
You may delete cards yourself, one at a time, using ?? (§16.13).
(2)
You may read an AST Object from the FitsChan (using ??), which will have the effect of removing those cards from the FitsChan which describe the Object (§16.10).
(3)
You may assign a value to the FitsChan’s ?? attribute. When the FitsChan is deleted, any remaining headers are written out to a text file with path equal to the value of the SinkFile attribute.
(4)
Alternatively, You may specify a sink routine which writes data to some external store of FITS cards, just like the sink associated with a basic Channel (§15.14). If you supply a sink routine, it will be called when the FitsChan is deleted in order to write out any FITS cards that remain in it (§16.14). Note, the sink routine is not called if the SinkFile attribute has been set.

Note, in particular, that reading an AST Object from a FitsChan is destructive. That is, it deletes the FITS cards that describe the Object. The reason for this is explained in §17.5.

In addition to the above, you may also read individual cards from a FitsChan using the function ?? (which is not destructive). This is the main means of writing out FITS cards if you have not supplied a sink routine. AST_FINDFITS also provides a means of searching for particular FITS cards (by keyword, for example) and there are other facilities for overwriting cards when required (§16.13).

16.3 Creating a FitsChan

The ?? constructor function, ??, is straightforward to use:

        INCLUDE ’AST_PAR’
        INTEGER FITSCHAN, STATUS
  
        STATUS = 0
  
        ...
  
        FITSCHAN = AST_FITSCHAN( AST_NULL, AST_NULL, ’Encoding=NATIVE’, STATUS )

Here, we have omitted any source or sink functions by supplying the AST_NULL routine for the first two arguments (remember to include the AST_PAR include file which contains the required EXTERNAL statement for this routine). We have also initialised the FitsChan’s ?? attribute to NATIVE. This indicates that we will be using the native encoding (§16.1) to store and retrieve Objects. If this was left unspecified, the default would depend on the FitsChan’s contents. An attempt is made to use whatever encoding appears to have been used previously. For an empty FitsChan, the default is NATIVE, but it does no harm to be sure.

16.4 Addressing Cards in a FitsChan

Because a ?? contains an ordered sequence of header cards, a mechanism is needed for addressing them. This allows you to specify where new cards are to be added, for example, or which card is to be deleted.

This role is filled by the FitsChan’s integer ?? attribute, which gives the index of the current card in the FitsChan. You can nominate any card you like to be current, simply by setting a new value for the Card attribute, for example:

        INTEGER ICARD
  
        ...
  
        CALL AST_SETI( FITSCHAN, ’Card’, ICARD, STATUS )

where ICARD contains the index of the card on which you wish to operate next. Some functions will update the Card attribute as a means of advancing through the sequence of cards, when reading them for example, or to indicate which card matches a search criterion.

The default value for Card is one, which is the index of the first card. This means that you can “rewind” a FitsChan to access its first card by clearing the Card attribute:

        CALL AST_CLEAR( FITSCHAN, ’Card’, STATUS )

The total number of cards in a FitsChan is given by the integer ?? attribute. This is a read-only attribute whose value is automatically updated as you add or remove cards. It means you can address all the cards in sequence using a loop such as the following:

        DO 1 ICARD = 1, AST_GETI( FITSCHAN, ’Ncard’, STATUS )
           CALL AST_SETI( FITSCHAN, ’Card’, ICARD, STATUS )
           <access the current card>
   1    CONTINUE

However, it is usually possible to write slightly tidier loops based on the ?? function described later (§16.6 and §16.13).

If you set the Card attribute to a value larger than Ncard, the FitsChan is regarded as being positioned at its end-of-file. In this case there is no current card and an attempt to obtain a value for the Card attribute will always return the value Ncard + 1. When a FitsChan is empty, it is always at the end-of-file.

16.5 Writing Native Objects to a FitsChan

Having created an empty ?? (§16.3), you can write any AST ?? to it in the native encoding using the ?? function. Let us assume we are writing a ??,28 as follows:

        INTEGER NOBJ, SKYFRAME
  
        ...
  
        NOBJ = AST_WRITE( FITSCHAN, SKYFRAME, STATUS )

Since we have selected the native encoding (§16.1), there are no restrictions on the class of Object we may write, so AST_WRITE should always return a value of one, unless an error occurs. Unlike a basic ?? (§15.3), this write operation will not produce any output from our program. The FITS headers produced are simply stored inside the FitsChan.

After this write operation, the ?? attribute will be updated to reflect the number of new cards added to the FitsChan and the ?? attribute will point at the card immediately after the last one written. Since our FitsChan was initially empty, the Card attribute will, in this example, point at the end-of-file (§16.4).

The FITS standard imposes a limit of 68 characters on the length of strings which may be stored in a single header card. Sometimes, a description of an AST Object involves the use of strings which exceed this limit (e.g. a ?? title can be of arbitrary length). If this occurs, the long string will be split over two or more header cards. Each “continuation” card will have the keyword CONTINUE in columns 1 to 8, and will contain a space in column 9 (instead of the usual equals sign). An ampersand (“&”) is appended to the end of each of the strings (except the last one) to indicate that the string is continued on the next card.

Note, this splitting of long strings over several cards only occurs when writing AST Objects to a FitsChan using the AST_WRITE routine and the native encoding. If a long string is stored in a FitsChan using (for instance) the ?? or ?? routine, it will simply be truncated.

16.6 Extracting Individual Cards from a FitsChan

To examine the contents of the ?? after writing the ?? above (§16.5), we must write a simple loop to extract each card in turn and print it out. We must also remember to rewind the FitsChan first, e.g. using ??. The following loop would do:

        CHARACTER * ( 80 ) CARD
  
        ...
  
        CALL AST_CLEAR( FITSCHAN, ’Card’, STATUS )
  
   2    CONTINUE
        IF ( AST_FINDFITS( FITSCHAN, ’%f’, CARD, .TRUE., STATUS ) ) THEN
           WRITE ( *, ’(A)’ ) CARD
           GO TO 2
        END IF

Here, we have used the ?? function to find a FITS card by keyword. It is given a keyword template of “%f”, which matches any FITS keyword, so it always finds the current card, which it returns. Its fourth argument is set to .TRUE., to indicate that the ?? attribute should be incremented afterwards so that the following card will be found the next time around the loop. AST_FINDFITS returns .FALSE. when it reaches the end-of-file and this terminates the loop.

If we were storing the FITS headers in an output FITS file instead of printing them out, we might use a loop like this but replace the WRITE statement with a call to a suitable data access routine to store the header card. This would only be necessary if we had not provided a sink routine for the FitsChan (§16.14).

16.7 The Native FitsChan Output Format

If we print out the FITS header cards describing the ?? we wrote earlier (§16.5), we should obtain something like the following:

  COMMENT AST ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ AST
  COMMENT AST            Beginning of AST data for SkyFrame object             AST
  COMMENT AST ................................................................ AST
  BEGAST_A= ’SkyFrame’           / Description of celestial coordinate system
  NAXES_A =                    2 / Number of coordinate axes
  AX1_A   = ’        ’           / Axis number 1
  BEGAST_B= ’SkyAxis ’           / Celestial coordinate axis
  ENDAST_A= ’SkyAxis ’           / End of object definition
  AX2_A   = ’        ’           / Axis number 2
  BEGAST_C= ’SkyAxis ’           / Celestial coordinate axis
  ENDAST_B= ’SkyAxis ’           / End of object definition
  ISA_A   = ’Frame   ’           / Coordinate system description
  SYSTEM_A= ’FK4-NO-E’           / Celestial coordinate system type
  EPOCH_A =               1958.0 / Besselian epoch of observation
  ENDAST_C= ’SkyFrame’           / End of object definition
  COMMENT AST ................................................................ AST
  COMMENT AST               End of AST data for SkyFrame object                AST
  COMMENT AST ---------------------------------------------------------------- AST

As you can see, this resembles the information that would be written to a basic ?? to describe the same SkyFrame (§15.8), except that it has been formatted into 80-character header cards according to FITS conventions.

There are also a number of other differences worth noting:

(1)
There is no unnecessary information about default values provided for the benefit of the human reader. This is because the ?? attribute for a ?? defaults to 1, thus suppressing this information (c.f. §15.9). You can restore the information if you wish by setting Full to 0 or +1, in which case additional COMMENT cards will be generated to hold it.
(2)
The information is not indented, because FITS does not allow this. However, if you change the Full attribute to 0 or +1, comments will be included that are intended to help break up the sequence of headers and highlight its structure. This will probably only be of use if you are attempting to track down a problem by examining the FITS cards produced in detail.
(3)
The FITS keywords which appear to the left of the “ =” signs have additional characters (“_A”, “_B”, etc.) appended to them. This is done in order to make each keyword unique.

This last point is worth further comment and is necessary because the FITS standard only allows for certain keywords (such as COMMENT and HISTORY) to appear more than once. ?? therefore appends an arbitrary sequence of two characters to each new keyword it generates in order to ensure that it does not duplicate any already present in the FitsChan.

The main risk from not following this convention is that some software might ignore (say) all but the last occurrence of a keyword before passing the FITS headers on. Such an event is unlikely, but would obviously destroy the information present, so AST_WRITE enforces the uniqueness of the keywords it uses. The extra characters added are ignored when the information is read back.

As with a basic Channel, you can also suppress the comments produced in a FitsChan by setting the boolean (integer) ?? attribute to zero (§15.10). However, FITS headers are traditionally generously commented, so this is not recommended.

16.8 Adding Individual Cards to a FitsChan

To insert individual cards into a ??, prior to reading them back as Objects for example, you should use the ?? routine. You can insert a card in front of the current one as follows:

        CALL AST_PUTFITS( FITSCHAN, CARD, .FALSE., STATUS )

where the third argument of .FALSE. indicates that the current card should not be overwritten. Note that facilities are not provided by AST for formatting the card contents.

After inserting a card, the FitsChan’s ?? attribute points at the original Card, or at the end-of-file if the FitsChan was originally empty. Entering a sequence of cards is therefore straightforward. If CARDS is an array of character strings containing FITS header cards and NCARDS is the number of cards, then a loop such as the following will insert the cards in sequence into a FitsChan:

        INTEGER NCARD
        CHARACTER * ( 80 ) CARDS( NCARD )
  
        ...
  
        DO 3 ICARD = 1, NCARD
           CALL AST_PUTFITS( FITSCHAN, CARDS( ICARD ), .FALSE., STATUS )
   3    CONTINUE

Note that AST_PUTFITS enforces the validity of a FitsChan by rejecting any cards which do not adhere to the FITS standard. If any such cards are detected, an error will result.

16.9 Adding Concatenated Cards to a FitsChan

If you have all your cards concatenated together into a single long string, each occupying 80 characters (with no delimiters), you can insert them into a ?? in a single call using ??. This call first empties the supplied FitsChan of any existing cards, then inserts the new cards, and finally rewinds the FitsChan so that a subsequent call to ?? will start reading from the first supplied card. The AST_PUTCARDS routine uses ?? internally to interpret and store each individual card, and so the caveats in §16.8 should be read.

16.10 Reading Native Objects From a FitsChan

Once you have stored a FITS header description of an ?? in a ?? using the native encoding (§16.5), you can read it back using ?? in much the same way as with a basic ?? (§15.4). Similar comments about validating the Object you read also apply (§15.6). If you have just written to the FitsChan, you must remember to rewind it first:

        INTEGER OBJECT
  
        ...
  
        CALL AST_CLEAR( FITSCHAN, ’Card’, STATUS )
        OBJECT = AST_READ( FITSCHAN, STATUS )

An important feature of a FitsChan is that read operations are destructive. This means that if an Object description is found, it will be consumed by AST_READ which will remove all the cards involved, including associated COMMENT cards, from the FitsChan. Thus, if you write an Object to a FitsChan, rewind, and read the same Object back, you should end up with the original FitsChan contents. If you need to circumvent this behaviour for any reason, it is a simple matter to make a copy of a FitsChan using ?? (§4.12). If you then read from the copy, the original FitsChan will remain untouched.

After a read completes, the FitsChan’s ?? attribute identifies the card immediately following the last card read, or the end-of-file of there are no more cards.

Since the native encoding is being used, any long strings involved in the object description will have been split into two or more adjacent contuation cards when the Object was stored in the header using routine ??. The AST_READ routine reverses this process by concatenating any such adjacent continuation cards to re-create the original long string.

16.11 Saving and Restoring Multiple Objects in a FitsChan

When using the native FITS encoding, multiple Objects may be stored and all I/O operations are sequential. This means that you can simply write a sequence of Objects to a ??. After each write operation, the ?? attribute will be updated so that the next write appends the next ?? description to the previous one.

If you then rewind the FitsChan, you can read the Objects back in the original order. Reading them back will, of course, remove their descriptions from the FitsChan (§16.10) but the behaviour of the Card attribute is such that successive reads will simply return each Object in sequence.

The only thing that may require care, given that a FitsChan can always be addressed randomly by setting its Card attribute, is to avoid writing one Object on top of another. For obvious reasons, the Object descriptions in a FitsChan must remain separate if they are to make sense when read back.

16.12 Mixing Native Objects with Other FITS Cards

Of course, any real FITS header will contain other information besides AST Objects, if only the mandatory FITS cards that must accompany all FITS data. When FITS headers are read in from a real dataset, therefore, any native AST ?? descriptions will be inter-mixed with many other cards.

Because this is the normal state of affairs, the boolean (integer) ?? attribute for a ?? defaults to one. This means that when you read an Object From a FitsChan, any irrelevant cards will simply be skipped over until the start of the next Object description, if any, is found. If you start reading part way through an Object description, no error will result. The remainder of the description will simply be skipped.

Setting Skip to zero will change this behaviour to resemble that of a basic ?? (§15.12), where extraneous data are not permitted by default, but this will probably rarely be useful.

16.13 Finding and Changing Cards in a FitsChan

You can search for, and retrieve, particular cards in a ?? by keyword, using the function ??. This performs a search, starting at the current card, until it finds a card whose keyword matches the template you supply, or the end-of-file is reached.

If a suitable card is found, AST_FINDFITS returns the card’s contents and then sets the FitsChan’s ?? attribute either to identify the card found, or the one following it. The way you want the Card attribute to be set is indicated by the fourth (logical) argument to AST_FINDFITS. A value of .TRUE. is returned to indicate success. If a suitable card cannot be found, AST_FINDFITS returns a value of .FALSE. to indicate failure and sets the FitsChan’s Card attribute to the end-of-file.

Requesting that the Card attribute be set to indicate the card that AST_FINDFITS finds is useful if you want to replace that card with a new one, as in this example:

        CHARACTER * ( 80 ) NEWCARD
        LOGICAL JUNK
  
        ...
  
        JUNK = AST_FINDFITS( FITSCHAN, ’AIRMASS’, CARD, .FALSE., STATUS )
        CALL AST_PUTFITS( FITSCHAN, NEWCARD, .TRUE., STATUS )

Here, AST_FINDFITS is used to search for a card with the keyword AIRMASS. If the card is found, ?? then overwrites it with a new card. Otherwise, the Card attribute ends up pointing at the end-of-file and the new card is simply appended to the end of the FitsChan.

A similar approach can be used to delete selected cards from a FitsChan using ??, which deletes the current card:

        IF ( AST_FINDFITS( FITSCHAN, ’BSCALE’, CARD, .FALSE., STATUS ) ) THEN
           CALL AST_DELFITS( FITSCHAN, STATUS )
        END IF

This deletes the first card, if any, with the BSCALE keyword.

Requesting that AST_FINDFITS increments the Card attribute to identify the card following the one found is more useful when writing loops. For example, the following loop extracts each card whose keyword matches the template “CD%6d” (that is, “CD” followed by six decimal digits):

   4    CONTINUE
        IF ( AST_FINDFITS( FITSCHAN, ’CD%6d’, CARD, .TRUE., STATUS ) ) THEN
           <process the card’s contents>
           GO TO 4
        END IF

For further details of keyword templates, see the description of AST_FINDFITS in Appendix B.

16.14 Source and Sink Routines for FitsChans

The use of source and sink routines with a ?? is optional. This is because you can always arrange to explicitly fill a FitsChan with FITS cards (§16.8 and §16.9) and you can also extract any cards that remain and write them out yourself (§16.6) before you delete the FitsChan.

If you choose to use these routines, however, they behave in a very similar manner to those used by a ?? (§15.13 and §15.14). You supply these routines, as arguments to the constructor function ?? when you create the FitsChan (§16.3). The source routine is invoked implicitly at this point to fill the FitsChan with FITS cards and the FitsChan is then rewound, so that the first card becomes current. The sink routine is automatically invoked later, when the FitsChan is deleted, in order to write out any cards that remain in it.

The only real difference between the source and sink routines for a FitsChan and a basic Channel is that FITS cards are limited in length to 80 characters, so the choice of buffer size is simplified. This affects the way the card contents are passed, so the routines themselves are slightly different. The following is therefore the FitsChan equivalent of the Channel SOURCE routine given in §15.13:

        INTEGER FUNCTION FITSSOURCE( CARD, STATUS )
        CHARACTER * ( 80 ) CARD
        INTEGER STATUS
  
        READ( 1, ’(A)’, END = 99 ) CARD
        FITSSOURCE = 1
        RETURN
  
   99   FITSSOURCE = 0
        END

Here, the FITS card contents are returned via the CARD argument (the ?? routine should not be used) and the function returns 1 to indicate that a card has been read. A value of zero is returned if there are no more cards to read.

The sink routine for a FitsChan is also a little different (c.f. the SINK routine in §15.14), as follows:

        SUBROUTINE FITSSINK( CARD, STATUS )
        CHARACTER * ( 80 ) CARD
        INTEGER STATUS
  
        WRITE( 2, ’(A)’ ) CARD
  
        END

The contents of the FITS card being written are passed via the CARD argument (the ?? routine should not be used).

Of course, both of these examples assume that you are accessing text files. If this is not the case, then appropriate changes to the I/O statements would be needed. The details obviously depend on the format of the file you are handling, which need not necessarily be a true FITS file.

27http://fits.gsfc.nasa.gov/

28More probably, you would want to write a ??, but for purposes of illustration a SkyFrame contains a more manageable amount of data.