15 Saving and Restoring Objects (Channels)

 15.1 The Channel Model
 15.2 Creating a Channel
 15.3 Writing Objects to a Channel
 15.4 Reading Objects from a Channel
 15.5 Saving and Restoring Multiple Objects
 15.6 Validating Input
 15.7 Storing an ID String with an Object
 15.8 The Textual Output Format
 15.9 Controlling the Amount of Output
 15.10 Controlling Commenting
 15.11 Editing Textual Output
 15.12 Mixing Objects with other Text
 15.13 Reading Objects from Files
 15.14 Writing Objects to Files
 15.15 Reading and Writing Objects to other Places

Facilities are provided by the AST library for performing input and output (I/O) with any kind of ??. This means it is possible to write any Object into various external representations for storage, and then to read these representations back in, so as to restore the original Object. Typically, an Object would be written by one program and read back in by another.

We refer to “external representations” in the plural because AST is designed to function independently of any particular data storage system. This means that Objects may need converting into a number of different external representations in order to be compatible with (say) the astronomical data storage system in which they will reside.

In this section, we discuss the basic I/O facilities which support external representations based on a textual format referred to as the AST “native format”. These are implemented using a new kind of Object—a ??. We will examine later how to use other representations, based on an XML format or on the use of FITS headers, for storing Objects. These are implemented using more specialised forms of Channel called ?? (§18) and ?? (§16).

15.1 The Channel Model

The best way to start thinking about a ?? is like a Fortran I/O unit (also represented by an integer, as it happens) and to think of the process of creating a Channel as the combined process of allocating a unit number and attaching it to a file by opening the file on that unit. Subsequently, you can read and write Objects via the Channel.

This analogy is not quite perfect, however, because a Channel has, in principle, two “files” attached to it. One is used when reading, and the other when writing. These are termed the Channel’s source and sink respectively. In practice, the source and sink may both be the same, in which case the analogy with the Fortran I/O unit is correct, but this need not always be so. It is not necessarily so with the basic Channel, as we will now see (§15.2).

15.2 Creating a Channel

The process of creating a ?? is straightforward. As you might expect, it uses the constructor function ??:

        INCLUDE ’AST_PAR’
        INTEGER CHANNEL, STATUS
  
        STATUS = 0
  
        ...
  
        CHANNEL = AST_CHANNEL( AST_NULL, AST_NULL, ’ ’, STATUS )

The first two arguments to AST_CHANNEL specify the external source and sink that the Channel is to use. There arguments are the names of Fortran subroutines and we will examine their use in more detail later (§15.13 and §15.14).

In this very simple example we have supplied the name of the null routine AST_NULL26 for both the source and sink routines. This requests the default behaviour, which means that textual input will be read from the program’s standard input stream (typically, this means your keyboard) while textual output will go to the standard output stream (typically appearing on your screen). On UNIX systems, of course, either of these streams can easily be redirected to files.

15.3 Writing Objects to a Channel

The process of saving Objects is very straightforward. You can simply write any ?? to a ?? using the ?? function, as follows:

        INTEGER NOBJ, OBJECT
  
        ...
  
        NOBJ = AST_WRITE( CHANNEL, OBJECT, STATUS )

The effect of this will be to produce a textual description of the Object which will appear, by default, on your program’s standard output stream. Any class of Object may be converted into text in this way.

AST_WRITE returns a count of the number of Objects written. Usually, this will be one, unless the Object supplied cannot be represented. With a basic Channel all Objects can be represented, so a value of one will always be returned unless there has been an error. We will see later, however, that more specialised forms of Channel may impose restrictions on the kind of Object you can write (§17.2). In such cases, AST_WRITE may return zero to indicate that the Object was not acceptable.

15.4 Reading Objects from a Channel

Before discussing the format of the output produced above (§15.3), let us consider how to read it back, so as to reconstruct the original ??. Naturally, we would first need to save the output in a file. We can do that either by using the ?? attribute, or (on UNIX systems), by redirecting standard output to a file using a shell command like:

  program1 >file

Within a subsequent program, we can read this Object back in by using the ?? function, having first created a suitable ??:

        OBJECT = AST_READ( CHANNEL, STATUS )

By default, this function will read from the standard input stream (the default source for a basic Channel), so we would need to ensure that our second program reads its input from the file in which the Object description is stored. On UNIX systems, we could again use a shell redirection command such as:

  program2 <file

Alternatively, we could have assigned a value to the SinkFile attribute before invoking AST_READ.

15.5 Saving and Restoring Multiple Objects

I/O operations performed on a basic ?? are sequential. This means that if you write more than one ?? to a Channel, each new Object’s textual description is simply appended to the previous one. You can store any number of Objects in this way, subject only to the storage space you have available.

After you read an Object back from a basic Channel, the Channel is “positioned” at the end of that Object’s textual description. If you then perform another read, you will read the next Object’s textual description and therefore retrieve the next Object. This process may be repeated to read each Object in turn. When there are no more Objects to be read, ?? will return the value AST__NULL to indicate an end-of-file.

15.6 Validating Input

The pointer returned by ?? (§15.4) could identify any class of ??—this is determined entirely by the external data being read. If it is necessary to test for a particular class (say a ??), this may be done as follows using the appropriate member of the ?? family of functions:

        LOGICAL OK
  
        ...
  
        OK = AST_ISAFRAME( OBJECT, STATUS )

Note, however, that this will accept any Frame, so would be equally happy with a basic Frame or a ??. An alternative validation strategy would be to obtain the value of the Object’s ?? attribute and then test this character string, as follows:

        OK = AST_GETC( OBJECT, ’Class’, STATUS ) .EQ. ’Frame’

This would only accept a basic Frame and would reject a SkyFrame.

15.7 Storing an ID String with an Object

Occasionally, you may want to store a number of Objects and later retrieve them and use each for a different purpose. If the Objects are of the same class, you cannot use the ?? attribute to distinguish them when you read them back (c.f. §15.6). Although relying on the order in which they are stored is a possible solution, this becomes complicated if some of the Objects are optional and may not always be present. It also makes extending your data format in future more difficult.

To help with this, every AST ?? has an ?? attribute and an ?? attribute, both of which allows you, in effect, to attach a textual identification label to it. You simply set the ID or Ident attribute before writing the Object:

        CALL AST_SET( OBJECT, ’ID=Calibration’, STATUS )
        NOBJ = AST_WRITE( CHANNEL, OBJECT, STATUS )

You can then test its value after you read the Object back:

        OBJECT = AST_READ( CHANNEL, STATUS )
        IF ( AST_GETC( OBJECT, ’ID’, STATUS ) .EQ. ’Calibration’ ) THEN
           <the Calibration Object has been read>
        ELSE
           <some other Object has been read>
        END IF

The only difference between the ID and Ident attributes is that the ID attribute is unique to a particular Object and is lost if, for example, you make a copy of the Object. The Ident attrubute, on the other hand, is transferred to the new Object when a copy is made. Consequently, it is safest to set the value of the ID attribute immediately before you perform the write.

15.8 The Textual Output Format

Let us now examine the format of the textual output produced by writing an ?? to a basic ?? (§15.3). To give a concrete example, suppose the Object in question is a ??, written out as follows:

        INTEGER SKYFRAME
  
        ...
  
        NOBJ = AST_WRITE( CHANNEL, SKYFRAME, STATUS )

The output should then look like the following:

   Begin SkyFrame  # Description of celestial coordinate system
  #   Title = "FK4 Equatorial Coordinates, no E-terms, Mean Equinox B1950.0, Epoch B1958.0"  # Title of coordinate system
      Naxes = 2  # Number of coordinate axes
  #   Domain = "SKY"  # Coordinate system domain
  #   Lbl1 = "Right Ascension"  # Label for axis 1
  #   Lbl2 = "Declination"  # Label for axis 2
  #   Uni1 = "hh:mm:ss.s"  # Units for axis 1
  #   Uni2 = "ddd:mm:ss"  # Units for axis 2
  #   Dir1 = 0  # Plot axis 1 in reverse direction (hint)
      Ax1 =  # Axis number 1
         Begin SkyAxis  # Celestial coordinate axis
         End SkyAxis
      Ax2 =  # Axis number 2
         Begin SkyAxis  # Celestial coordinate axis
         End SkyAxis
   IsA Frame  # Coordinate system description
      System = "FK4-NO-E"  # Celestial coordinate system type
      Epoch = 1958  # Besselian epoch of observation
  #   Eqnox = 1950  # Besselian epoch of mean equinox
   End SkyFrame

You will notice that this output is designed both for a human reader, in that it is formatted, and also to be read back by a computer in order to reconstruct the SkyFrame. In fact, this is precisely the way that ?? works (§4.4), this routine being roughly equivalent to the following use of a Channel:

        CHANNEL = AST_CHANNEL( AST_NULL, AST_NULL, ’ ’, STATUS )
        NOBJ = AST_WRITE( CHANNEL, OBJECT, STATUS )
        CALL AST_ANNUL( CHANNEL, STATUS )

Some lines of the output start with a “#” comment character, which turns the rest of the line into a comment. These lines will be ignored when read back in by ??. They typically contain default values, or values that can be derived in some way from the other data present, so that they do not actually need to be stored in order to reconstruct the original Object. They are provided purely for human information. The same comment character is also used to append explanatory comments to most output lines.

It is not sensible to attempt a complete description of this output format because every class of Object is potentially different and each can define how its own data should be represented. However, there are some basic rules, which mean that the following common features will usually be present:

(1)
Each Object is delimited by matching “Begin” and “End” lines, which also identify the class of Object involved.
(2)
Within each Object description, data values are represented by a simple “keyword  = value” syntax, with one value to a line.
(3)
Lines beginning “IsA” are used to mark the divisions between data belonging to different levels in the class hierarchy (Appendix A). Thus, “IsA ??” marks the end of data associated with the Frame class and the start of data associated with some derived class (a SkyFrame in the above example). “IsA” lines may be omitted if associated data values are absent and no confusion arises.
(4)
Objects may contain other Objects as data. This is indicated by an absent value, with the description of the data Object following on subsequent lines.
(5)
Indentation is used to clarify the overall structure.

Beyond these general principles, the best guide to what a particular line of output represents will generally be the comment which accompanies it together with a general knowledge of the class of Object being described.

15.9 Controlling the Amount of Output

It is not always necessary for the output from ?? (§15.3) to be human-readable, so a ?? has attributes that allow the amount of detail in the output to be controlled.

The first of these is the integer attribute ??, which controls the extent to which optional, commented out, output lines are produced. By default, Full is zero, and this results in the standard style of output (§15.8) where default values that may be helpful to humans are included. To suppress these optional lines, Full should be set to 1. This is most conveniently done when the Channel is created, so that:

        CHANNEL = AST_CHANNEL( AST_NULL, AST_NULL, ’Full=-1’, STATUS )
        NOBJ = AST_WRITE( CHANNEL, SKYFRAME, STATUS )
        CALL AST_ANNUL( CHANNEL, STATUS )

would result in output containing only the essential information, such as:

   Begin SkyFrame  # Description of celestial coordinate system
      Naxes = 2  # Number of coordinate axes
      Ax1 =  # Axis number 1
         Begin SkyAxis  # Celestial coordinate axis
         End SkyAxis
      Ax2 =  # Axis number 2
         Begin SkyAxis  # Celestial coordinate axis
         End SkyAxis
   IsA Frame  # Coordinate system description
      System = "FK4-NO-E"  # Celestial coordinate system type
      Epoch = 1958  # Besselian epoch of observation
   End SkyFrame

In contrast, setting Full to +1 will result in additional output lines which will reveal every last detail of the ??’s construction. Often this will be rather more than you want, especially for more complex Objects, but it can sometimes help when debugging programs. This is how a ?? appears at this level of detail:

   Begin SkyFrame  # Description of celestial coordinate system
  #   RefCnt = 1  # Count of active Object pointers
  #   Nobj = 1  # Count of active Objects in same class
   IsA Object  # Astrometry Object
  #   Nin = 2  # Number of input coordinates
  #   Nout = 2  # Number of output coordinates
  #   Invert = 0  # Mapping not inverted
  #   Fwd = 1  # Forward transformation defined
  #   Inv = 1  # Inverse transformation defined
  #   Report = 0  # Don’t report coordinate transformations
   IsA Mapping  # Mapping between coordinate systems
  #   Title = "FK4 Equatorial Coordinates, no E-terms, Mean Equinox B1950.0, Epoch B1958.0"  # Title of coordinate system
      Naxes = 2  # Number of coordinate axes
  #   Domain = "SKY"  # Coordinate system domain
  #   Lbl1 = "Right Ascension"  # Label for axis 1
  #   Lbl2 = "Declination"  # Label for axis 2
  #   Sym1 = "RA"  # Symbol for axis 1
  #   Sym2 = "Dec"  # Symbol for axis 2
  #   Uni1 = "hh:mm:ss.s"  # Units for axis 1
  #   Uni2 = "ddd:mm:ss"  # Units for axis 2
  #   Dig1 = 7  # Individual precision for axis 1
  #   Dig2 = 7  # Individual precision for axis 2
  #   Digits = 7  # Default formatting precision
  #   Fmt1 = "hms.1"  # Format specifier for axis 1
  #   Fmt2 = "dms"  # Format specifier for axis 2
  #   Dir1 = 0  # Plot axis 1 in reverse direction (hint)
  #   Dir2 = 1  # Plot axis 2 in conventional direction (hint)
  #   Presrv = 0  # Don’t preserve target axes
  #   Permut = 1  # Axes may be permuted to match
  #   MinAx = 2  # Minimum number of axes to match
  #   MaxAx = 2  # Maximum number of axes to match
  #   MchEnd = 0  # Match initial target axes
  #   Prm1 = 1  # Axis 1 not permuted
  #   Prm2 = 2  # Axis 2 not permuted
      Ax1 =  # Axis number 1
         Begin SkyAxis  # Celestial coordinate axis
  #         RefCnt = 1  # Count of active Object pointers
  #         Nobj = 2  # Count of active Objects in same class
         IsA Object  # Astrometry Object
  #         Label = "Angle on Sky"  # Axis Label
  #         Symbol = "delta"  # Axis symbol
  #         Unit = "ddd:mm:ss"  # Axis units
  #         Digits = 7  # Default formatting precision
  #         Format = "dms"  # Format specifier
  #         Dirn = 1  # Plot in conventional direction
         IsA Axis  # Coordinate axis
  #         Format = "dms"  # Format specifier
  #         IsLat = 0  # Longitude axis (not latitude)
  #         AsTime = 0  # Display values as angles (not times)
         End SkyAxis
      Ax2 =  # Axis number 2
         Begin SkyAxis  # Celestial coordinate axis
  #         RefCnt = 1  # Count of active Object pointers
  #         Nobj = 2  # Count of active Objects in same class
         IsA Object  # Astrometry Object
  #         Label = "Angle on Sky"  # Axis Label
  #         Symbol = "delta"  # Axis symbol
  #         Unit = "ddd:mm:ss"  # Axis units
  #         Digits = 7  # Default formatting precision
  #         Format = "dms"  # Format specifier
  #         Dirn = 1  # Plot in conventional direction
         IsA Axis  # Coordinate axis
  #         Format = "dms"  # Format specifier
  #         IsLat = 0  # Longitude axis (not latitude)
  #         AsTime = 0  # Display values as angles (not times)
         End SkyAxis
   IsA Frame  # Coordinate system description
      System = "FK4-NO-E"  # Celestial coordinate system type
      Epoch = 1958  # Besselian epoch of observation
  #   Eqnox = 1950  # Besselian epoch of mean equinox
   End SkyFrame

15.10 Controlling Commenting

Another way of controlling output from a ?? is via the boolean (integer) ?? attribute, which controls whether comments are appended to describe the purpose of each value. Comment has the value 1 by default but, if set to zero, will suppress these comments. This is normally appropriate only if you wish to minimise the amount of output, for example:

        CALL AST_SET( CHANNEL, ’Full=-1, Comment=0’, STATUS )
        NOBJ = AST_WRITE( CHANNEL, SKYFRAME, STATUS )

might result in the following more compact output:

   Begin SkyFrame
      Naxes = 2
      Ax1 =
         Begin SkyAxis
         End SkyAxis
      Ax2 =
         Begin SkyAxis
         End SkyAxis
   IsA Frame
      System = "FK4-NO-E"
      Epoch = 1958
   End SkyFrame

15.11 Editing Textual Output

The safest advice about editing the textual output from ?? (or ??) is “don’t!”—unless you know what you are doing.

Having given that warning, however, it is sometimes possible to make changes to the text, or even to write entire ?? descriptions from scratch, and to read the results back in to construct new Objects. Normally, simple changes to numerical values are safest, but be aware that this is a back door method of creating Objects, so you are on your own! There are a number of potential pitfalls. In particular:

15.12 Mixing Objects with other Text

By default, when you use ?? to read from a basic ?? (§15.4), it is assumed that you are reading a stream of text containing only AST Objects, which follow each other end-to-end. If any extraneous input data are encountered which do not appear to form part of the textual description of an ??, then an error will result. In particular, the first input line must identify the start of an Object description, so you cannot start reading half way through an Object.

Sometimes, however, you may want to store AST Object descriptions intermixed with other textual data. You can do this by setting the Channel’s boolean (integer) ?? attribute to 1. This will cause every read to skip over extraneous data until the start of a new AST Object description, if any, is found. So long as your other data do not mimic the appearance of an AST Object description, the two sets of data can co-exist.

For example, by setting Skip to 1, the following complete Fortran program will read all the AST Objects whose descriptions appear in the source of this document, ignoring the other text. ?? is used to display those found:

        INCLUDE ’AST_PAR’
        INTEGER CHANNEL, OBJECT, STATUS
  
        STATUS = 0
        CHANNEL = AST_CHANNEL( AST_NULL, AST_NULL, ’Skip=1’, STATUS )
   1    OBJECT = AST_READ( CHANNEL, STATUS )
        IF ( OBJECT .NE. AST__NULL ) THEN
           CALL AST_SHOW( OBJECT, STATUS )
           CALL AST_ANNUL( OBJECT, STATUS )
           GO TO 1
        END IF
        CALL AST_ANNUL( CHANNEL, STATUS )
        END

15.13 Reading Objects from Files

Thus far, we have only considered the default behaviour of a ?? in reading and writing Objects through a program’s standard input and output streams. We will now consider how to access Objects stored in files more directly.

The simple approach is to use the ?? and ?? attributes of the Channel. For instance, the following will read a pair of Objects from a text file called “fred.txt”:

     CALL AST_SET( CHANNEL, ’SourceFile=fred.txt’, STATUS )
     OBJ1 = AST_READ( CHANNEL, STATUS )
     OBJ2 = AST_READ( CHANNEL, STATUS )
     CALL AST_CLEAR( CHANNEL, ’SourceFile’, STATUS )

Note, the act of clearing the attribute tells AST that no more Objects are to be read from the file and so the file is then closed. If the attribute is not cleared, the file will remain open and further Objects can be read from it. The file will always be closed when the Channel is deleted.

This simple approach will normally be sufficient. However, because the AST library is designed to be used from more than one language, it has to be a little careful about reading and writing to files. This is due to incompatibilities that may exist between the file I/O facilities provided by different languages. If such incompatibilities prevent the above simple system being used, we need to adopt a system that off-loads all file I/O to external code.

What this means in practice is that if the above simple approach cannot be used, you must instead provide some simple Fortran routines that perform the actual transfer of data to and from files and similar external data stores. The routines you provide are supplied as the source and/or sink routine arguments to ?? when you create a Channel (§15.2). An example is the best way to illustrate this.

Consider the following simple subroutine called SOURCE. It reads a single line of text from a Fortran I/O unit and then calls ?? to pass it to the AST library, together with its length. It sets this length to be negative if there is no more input:

        SUBROUTINE SOURCE( STATUS )
        INTEGER STATUS
        CHARACTER * ( 200 ) BUFFER
  
        READ( 1, ’(A)’, END = 99 ) BUFFER
        CALL AST_PUTLINE( BUFFER, LEN( BUFFER ), STATUS )
        RETURN
  
   99   CALL AST_PUTLINE( BUFFER, -1, STATUS )
        END

Our main program might then look something like this (omitting error checking for brevity):

        EXTERNAL SOURCE
  
        ...
  
  *  Open the input file.
        OPEN( UNIT = 1, FILE = ’infile.ast’, STATUS = ’OLD’ )
  
  *  Create the Channel and read an Object from it.
        CHANNEL = AST_CHANNEL( SOURCE, AST_NULL, ’ ’, STATUS )
        OBJECT = AST_READ( CHANNEL, STATUS )
  
        ...
  
  *  Annul the Channel and close the file when done.
        CALL AST_ANNUL( CHANNEL, STATUS )
        CLOSE( 1 )

Here, we first open the required input file. We then pass the name of our SOURCE routine as the first argument to AST_CHANNEL when creating a new Channel (ensuring that SOURCE also appears in an EXTERNAL statement). When we read an ?? from this Channel using ??, the SOURCE routine will be called to obtain the textual data from the file, the end-of-file being detected when it yields a negative line length.

Note, if a value is set for the SourceFile attribute, the AST_READ function will ignore any source routine specified when the Channel was created.

15.14 Writing Objects to Files

As for reading, writing Objects to files can be done in two different ways. Again, the simple approach is to use the ?? attribute of the ??. For instance, the following will write a pair of Objects to a text file called “fred.txt”:

     CALL AST_SET( CHANNEL, ’SinkFile=fred.txt’, STATUS )
     NOBJ = AST_WRITE( CHANNEL, OBJECT1, STATUS )
     NOBJ = AST_WRITE( CHANNEL, OBJECT2, STATUS )
     CALL AST_CLEAR( CHANNEL, ’SinkFile’, STATUS )

Note, the act of clearing the attribute tells AST that no more output will be written to the file and so the file is then closed. If the attribute is not cleared, the file will remain open and further Objects can be written to it. The file will always be closed when the Channel is deleted.

If the details of the language’s I/O system on the computer you are using means that the above approach cannot be used, then we can write a SINK routine, that obtains a line of output text from the AST library by calling ?? and then writes it to a file. We can use this in basically the same way as the SOURCE routine in the previous section (§15.13):

        SUBROUTINE SINK( STATUS )
        INTEGER L, STATUS
        CHARACTER * ( 200 ) BUFFER
  
        CALL AST_GETLINE( BUFFER, L, STATUS )
        IF ( L .GT. 0 ) WRITE( 2, ’(A)’ ) BUFFER( : L )
  
        END

In this case, our main program would supply the name of this SINK routine as the second argument to ?? (ensuring that it also appears in an EXTERNAL statement), as follows:

        EXTERNAL SINK
  
        ...
  
  *  Open the output file.
        OPEN( UNIT = 2, FILE = ’outfile.ast’, STATUS = ’NEW’ )
  
  *  Create a Channel and write an Object to it.
        CHANNEL = AST_CHANNEL( SOURCE, SINK, ’ ’, STATUS )
        NOBJ = AST_WRITE( CHANNEL, OBJECT, STATUS )
  
        ...
  
  *  Annul the Channel and close the file when done.
        CALL AST_ANNUL( CHANNEL, STATUS )
        CLOSE( 2 )

Note that we can specify a source and/or a sink routine for the Channel, and that these may use either the same file, or different files according to whether we are reading or writing. AST has no knowledge of the underlying file system, nor of file positioning. It just reads and writes sequentially. If you wish, for example, to reposition a file at the beginning in between reads and writes, then this can be done directly (and completely independently of AST) using standard Fortran statements.

If an error occurs in your source or sink routine, you can communicate this to the AST library by setting the STATUS argument to any error value. This will immediately terminate the read or write operation.

Note, if a value is set for the SinkFile attribute, the ?? function will ignore any sink routine specified when the Channel was created.

15.15 Reading and Writing Objects to other Places

It should be obvious from the above (§15.13 and §15.14) that a ??’s source and sink routines provide a flexible means of intercepting textual data that describes AST Objects as it flows in and out of your program. In fact, you might like to regard a Channel simply as a filter for converting AST Objects to and from a stream of text which is then handled by your source and sink routines, where the real I/O occurs.

This gives you the ability to store AST Objects in virtually any data system, so long as you can convert a stream of text into something that can be stored (it need no longer be text) and retrieve it again. There is generally no need to retain comments. Other possibilities, such as inter-process and network communication, could also be implemented via source and sink functions in basically the same way.

26Note that AST_NULL (one underscore) is a routine name and is distinct from AST__NULL (two underscores) which is a null ?? pointer. Since we are passing the name of one routine to another routine, AST_NULL would normally have to appear in a Fortran EXTERNAL statement. In this example, however, a suitable statement is already present in the AST_PAR include file.