15 Saving and Restoring Objects (Channels)

 15.1 The Channel Model
 15.2 Creating a Channel
 15.3 Writing Objects to a Channel
 15.4 Reading Objects from a Channel
 15.5 Saving and Restoring Multiple Objects
 15.6 Validating Input
 15.7 Storing an ID String with an Object
 15.8 The Textual Output Format
 15.9 Controlling the Amount of Output
 15.10 Controlling Commenting
 15.11 Editing Textual Output
 15.12 Mixing Objects with other Text
 15.13 Reading Objects from Files
 15.14 Writing Objects to Files
 15.15 Reading and Writing Objects to other Places

Facilities are provided by the AST library for performing input and output (I/O) with any kind of Object. This means it is possible to write any Object into various external representations for storage, and then to read these representations back in, so as to restore the original Object. Typically, an Object would be written by one program and read back in by another.

We refer to “external representations” in the plural because AST is designed to function independently of any particular data storage system. This means that Objects may need converting into a number of different external representations in order to be compatible with (say) the astronomical data storage system in which they will reside.

In this section, we discuss the basic I/O facilities which support external representations based on a textual format referred to as the AST “native format”. These are implemented using a new kind of Object—a Channel. We will examine later how to use other representations, based on an XML format or on the use of FITS headers, for storing Objects. These are implemented using more specialised forms of Channel called XmlChan18) and FitsChan16).

15.1 The Channel Model

The best way to start thinking about a Channel is like a C file stream, and to think of the process of creating a Channel as that of opening a file and obtaining a FILE pointer. Subsequently, you can read and write Objects via the Channel.

This analogy is not quite perfect, however, because a Channel has, in principle, two “files” attached to it. One is used when reading, and the other when writing. These are termed the Channel’s source and sink respectively. In practice, the source and sink may both be the same, in which case the analogy with the C file stream is correct, but this need not always be so. It is not necessarily so with the basic Channel, as we will now see (§15.2).

15.2 Creating a Channel

The process of creating a Channel is straightforward. As you might expect, it uses the constructor function astChannel:

  #include "star/ast.h"
  AstChannel *channel;
  
  ...
  
  channel = astChannel( NULL, NULL, "" );

The first two arguments to astChannel specify the external source and sink that the Channel is to use. There arguments are pointers to C functions and we will examine their use in more detail later (§15.13 and §15.14).

In this very simple example we have supplied NULL pointers for both the source and sink functions. This requests the default behaviour, which means that textual input will be read from the program’s standard input stream (typically, this means your keyboard) while textual output will go to the standard output stream (typically appearing on your screen). On UNIX systems, of course, either of these streams can easily be redirected to files. This default behaviour can be changed by assigning values to the Channel’s SinkFile and/or SourceFile attributes. These attributes specify the paths to text files that are to be used in place of the standard input and output streams.

15.3 Writing Objects to a Channel

The process of saving Objects is very straightforward. You can simply write any Object to a Channel using the astWrite function, as follows:

  int nobj;
  AstObject *object;
  
  ...
  
  nobj = astWrite( channel, object );

The effect of this will be to produce a textual description of the Object which will appear, by default, on your program’s standard output stream. Any class of Object may be converted into text in this way.

astWrite returns a count of the number of Objects written. Usually, this will be one, unless the Object supplied cannot be represented. With a basic Channel all Objects can be represented, so a value of one will always be returned unless there has been an error. We will see later, however, that more specialised forms of Channel may impose restrictions on the kind of Object you can write (§17.2). In such cases, astWrite may return zero to indicate that the Object was not acceptable.

15.4 Reading Objects from a Channel

Before discussing the format of the output produced above (§15.3), let us consider how to read it back, so as to reconstruct the original Object. Naturally, we would first need to save the output in a file. We can do that either by using the SinkFile attribute, or (on UNIX systems), by redirecting standard output to a file using a shell command like:

  program1 >file

Within a subsequent program, we can read this Object back in by using the astRead function, having first created a suitable Channel:

  object = astRead( channel );

By default, this function will read from the standard input stream (the default source for a basic Channel), so we would need to ensure that our second program reads its input from the file in which the Object description is stored. On UNIX systems, we could again use a shell redirection command such as:

  program2 <file

Alternatively, we could have assigned a value to the SinkFile attribute before invoking astRead.

15.5 Saving and Restoring Multiple Objects

I/O operations performed on a basic Channel are sequential. This means that if you write more than one Object to a Channel, each new Object’s textual description is simply appended to the previous one. You can store any number of Objects in this way, subject only to the storage space you have available.

After you read an Object back from a basic Channel, the Channel is “positioned” at the end of that Object’s textual description. If you then perform another read, you will read the next Object’s textual description and therefore retrieve the next Object. This process may be repeated to read each Object in turn. When there are no more Objects to be read, astRead will return the value AST__NULL to indicate an end-of-file.

15.6 Validating Input

The pointer returned by astRead15.4) could identify any class of Object—this is determined entirely by the external data being read. If it is necessary to test for a particular class (say a Frame), this may be done as follows using the appropriate member of the astIsA <Class > family of functions:

  int ok;
  
  ...
  
  ok = astIsAFrame( object );

Note, however, that this will accept any Frame, so would be equally happy with a basic Frame or a SkyFrame. An alternative validation strategy would be to obtain the value of the Object’s Class attribute and then test this character string, as follows:

  #include <string.h>
  
  ...
  
  ok = !strcmp( astGetC( object, "Class" ), "Frame" );

This would only accept a basic Frame and would reject a SkyFrame.

15.7 Storing an ID String with an Object

Occasionally, you may want to store a number of Objects and later retrieve them and use each for a different purpose. If the Objects are of the same class, you cannot use the Class attribute to distinguish them when you read them back (c.f. §15.6). Although relying on the order in which they are stored is a possible solution, this becomes complicated if some of the Objects are optional and may not always be present. It also makes extending your data format in future more difficult.

To help with this, every AST Object has an ID attribute and an Ident attribute, both of which allows you, in effect, to attach a textual identification label to it. You simply set the ID or Ident attribute before writing the Object:

  astSet( object, "ID=Calibration" );
  nobj = astWrite( channel, object );

You can then test its value after you read the Object back:

  object = astRead( channel );
  if ( !strcmp( astGetC( object, "ID" ), "Calibration" ) ) {
     <the Calibration Object has been read>
  } else {
     <some other Object has been read>
  }

The only difference between the ID and Ident attributes is that the ID attribute is unique to a particular Object and is lost if, for example, you make a copy of the Object. The Ident attrubute, on the other hand, is transferred to the new Object when a copy is made. Consequently, it is safest to set the value of the ID attribute immediately before you perform the write.

15.8 The Textual Output Format

Let us now examine the format of the textual output produced by writing an Object to a basic Channel15.3). To give a concrete example, suppose the Object in question is a SkyFrame, written out as follows:

  AstSkyFrame *skyframe;
  
  ...
  
  nobj = astWrite( channel, skyframe );

The output should then look like the following:

   Begin SkyFrame  # Description of celestial coordinate system
  #   Title = "FK4 Equatorial Coordinates, no E-terms, Mean Equinox B1950.0, Epoch B1958.0"  # Title of coordinate system
      Naxes = 2  # Number of coordinate axes
  #   Domain = "SKY"  # Coordinate system domain
  #   Lbl1 = "Right Ascension"  # Label for axis 1
  #   Lbl2 = "Declination"  # Label for axis 2
  #   Uni1 = "hh:mm:ss.s"  # Units for axis 1
  #   Uni2 = "ddd:mm:ss"  # Units for axis 2
  #   Dir1 = 0  # Plot axis 1 in reverse direction (hint)
      Ax1 =  # Axis number 1
         Begin SkyAxis  # Celestial coordinate axis
         End SkyAxis
      Ax2 =  # Axis number 2
         Begin SkyAxis  # Celestial coordinate axis
         End SkyAxis
   IsA Frame  # Coordinate system description
      System = "FK4-NO-E"  # Celestial coordinate system type
      Epoch = 1958  # Besselian epoch of observation
  #   Eqnox = 1950  # Besselian epoch of mean equinox
   End SkyFrame

You will notice that this output is designed both for a human reader, in that it is formatted, and also to be read back by a computer in order to reconstruct the SkyFrame. In fact, this is precisely the way that astShow works (§4.4), this function being roughly equivalent to the following use of a Channel:

  channel = astChannel( NULL, NULL, "" );
  (void) astWrite( channel, object );
  channel = astAnnul( channel );

Some lines of the output start with a “#” comment character, which turns the rest of the line into a comment. These lines will be ignored when read back in by astRead. They typically contain default values, or values that can be derived in some way from the other data present, so that they do not actually need to be stored in order to reconstruct the original Object. They are provided purely for human information. The same comment character is also used to append explanatory comments to most output lines.

It is not sensible to attempt a complete description of this output format because every class of Object is potentially different and each can define how its own data should be represented. However, there are some basic rules, which mean that the following common features will usually be present:

(1)
Each Object is delimited by matching “Begin” and “End” lines, which also identify the class of Object involved.
(2)
Within each Object description, data values are represented by a simple “keyword  = value” syntax, with one value to a line.
(3)
Lines beginning “IsA” are used to mark the divisions between data belonging to different levels in the class hierarchy (Appendix A). Thus, “IsA Frame” marks the end of data associated with the Frame class and the start of data associated with some derived class (a SkyFrame in the above example). “IsA” lines may be omitted if associated data values are absent and no confusion arises.
(4)
Objects may contain other Objects as data. This is indicated by an absent value, with the description of the data Object following on subsequent lines.
(5)
Indentation is used to clarify the overall structure.

Beyond these general principles, the best guide to what a particular line of output represents will generally be the comment which accompanies it together with a general knowledge of the class of Object being described.

15.9 Controlling the Amount of Output

It is not always necessary for the output from astWrite15.3) to be human-readable, so a Channel has attributes that allow the amount of detail in the output to be controlled.

The first of these is the integer attribute Full, which controls the extent to which optional, commented out, output lines are produced. By default, Full is zero, and this results in the standard style of output (§15.8) where default values that may be helpful to humans are included. To suppress these optional lines, Full should be set to 1. This is most conveniently done when the Channel is created, so that:

  channel = astChannel( NULL, NULL, "Full=-1" );
  (void) astWrite( channel, skyframe );
  channel = astAnnul( channel );

would result in output containing only the essential information, such as:

   Begin SkyFrame  # Description of celestial coordinate system
      Naxes = 2  # Number of coordinate axes
      Ax1 =  # Axis number 1
         Begin SkyAxis  # Celestial coordinate axis
         End SkyAxis
      Ax2 =  # Axis number 2
         Begin SkyAxis  # Celestial coordinate axis
         End SkyAxis
   IsA Frame  # Coordinate system description
      System = "FK4-NO-E"  # Celestial coordinate system type
      Epoch = 1958  # Besselian epoch of observation
   End SkyFrame

In contrast, setting Full to +1 will result in additional output lines which will reveal every last detail of the Object’s construction. Often this will be rather more than you want, especially for more complex Objects, but it can sometimes help when debugging programs. This is how a SkyFrame appears at this level of detail:

   Begin SkyFrame  # Description of celestial coordinate system
  #   RefCnt = 1  # Count of active Object pointers
  #   Nobj = 1  # Count of active Objects in same class
   IsA Object  # Astrometry Object
  #   Nin = 2  # Number of input coordinates
  #   Nout = 2  # Number of output coordinates
  #   Invert = 0  # Mapping not inverted
  #   Fwd = 1  # Forward transformation defined
  #   Inv = 1  # Inverse transformation defined
  #   Report = 0  # Don’t report coordinate transformations
   IsA Mapping  # Mapping between coordinate systems
  #   Title = "FK4 Equatorial Coordinates, no E-terms, Mean Equinox B1950.0, Epoch B1958.0"  # Title of coordinate system
      Naxes = 2  # Number of coordinate axes
  #   Domain = "SKY"  # Coordinate system domain
  #   Lbl1 = "Right Ascension"  # Label for axis 1
  #   Lbl2 = "Declination"  # Label for axis 2
  #   Sym1 = "RA"  # Symbol for axis 1
  #   Sym2 = "Dec"  # Symbol for axis 2
  #   Uni1 = "hh:mm:ss.s"  # Units for axis 1
  #   Uni2 = "ddd:mm:ss"  # Units for axis 2
  #   Dig1 = 7  # Individual precision for axis 1
  #   Dig2 = 7  # Individual precision for axis 2
  #   Digits = 7  # Default formatting precision
  #   Fmt1 = "hms.1"  # Format specifier for axis 1
  #   Fmt2 = "dms"  # Format specifier for axis 2
  #   Dir1 = 0  # Plot axis 1 in reverse direction (hint)
  #   Dir2 = 1  # Plot axis 2 in conventional direction (hint)
  #   Presrv = 0  # Don’t preserve target axes
  #   Permut = 1  # Axes may be permuted to match
  #   MinAx = 2  # Minimum number of axes to match
  #   MaxAx = 2  # Maximum number of axes to match
  #   MchEnd = 0  # Match initial target axes
  #   Prm1 = 1  # Axis 1 not permuted
  #   Prm2 = 2  # Axis 2 not permuted
      Ax1 =  # Axis number 1
         Begin SkyAxis  # Celestial coordinate axis
  #         RefCnt = 1  # Count of active Object pointers
  #         Nobj = 2  # Count of active Objects in same class
         IsA Object  # Astrometry Object
  #         Label = "Angle on Sky"  # Axis Label
  #         Symbol = "delta"  # Axis symbol
  #         Unit = "ddd:mm:ss"  # Axis units
  #         Digits = 7  # Default formatting precision
  #         Format = "dms"  # Format specifier
  #         Dirn = 1  # Plot in conventional direction
         IsA Axis  # Coordinate axis
  #         Format = "dms"  # Format specifier
  #         IsLat = 0  # Longitude axis (not latitude)
  #         AsTime = 0  # Display values as angles (not times)
         End SkyAxis
      Ax2 =  # Axis number 2
         Begin SkyAxis  # Celestial coordinate axis
  #         RefCnt = 1  # Count of active Object pointers
  #         Nobj = 2  # Count of active Objects in same class
         IsA Object  # Astrometry Object
  #         Label = "Angle on Sky"  # Axis Label
  #         Symbol = "delta"  # Axis symbol
  #         Unit = "ddd:mm:ss"  # Axis units
  #         Digits = 7  # Default formatting precision
  #         Format = "dms"  # Format specifier
  #         Dirn = 1  # Plot in conventional direction
         IsA Axis  # Coordinate axis
  #         Format = "dms"  # Format specifier
  #         IsLat = 0  # Longitude axis (not latitude)
  #         AsTime = 0  # Display values as angles (not times)
         End SkyAxis
   IsA Frame  # Coordinate system description
      System = "FK4-NO-E"  # Celestial coordinate system type
      Epoch = 1958  # Besselian epoch of observation
  #   Eqnox = 1950  # Besselian epoch of mean equinox
   End SkyFrame

15.10 Controlling Commenting

Another way of controlling output from a Channel is via the boolean (integer) Comment attribute, which controls whether comments are appended to describe the purpose of each value. Comment has the value 1 by default but, if set to zero, will suppress these comments. This is normally appropriate only if you wish to minimise the amount of output, for example:

  astSet( channel, "Full=-1, Comment=0" );
  nobj = astWrite( channel, skyframe );

might result in the following more compact output:

   Begin SkyFrame
      Naxes = 2
      Ax1 =
         Begin SkyAxis
         End SkyAxis
      Ax2 =
         Begin SkyAxis
         End SkyAxis
   IsA Frame
      System = "FK4-NO-E"
      Epoch = 1958
   End SkyFrame

15.11 Editing Textual Output

The safest advice about editing the textual output from astWrite (or astShow) is “don’t!”—unless you know what you are doing.

Having given that warning, however, it is sometimes possible to make changes to the text, or even to write entire Object descriptions from scratch, and to read the results back in to construct new Objects. Normally, simple changes to numerical values are safest, but be aware that this is a back door method of creating Objects, so you are on your own! There are a number of potential pitfalls. In particular:

15.12 Mixing Objects with other Text

By default, when you use astRead to read from a basic Channel15.4), it is assumed that you are reading a stream of text containing only AST Objects, which follow each other end-to-end. If any extraneous input data are encountered which do not appear to form part of the textual description of an Object, then an error will result. In particular, the first input line must identify the start of an Object description, so you cannot start reading half way through an Object.

Sometimes, however, you may want to store AST Object descriptions intermixed with other textual data. You can do this by setting the Channel’s boolean (integer) Skip attribute to 1. This will cause every read to skip over extraneous data until the start of a new AST Object description, if any, is found. So long as your other data do not mimic the appearance of an AST Object description, the two sets of data can co-exist.

For example, by setting Skip to 1, the following complete C program will read all the AST Objects whose descriptions appear in the source of this document, ignoring the other text. astShow is used to display those found:

  #include "star/ast.h"
  main() {
     AstChannel *channel;
     AstObject *object;
  
     channel = astChannel( NULL, NULL, "Skip=1" );
     while ( ( object = astRead( channel ) ) != AST__NULL ) {
        astShow( object );
        object = astAnnul( object );
     }
     channel = astAnnul( channel );
  }

15.13 Reading Objects from Files

Thus far, we have only considered the default behaviour of a Channel in reading and writing Objects through a program’s standard input and output streams. We will now consider how to access Objects stored in files more directly.

The simple approach is to use the SinkFile and SourceFile attributes of the Channel. For instance, the following will read a pair of Objects from a text file called “fred.txt”:

  astSet( channel, "SourceFile=fred.txt" );
  obj1 = astRead( channel );
  obj2 = astRead( channel );
  astClear( channel, "SourceFile" );

Note, the act of clearing the attribute tells AST that no more Objects are to be read from the file and so the file is then closed. If the attribute is not cleared, the file will remain open and further Objects can be read from it. The file will always be closed when the Channel is deleted.

This simple approach will normally be sufficient. However, because the AST library is designed to be used from more than one language, it has to be a little careful about reading and writing to files. This is due to incompatibilities that may exist between the file I/O facilities provided by different languages. If such incompatibilities prevent the above simple system being used, we need to adopt a system that off-loads all file I/O to external code.

What this means in practice is that if the above simple approach cannot be used, you must instead provide some simple C functions that perform the actual transfer of data to and from files and similar external data stores. The functions you provide are supplied as the source and/or sink function arguments to astChannel when you create a Channel (§15.2). An example is the best way to illustrate this.

Consider the following simple function called Source. It reads a single line of text from a C input stream and returns a pointer to it, or NULL if there is no more input:

  #include <stdio.h>
  #define LEN 200
  static FILE *input_stream;
  
  const char *Source( void ) {
     static char buffer[ LEN + 2 ];
     return fgets( buffer, LEN + 2, input_stream );
  }

Note that the input stream is a static variable which we will also access from our main program. This might look something like this (omitting error checking for brevity):

  /* Open the input file. */
  input_stream = fopen( "infile.ast", "r" );
  
  /* Create a Channel and read an Object from it. */
  channel = astChannel( Source, NULL, "" );
  object = astRead( channel );
  
  ...
  
  /* Annul the Channel and close the file when done. */
  channel = astAnnul( channel );
  (void) fclose( input_stream );

Here, we first open the required input file, saving the resulting FILE pointer. We then pass a pointer to our Source function as the first argument to astChannel when creating a new Channel. When we read an Object from this Channel with astRead, the Source function will be called to obtain the textual data from the file, the end-of-file being detected when this function returns NULL.

Note, if a value is set for the SourceFile attribute, the astRead function will ignore any source function specified when the Channel was created.

15.14 Writing Objects to Files

As for reading, writing Objects to files can be done in two different ways. Again, the simple approach is to use the SinkFile attribute of the Channel. For instance, the following will write a pair of Objects to a text file called “fred.txt”:

  astSet( channel, "SinkFile=fred.txt" );
  nobj = astWrite( channel, object1 );
  nobj = astWrite( channel, object2 );
  astClear( channel, "SinkFile" );

Note, the act of clearing the attribute tells AST that no more output will be written to the file and so the file is then closed. If the attribute is not cleared, the file will remain open and further Objects can be written to it. The file will always be closed when the Channel is deleted.

If the details of the language’s I/O system on the computer you are using means that the above approach cannot be used, then we can write a Sink function, that writes a line of output text to a file, and use it in basically the same way as the Source function in the previous section (§15.13):

  static FILE *output_stream;
  
  void Sink( const char *line ) {
     (void) fprintf( output_stream, "%s\n", line );
  }

Note that we must supply the final newline character ourselves.

In this case, our main program would supply a pointer to this Sink function as the second argument to astChannel, as follows:

  /* Open the output file. */
  output_stream = fopen( "outfile.ast", "w" );
  
  /* Create a Channel and write an Object to it. */
  channel = astChannel( Source, Sink, "" );
  nobj = astWrite( channel, object );
  
     ...
  
  /* Annul the Channel and close the file when done. */
  channel = astAnnul( channel );
  (void) fclose( output_stream );

Note that we can specify a source and/or a sink function for the Channel, and that these may use either the same file, or different files according to whether we are reading or writing. AST has no knowledge of the underlying file system, nor of file positioning. It just reads and writes sequentially. If you wish, for example, to reposition a file at the beginning in between reads and writes, then this can be done directly (and completely independently of AST) using standard C functions.

If an error occurs in your source or sink function, you can communicate this to the AST library by setting its error status to any error value using astSetStatus4.15). This will immediately terminate the read or write operation.

Note, if a value is set for the SinkFile attribute, the astWrite function will ignore any sink function specified when the Channel was created.

15.15 Reading and Writing Objects to other Places

It should be obvious from the above (§15.13 and §15.14) that a Channel’s source and sink functions provide a flexible means of intercepting textual data that describes AST Objects as it flows in and out of your program. In fact, you might like to regard a Channel simply as a filter for converting AST Objects to and from a stream of text which is then handled by your source and sink functions, where the real I/O occurs.

This gives you the ability to store AST Objects in virtually any data system, so long as you can convert a stream of text into something that can be stored (it need no longer be text) and retrieve it again. There is generally no need to retain comments. Other possibilities, such as inter-process and network communication, could also be implemented via source and sink functions in basically the same way.