8 Features of the library

This section describes some aspects of the behaviour of the CAT library.

8.1 Copying parameters and fields

Parameters and fields have a defined data type (_DOUBLE, _REAL etc.). Families of subroutines are available for transferring the values of parameters, columns and fields between a catalogue and Fortran variables in an application program invoking the CAT interface. These Fortran variables may be of a variety of types. The type of the column or parameter need not correspond to the type of the Fortran variable; the interface will automatically attempt a type conversion. However, depending on the types involved, a conversion may not be possible for some values. The matrix of possibilities is shown in Table 10. This matrix is used for all conversions: both GET and PUT operations for both columns (or fields) and parameters.

If a type conversion fails the appropriate exception value (null or locum, see Section 8.2, below) is substituted for the missing value. The error status is not set (that is, it remains ok) and no error message is generated. This procedure is followed consistently for all type conversions. It is adopted because, in practice, conversion failures are common.

The rules for converting to and from the _LOGICAL data type are as follows

From logical

 

• Conversion from LOGICAL to CHARACTER*$n$ gives ‘TRUE’ or ‘FALSE’ if $n$ is greater than or equal to 5 and ‘T’ or ‘F’ otherwise.
• Conversion from LOGICAL to INTEGER is defined such that true converts to 1 and false converts to 0.
• Conversion from LOGICAL to a numeric type except INTEGER is equivalent to converting LOGICAL to INTEGER and then INTEGER to the required numeric type.

To logical

 

• Any character string beginning with ‘T’ (upper or lower case) converts to true; any other string converts to false.
• Any even INTEGER converts to false; any odd INTEGER converts to true (this behaviour is specified for compatibility with HDS).
• Conversion for other numeric types is equivalent to conversion from the type to INTEGER and then from INTEGER to LOGICAL.

8.2 Null and locum values

8.2.1 Rationale and behaviour

Null values are used to represent a datum where no actual value is available. An example might be multi-colour photometry for a set of stars where measures for some colours are missing for some of the stars. Null values would be used to represent the missing values. Throughout CAT, and the applications that call it, nulls have the single, simple meaning that ‘no value is available for this datum’. It is possible to invent schemes where a set of null values are supported, each with a subtly different gradation of meaning (see, Roth et al.[6]). However, such schemes were adjudged un-necessary for manipulating astronomical catalogues and tables, and they are not supported.

The treatment of nulls in CAT was designed with the following assertions about null values in astronomical catalogues in mind:

• null values occur in existing catalogues. A wide variety of values are used to represent null values. Null values are common rather than rare,
• in general, it is always possible that a genuine datum will not be available when any field in a catalogue is being written; in general an arithmetic exception, such as $÷$ by zero, is always possible. It is not possible to proscribe exceptions when generating a column,
• conversely, for some columns, all the values permitted by the data type of the column will be required for genuine values, leaving no value available for representing the null value. An example might be a column of type _UBYTE used to record values generated by an astronomical instrument. The entire range of values permitted by this data type is 0 to 255, and if the instrument could generate genuine values in this entire range, there is no unused value left to represent the null case.

An additional, though less compelling, constraint was that it may be necessary to generate tables where the value of the null was specified, rather than adopting a fixed, pre-defined value. This feature may be required, for example, so that the table can be input to existing programs.

In order to meet these various requirements, the following behaviour was specified for null values in CAT:

• when a column is created, it may be specified as either supporting, or not supporting, null values,
• if a column supports nulls then a fixed, pre-defined null value will be used by default. These default nulls are the standard HDS ‘bad values’ for the various data types, where they exist. The HDS bad values are defined in SUN/39[10]. See in particular Sections 1.2 and 5.1. It is necessary to define a set of values for each operating system and type of computer that CAT runs on. Table 11 gives the values for some of the types currently used by Starlink. Note that the HDS parametric constants for the values are used to define the corresponding CAT types, rather than merely making their values the same. This approach reduces the possibility of incompatibilities arising if the values change,
• alternatively, for a column which supports nulls, the null value may be replaced with a specified value,
• if a column does not support nulls, exceptions can still occur when its fields are generated (as described above). When an exception occurs a locum value is written. The locum value for a column must be specified when a column which does not support nulls is created.

The difference between null and locum values is subtle and, at first, a little confusing. It may be summarized as follows:

• on writing a field there is no difference; if an exception occurs, a column with nulls generates a null and a column without nulls generates a locum,
• on reading a field (and this is the crux of the matter), CAT can differentiate a null from a genuine data value, but it cannot differentiate a locum: once written a locum is indistinguishable from a genuine datum.

Thus, generating a locum corresponds to CAT inventing data, and in theory (and often in practice) will invalidate the contents of the column. This invalidation is the reason why the value of a locum must be specified, rather than making a default available; a value can at least be chosen to minimize the damage.

8.2.2 Reporting the generation of null and locum values

The generation of nulls when writing a new field is common. Therefore, the generation of a null is not reported in any way: no error message is generated and no error status is set.

8.2.3 CAT subroutine interface

The NULL and EXCEPT attributes of a column (see Section 6.7) control the way that null or locum values are handled for that column. They are specified when the column is created and are immutable thereafter. The NULL attribute defines the type of null or locum which the column supports. The possibilities are:

• standard, HDS bad value (CAT__NULLD),
• explicitly specified null value (CAT__NULLS),
• locum (CAT__LOCUM).

If the standard HDS null value is being used then it is not necessary to associate its actual value with the column; knowing that the standard value is to be used is adequate. Conversely, if either:

• the value of the null has been specified explicitly,
• a locum value has been specified,

then the required value must be associated with the column. In either case, this value is stored in the exception attribute of the column.

In the routines to GET and PUT values through the CAT subroutine interface, null values are indicated by flags which appear as separate arguments in the calling sequences for the GET and PUT subroutines. However, the appropriate Starlink bad value is also substituted for the value in the actual data variable or array. The advantages of this approach are:

• the separate flag for nulls is convenient for processing in database applications,
• the embedded values in the data variable or array mean that arrays (or variables, though variables are less likely to be important in practice) can be passed straight into existing Starlink subroutines.

8.2.4 Handling nulls in expressions

The following rules apply to evaluating an expression in which one or more of the fields is null.

(1)

A numeric expression containing column names evaluates to null if any of the fields are null.

(2)

Logical expressions can evaluate to one of three states: true, false or null. However, the expression is only satisfied if the value is true.

(3)

A relational operation ($><=$, etc.) between two columns evaluates to null if either or both of the fields is null.

(4)

The logical operators AND, OR and NOT operate on a three-valued logic¹⁵. Their truth tables are shown in Table 12.

(5)

The parser provides the function NULL to detect null values. It returns the logical value .TRUE. if its argument is null and .FALSE. if its argument is not null.

8.2.5 Handling nulls in applications

Normally applications should observe the following rules when manipulating null values. If applications usually follow these rules they will all present consistent and predictable behaviour to their users. In a perfect world applications would always follow the rules. However, I recognize that some ad hoc applications may need to behave differently because of some idiosyncrasy of their required functionality. Nonetheless, applications should follow the rules unless there are very good reasons not to. The recommended rules are:

(1)

applications extracting a single column from a table (for example, to plot a histogram) should skip (that is, detect, but not process) null values. They may, however, count and report the number of null values encountered,

(2)

applications extracting two or more columns from a table (for example, to plot a scatter-gram) should skip (that is, detect, but not process) all rows where any of the extracted values is null. They may, however, count and report the number of null values and/or rows encountered,

8.2.6 Inquiring null values

An application may determine the null value specified for a column by inquiring the value of the EXCEPT attribute using routine CAT_TIQAC. The value is forced into type CHARACTER. If it is impossible to represent the value adequately in a character string then it cannot be made accessible to the application.

8.3 Storing and representing columns of angles

CAT provides special facilities for representing columns which contain angles. Usually angular columns are used to store celestial coordinates such as Right Ascension and Declination, though they can contain any angular measure. There are two requirements for the treatment of angles stored in astronomical catalogues:

• inside a program angles should be expressed in radians and stored in DOUBLE PRECISION or REAL variables for ease of processing,
• usually they are best displayed to a user in units of hours or degrees, subdivided into sexagesimal minutes and seconds. Alternatively, small angles might be best displayed as minutes or seconds of arc or time and represented to some number of places of decimals.

CAT provides both these facilities, as follows:

• when the value of a field in an angular column is obtained using CAT_EGT0D or CAT_EGT0R it is returned as DOUBLE PRECISION or REAL variable in radians,
• when the value of a field in an angular column is obtained formatted for display using CAT_EGT0F it is returned as a CHARACTER string containing the value expressed in hours, degrees, minutes or seconds and optionally formatted as a sexagesimal value.

In order for CAT to know that a column contains an angle it must satisfy the following two requirements.

• The data type must be DOUBLE PRECISION or REAL¹⁶,
• The units attribute of the column should be set to ‘RADIANS’ followed by an angular format specifier enclosed in curly brackets (‘{}’). The simplest forms of this angular format specifier are simply ‘HOURS’ and ‘DEGREES’ for hours and degrees respectively. Thus, examples of the UNITS attribute are:

  RADIANS{HOURS}

  to display the column in hours,

  RADIANS{DEGREES}

  to display the column in degrees.

  The angular format specifiers are described in full in the following section.

If the external display format attribute of the column, EXFMT, is explicitly set it should be set to a normal Fortran 77 format specifier corresponding to the data type of the column, for example ‘D16.8’ for a DOUBLE PRECISION column or ‘E14.6’ for a REAL column¹⁷.

When the catalogue is written fields should be written to angular columns using CAT_PUT0D or CAT_PUT0R with the angles expressed in radians.

8.3.1 Angular format specifiers

The angular format specifier forms part of the UNITS attribute for an angular column. The UNITS attribute for an angular column has the form:

The simplest angular format specifiers are ‘HOURS’ and ‘DEGREES’.

HOURS

will cause the angle to be displayed as hours, minutes and seconds, with the seconds displayed to one place of decimals,

DEGREES

will cause the angle to be displayed as degrees, minutes and seconds, with the seconds displayed as a whole number.

If the angular format specifier is omitted altogether and the UNITS attribute simply set to ‘RADIANS’ or ‘RADIANS{}’ then the angle will be interpreted exactly as though the angular format specifier had been ‘DEGREES’. There are additional simple angular format specifiers for displaying angles as minutes or seconds of arc or time to a specified number of decimal places:

ARCMIN.n

minutes of arc,

ARCSEC.n

seconds of arc,

TIMEMIN.n

minutes of time,

TIMESEC.n

seconds of time.

.n is the number of decimal places required. If .n is omitted then the value will be displayed as an integer number. Though these angular specifiers can be used to display any angle, obviously they are most likely to be useful for small angles.

These simple angular format specifiers will usually be adequate for representing columns of celestial coordinates. However, sometimes you might wish to specify a different representation for an angle. CAT accepts angular format specifiers which permit angles to be represented in a number of different formats. These specifiers are constructed from a selection from amongst the following elements:

The meaning of each of the individual elements is as follows.

I

Use the ISO standard separator for expressing times, a colon (‘:’), to separate hours or degrees, minutes and seconds.

B

Use a single blank space to separate hours or degrees, minutes and seconds.

L

Use a letter (h, d, m, or s, as appropriate) to separate hours or degrees, minutes and seconds.

+

Insert a plus sign (‘$+$’) before positive angles (a minus sign is, of course, always inserted before negative angles).

Z

Insert leading zeros before the hours, degrees, minutes or seconds. Hours, minutes and seconds are assumed to be two-digit numbers and degrees three-digit.

H

Express the angle in units of hours.

D

Express the angle in units of degrees.

M

If an M occurs when either H or D is present then it indicates that the hours or degrees are to be subdivided into sexagesimal minutes. If an M occurs when neither H nor D is present then it indicates that the units are minutes of either arc or time.

S

If an S occurs when an M is present then it indicates that the minutes are to be subdivided into sexagesimal seconds (the minutes may be either the actual units or themselves a sexagesimal subdivision of hours or degrees; see M above). If an S occurs when an M is not present then it indicates that the units are seconds of arc or time.

T

In the case where H and D are both absent and either or both of M and S are present then T indicates that the units are minutes or seconds of time. If it is omitted in this case then the units are minutes or seconds of arc. If either H or D is present then T is ignored.

.n

Display the least significant unit (seconds, minutes, degrees or hours, as appropriate) to n decimal places.

Any of the items may be omitted, down to and including a completely blank specifier.

The items can occur in any order, except that .n must occur last. However, for human readability I recommend that the items occur in the order:

If items are omitted the following defaults apply.

• If neither I, B nor L is specified then I is assumed.
• If + is omitted then positive angles are not preceded by a ‘$+$’ sign.
• If Z is omitted then leading zeros are omitted in the primary units (hours, degrees, minutes or seconds), but leading zeros are always included in any sexagesimal subdivisions.
• If none of H, D, M or S are specified then D is assumed (that is, the default units are degrees).
• If H or D are present but M is omitted then the hours or degrees are not subdivided into minutes.
• If M is present but S is omitted then the minutes are not subdivided into seconds.
• If S is present in addition to H or D but M is absent then S is ignored (this case is technically illegal).
• If .n is omitted then the least significant unit (seconds, minutes, degrees or hours, as appropriate) is displayed as a whole number, without any places of decimals.

Table 13 lists a number of examples of angular format specifiers which might be used to represent ‘large’ angles, such as celestial coordinates, together with examples of how they would represent an angle. Table 14 lists a number of examples of angular format specifiers which might be used to represent small angles, such as the great circle distance between two neighbouring objects or the angular size of an extended object, together with examples of how they would represent an angle.

The simple angular format specifiers, ‘HOURS’, ‘DEGREES’, ‘ARCMIN’, ‘ARCSEC’, ‘TIMEMIN’ and ‘TIMESEC’ are just synonyms for particular cases of the general specifiers. They are listed, together with the equivalent full specification in Table 15.

8.3.2 Displaying angles in radians

Displaying angles in units of hours, degrees, minutes or seconds, optionally formatted as sexagesimal values is usually the required behaviour. However, occasionally you may want to display angles as simple decimal numbers expressed in radians. CAT also provides this facility. It can be configured so that CAT_EGT0F returns an angle expressed in radians, written into a CHARACTER string using the Fortran 77 external format specifier for the column (defined by attribute EXFMT). In this case CAT is treating columns of angles just like any other column; CAT_EGT0F simply writes the value into a CHARACTER string using the external display format specifier.

To control whether angles are converted to hours, degrees, minutes or seconds and optionally formatted as sexagesimal values or simply written as a decimal number in radians you must set a configuration (or ‘tuning’) parameter in CAT. This parameter is called ‘ANGLE_LIST’ and is set using routine CAT_TUNES. Proceed as follows.

• to configure CAT to return angles as radians:
    CALL CAT\_TUNES (’ANGLE_LIST’, ’RADIANS’, STATUS)
• to configure CAT to return angles as hours, degrees, minutes or seconds, optionally formatted as sexagesimal values:
    CALL CAT_TUNES (’ANGLE_LIST’, ’SEXAGESIMAL’, STATUS)

Once one of these alternatives has been set it will remain in effect until another call is made to CAT_TUNES to reset it. The default if the representation is never explicitly set is that angles are converted to hours, degrees, minutes or seconds and optionally output as sexagesimal values.

You can inquire the current value of tuning parameter ‘ANGLE_LIST’ using routine CAT_TUNEG:

and TVALUE will be returned set to ‘RADIANS’ or ‘SEXAGESIMAL’, as appropriate.