In this section, we begin by describing a number of low-level structures, the basic building blocks which can be used to build NDFs and other composite structures.
The $<$POLYNOMIAL$>$ structure is used for storing the coefficients of an $n$-dimensional ordinary or Chebyshev polynomial, or the coefficients of a cubic B-spline. The value of the [VARIANT] component defines which of these types of polynomial is stored. Evaluation of the polynomial yields a $<$float$>$ scalar result.
In the following descriptions NAXIS is the dimensionality of the [DATA_ARRAY] and therefore the dimensionality of the space over which the polynomial function is defined.
Component Name | TYPE | Brief Description |
[VARIANT] | $<$_CHAR$>$ | ‘SIMPLE’ |
[DATA_ARRAY] | $<$farray$>$ | coefficients |
[VARIANCE] | $<$farray$>$ | variance of the coefficients |
The [DATA_ARRAY] is the $NAXIS$-dimensional array containing the $NAXIS$-dimensional polynomial coefficients. The function is evaluated as: $$VALUE=\sum DATA\text{\_}ARRAY\left({I}_{1},{I}_{2},\dots ,{I}_{NAXIS}\right)\prod _{j=1}^{NAXIS}{AXIS}_{j}{\phantom{\rule{0.3em}{0ex}}}^{{I}_{j}-1}$$
summed over all elements of the DATA_ARRAY.
Here, AXIS${}_{n}$ is the co-ordinate value along axis $n$, and NAXIS${}_{n}$ is the dimension of axis $n$.
Component Name | TYPE | Brief Description |
[VARIANT] | $<$_CHAR$>$ | ‘CHEBYSHEV’ |
[DATA_ARRAY] | $<$farray$>$ | Chebyshev coefficients |
[VARIANCE] | $<$farray$>$ | variance of the coefficients |
[TMIN(NAXIS)] | $<$float$>$ | lower bound of co-ordinate range |
[TMAX(NAXIS)] | $<$float$>$ | upper bound of co-ordinate range |
The [DATA_ARRAY] contains the NAXIS-dimensional array of Chebyshev-polynomial coefficients. The function is evaluated as:
$$VALUE=\sum DATA\text{\_}ARRAY\left({I}_{1},{I}_{2},\dots ,{I}_{NAXIS}\right)\prod _{j=1}^{NAXIS}T\left({I}_{j-1},{S}_{j}\right)$$
summed over all elements of the DATA_ARRAY, where $T\left(m,x\right)$ is the Chebyshev polynomial of order $m$ evaluated at $x$ (in the range $-$1 to $+$1), and
$${S}_{n}=(2\ast {AXIS}_{n}-\left(TMAX\left(n\right)+TMIN\left(n\right)\right)/\left(TMAX\left(n\right)-TMIN\left(n\right)\right)$$
If the polynomial is evaluated outside this interval, the result is bad, and set to the magic value.
Component Name | TYPE | Brief Description |
[VARIANT] | $<$_CHAR$>$ | ‘BSPLINE’ |
[DATA_ARRAY] | $<$farray$>$ | coefficients |
[VARIANCE] | $<$farray$>$ | variance of the coefficients |
[SCALE] | $<$float$>$ | normalisation |
[KNOTS] | $<$farray$>$ | co-ordinates of the knots |
$$VALUE=1/SCALE\phantom{\rule{1em}{0ex}}\phantom{\rule{1em}{0ex}}\sum DATA\underline{\phantom{\rule{1em}{0ex}}\phantom{\rule{1em}{0ex}}}ARRAY\left({I}_{1},{I}_{2},\dots ,{I}_{NAXIS}\right)\prod _{j=1}^{NAXIS}{B}_{j}\left({AXIS}_{j}\right)$$
where ${B}_{j}$(AXIS${}_{j}$) is the normalised cubic B-spline along the $j$th axis with the knots recorded in [KNOTS], and SCALE is the value of component [SCALE].
We will now describe the various structures for arrays of numbers introduced in Section 3.3. An $<$ARRAY$>$ structure has a number of variants.
Component Name | TYPE | Brief Description |
[VARIANT] | $<$_CHAR$>$ | ‘SIMPLE’ |
[ORIGIN(NAXIS)] | $<$integer$>$ | origin of the data array |
[DATA] | $<$narray$>$ | actual value at every pixel |
This information may be communicated to access routines via the upper and lower bounds on each array dimension, so that the Fortran dimensions would be
[ORIGIN] overcomes a number of quite serious problems, including the old chestnut “does the first pixel start at (1,1) or (0,0)?”, because the origin is specified explicitly. Also negative pixel indices become available, and therefore an application is freed from having every array co-located at the first pixel, and it may extend arrays in any direction. This property is particularly useful for storing data which require negative indices for a natural representation, such as Fourier-transform data, where zero often needs to be in the centre of the array; or image data after (say) rotation about any point. Storing a pixel origin also enable applications to cut out parts of a data array yet maintain the array’s coordinate system (it can be re-combined with the original if necessary). Since the coordinate system is preserved, all the axis scaling is also preserved. This means, for instance, that applications do not have to re-compute the polynomial coefficients in an $<$AXIS$>$ structure when the array size is changed.
A scaled $<$ARRAY$>$ is sometimes referred to as block floating point. When dealing with large-format data, disk space is a major consideration; significant space may be saved by storing data in a 16-bit form via the block floating-point format. Scaling may also be used as a means of implementing global scalar changes in the array’s values without the need to change the array, or to move to a more expensive storage type (e.g. to $<$float$>$ from $<$_WORD$>$) in order to work with very small or very large values.
Component Name | TYPE | Brief Description |
[VARIANT] | $<$_CHAR$>$ | ‘SCALED’ |
[ORIGIN(NAXIS)] | $<$integer$>$ | origin of the data array |
[DATA] | $<$narray$>$ | scaled numeric value at every pixel |
[SCALE] | $<$numeric$>$ | scale factor |
[ZERO] | $<$numeric$>$ | zero point |
General-purpose applications will process the scaled variant. However, users may notice extra delays while the array is converted to $<$narray$>$ for processing and rescaled for storage at the end of each application. In such cases, they may prefer the alternative strategy of first running a utility application to convert the data to primitive floating point in a new container file, then performing their processing, and finally running another utility to convert the data back to the scaled-array form.
The numeric value of a pixel is:
where MAX
, MIN
are the maximum and minimum valid (i.e. not bad) data values respectively, and
TMAX<T>
, TMIN<T>
are the largest and smallest valid values for the primitive type of [DATA],
represented by token <T>
.
The equivalent primitive type is the TYPE of [ZERO] or, if [ZERO] is absent, the TYPE of [SCALE]. If both [ZERO] and [SCALE] are missing the equivalent $<$numeric$>$ array defaults to TYPE $<$_REAL$>$.
A spaced $<$ARRAY$>$ is one where the data values vary linearly with array index. An example where this variant could be used is axis data, which are often equally spaced.
General-purpose applications will handle these structures. However, extra time will be expended (though probably much less than for a scaled $<$ARRAY$>$) whilst conversion to and from a $<$narray$>$ is performed. They do not contain magic-value bad pixels.
Component Name | TYPE | Brief Description |
[VARIANT] | $<$_CHAR$>$ | ‘SPACED’ |
[ORIGIN(NAXIS)] | $<$integer$>$ | origin of the data array |
[DIMENSIONS(NAXIS)] | $<$integer$>$ | dimensions of equivalent numeric array |
[SCALE(NAXIS)] | $<$numeric$>$ | scale factors |
[BASE(NAXIS)] | $<$numeric$>$ | zero points |
The value of an element of the equivalent $<$numeric$>$ array would be: $$VALUE=\sum _{i=1}^{NAXIS}\left(BASE\left(i\right)+\left({k}_{i}-1\right)\ast SCALE\left(i\right)\right)$$
where ${k}_{i}$ is the element number of the ${i}^{th}$ dimension.
A sparse $<$ARRAY$>$ structure is used for storing the values of an array where most of the pixels are equal in value (zero perhaps). The structure is composed of a list of array indices for the specified “non-grey” elements, and their corresponding data values. The “grey” value adopted for the majority of the array elements is also supplied.
Component Name | TYPE | Brief Description |
[VARIANT] | $<$_CHAR$>$ | ‘SPARSE’ |
[ORIGIN(NAXIS)] | $<$integer$>$ | origin of the data array |
[DATA(NDATA)] | $<$numeric$>$ | data values for pixel subset |
[LIST(NAXIS,NDATA)] | $<$integer$>$ | list of pixel array indices |
[DIMENSIONS(NAXIS)] | $<$integer$>$ | dimension of equivalent $<$numeric$>$ array |
[GREY] | $<$numeric$>$ | grey value |
The equivalent primitive type is the TYPE of [DATA].
The array is represented by an $n$-dimensional polynomial. (This variant does not conform to the guideline about variants since conversion of an array of numbers to a polynomial requires additional information. However, for clarity and because the reverse operation is possible—polynomial to an array of numbers—a variant of $<$ARRAY$>$ has been used.)
Component Name | TYPE | Brief Description |
[VARIANT] | $<$_CHAR$>$ | ‘POLYNOMIAL’ |
[ORIGIN(NAXIS)] | $<$integer$>$ | origin of the data array |
[DATA] | $<$POLYNOMIAL$>$ | polynomial representing array |
[DIMENSIONS(NAXIS)] | $<$integer$>$ | dimension of equivalent |
$<$numeric$>$ array | ||
The ‘SPACED’ variant may appear merely to be a degenerate form of the ‘POLYNOMIAL’ variant. However, there are reasons for having a ‘SPACED’ variant:
To reiterate, general-purpose applications will be able to process
$<$ARRAY$>$
structures, though their performance will be degraded when the [VARIANT] is not ‘SIMPLE’
because of the extra processing required. These options do not create a significant overhead
for the normal case, the base variant. When the [VARIANT] is not ‘SIMPLE’
, users will
probably be warned that their data are in a special format, be given the option to abort
the application, and be directed to documentation for applications better suited to the
data.
HDS does not provide a $<$_COMPLEX$>$ primitive type, and therefore if complex data are to be stored a $<$COMPLEX_ARRAY$>$ structure, defined in Table 19 must be used.
Component Name | TYPE | Brief Description |
[VARIANT] | $<$_CHAR$>$ | registered variant |
[REAL] | $<$p_array$>$ | real-component data values |
[IMAGINARY] | $<$p_array$>$ | imaginary-component data values |
Currently, the [[VARIANT]] can only be ‘SIMPLE’
.
The dimensions of [REAL] and [IMAGINARY] must be identical. Both components are mandatory. The pixel origin will be taken from [REAL] component’s structure, i.e. [REAL.ORIGIN].
If a pixel is undefined, the magic value should be assigned to both corresponding elements of the real and imaginary arrays. However, should the application encounter the case where only one element is flagged, the whole pixel should be regarded as undefined.
The [ORIGIN] components should match, but only [REAL.ORIGIN] is used.
This structure has TYPE $<$AXIS$>$ and is used to store information describing the size and spacing of the pixels in a multi-dimensional data array. In use, $<$AXIS$>$ structures are arranged as elements of a 1-dimensional array (of structures). The number of elements (structures) in this array is equal to the dimensionality (number of axes) of the data array being described, so that each $<$AXIS$>$ structure relates to a single data dimension. The following description applies to one element of that multi-dimensional structure.
Component Name | TYPE | Brief Description |
[VARIANT] | $<$_CHAR$>$ | registered variant |
[DATA_ARRAY] | $<$p_array$>$ | axis value at each pixel |
[LABEL] | $<$_CHAR$>$ | axis label |
[UNITS] | $<$_CHAR$>$ | axis units |
[VARIANCE] | $<$s_array$>$ | axis variance in [DATA_ARRAY] |
[NORMALISED] | $<$_LOGICAL$>$ | true if the data have been normalised |
[WIDTH] | $<$s_array$>$ | bin width of each pixel |
[MORE] | $<$EXT$>$ | extension structure |
‘SIMPLE’
.
The co-ordinates are given as a vector with length corresponding to that of the corresponding dimension of the data array. In practice, it will often be the polynomial or spaced variant of TYPE $<$ARRAY$>$.
where ${D}_{n}$ is the ${n}^{th}$ centre value given by [DATA_ARRAY]. The extreme values are twice the available half widths, viz. $${D}_{n}-0.5\ast WIDTH\phantom{\rule{1em}{0ex}}\phantom{\rule{1em}{0ex}}\phantom{\rule{1em}{0ex}}to\phantom{\rule{1em}{0ex}}\phantom{\rule{1em}{0ex}}\phantom{\rule{1em}{0ex}}{D}_{n}+0.5\ast WIDTH$$
for the ${n}^{th}$ element.
This structure is used for storing data-quality information.
Component Name | TYPE | Brief Description |
[VARIANT] | $<$_CHAR$>$ | registered variant |
[QUALITY] | $<$p_array$>$ | quality value of each pixel |
[BADBITS] | $<$_UBYTE$>$ | mask of allowed bad bits in quality |
‘SIMPLE’
.
The rôle of data-quality and how it can be used to advantage were described earlier.
This structure has NAME [HISTORY] and TYPE $<$HISTORY$>$. Its components are shown in Table 22.
Component Name | TYPE | Brief Description |
[VARIANT] | $<$_CHAR$>$ | registered variant |
[CREATED] | $<$_CHAR$>$ | creation date and time |
[EXTEND_SIZE] | $<$_INTEGER$>$ | increment number of records |
[CURRENT_RECORD] | $<$_INTEGER$>$ | record number being used |
[RECORDS($m$)] | $<$HIST_REC$>$ | array of $m$ history records |
‘SIMPLE’
.
Component Name | TYPE | Brief Description |
[VARIANT] | $<$_CHAR$>$ | registered variant |
[DATE] | $<$_CHAR$>$ | creation date and time |
[COMMAND] | $<$_CHAR$>$ | application name and version |
[TEXT($m$)] | $<$_CHAR$>$ | $m$ lines of text |
‘SIMPLE’
.
Note there are currently no variants of the $<$HISTORY$>$ and $<$HIST_REC$>$ structures.