NDFs contain a lot more than just the data values. They contain so called headers, along with values such as Quality and Variance associated with each pixel. For some NDFs there is also axes information.
The easiest way to take a look at what structures exist within an NDF is to use hdstrace. A sample trace might produce an output such as:
We can see from this that there are arrays for both the data and the variances. There is also some FITS information, and the file has at some point been processed by CCDPACK. Let’s write a new code that processes the variances in a similar way to how clip processed the data array.
Let’s illustrate the way in which the NDF library can handle variance information by developing a simple application. In the following example, an image is searched for “spikes” which are potential cosmic ray events.
When spikes are detected, the pixels are set to what is known as a BAD value. BAD pixels arise in situations where the values of the pixels are either unknown, useless, or meaningless. In this simple routine, we assign BAD values to pixels whose information is corrupted by cosmic rays.
In this example, any variance array present in the NDF is left untouched. This means that the variance array is no longer valid. It is, therefore, not passed on to the output image. Guidelines for when components of an NDF should and should not be propagated are listed in SUN/33. We will deal with a case where the variance information is propagated later in this section.
Note how the interface file now includes a parameter which is a REAL number as well as filenames.
The hdstrace application reveals that if the input NDF contained a variance array, it does not exist in the output NDF. This is because the NDF_PROP call did not explicitly list it. In order to get the variance array propagated through to the output, let’s adapt the code and call it zapper2.
This code propagates the variance array. Note you will need to copy the zapper.ifl file to a zapper2.ifl file.
Note that although the variance array was mapped, none of its values were changed in this simple application. The variance array was mapped and therefore allocated pointers (although this wasn’t necessary to force the variance propagation). The interested reader might wish to try to adapt the call to ZAP to fix the variance values using these pointers.
In addition to the Data and Variance components, some NDFs also contain a Quality component. The quality array is not usually accessed directly by the user. Instead, the processing of the Data and Variance arrays automatically takes into account the values in the Quality array unless the application explicitly accesses and maps the Quality array.
Why bother with a Quality array when there is a variance array? The Quality value of a pixel can be used to describe its suitability for performing a particular task. For example, imagine a situation where photometry is performed on an object. The quality array could be set to one value for the object and another for the sky pixels.
Manipulation of the Quality array is not commonly done in most applications and so there will be no further discussion of it here. SUN/33 contains a full discussion of how to access the Quality array.
It is not appropriate to propagate the values in the variance array in all circumstances. For example, adding two NDFs together will result in new variances for the result. If these new variances are not calculated, the output NDF must not be allowed to contain a variance array. On the other hand, if a constant is added to an NDF, the variances can be passed on to the output with no calculations necessary.
The default used by the routine NDF_PROP is not to propagate the data, quality, or variance arrays. This explains why is was necessary to explicitly list what we wanted to propagate in the last example.