Chapter 2
Tools

 2.1 Overview of the Autotools
  2.1.1 Autoconf
  2.1.2 Automake
  2.1.3 Libtool
  2.1.4 Autoreconf: why you don’t need to know about aclocal
  2.1.5 Running ./configure
 2.2 Starconf, and the Starlink autoconf macros
  2.2.1 STARCONF_DEFAULT_PREFIX and STARCONF_DEFAULT_STARLINK
  2.2.2 Components
  2.2.3 File templates
  2.2.4 The format of the component.xml file
 2.3 Version numbers
 2.4 A few remarks on state
 2.5 Preprocessable Fortran
 2.6 Using the Starlink CVS repository
 2.7 CVS
  2.7.1 CVS and tagging
  2.7.2 CVS and recursion – checking out only a subtree

This section is concerned with the tools which make up the Starlink build system. There are generally very few adjustments you need to make to the program source code (though the autotools can help you manage such things as installation directories and platform dependencies), and the changes you will make are to the auxiliary files which manage the build, most prominently the configure.ac file which organises the configuration, and the Makefile.am file which controls generating the makefile which does the work.

The build system as a whole consists of a number of components:

The Starlink autotools
These consist of versions of the GNU autotools, which are part of the Starlink CVS repository, and which are built and installed as part of the top-level bootstrap. They should be installed in /star/buildsupport/bin, or its equivalent if you have installed a set of tools in a tree other than /star. Since they have been extended and customised, they are essential; you cannot build the Starlink collection without them. There is an introduction to the autotools in Sec. 2.1, covering their interrelationships and basic usage.

We use (at the time of writing) an unmodified libtool, a moderately extended autoconf, and an extended and customised automake. The description below explains the use of these tools as modified; see Sec. B for details of the differences from the stock autotools.

The Starlink autoconf macros
There is an extensive set of autoconf macros which support building Starlink applications. You use these as you would any of the standard autoconf macros, by invoking them within the configure.ac file.

The macros are described in detail in Sec. A.

The autoconf program which is used is not itself customised for Starlink use. Instead, the macros are installed using the standard autoconf extension mechanism, which uses the application aclocal. This is discussed in passing in Sec. 2.1, and we mention it only in passing here.

starconf
This is the program which handles installing required macros, and configuring your directory ready to use the build system. Associated with starconf is the starconf-validate program which checks that your directory is configured correctly, and the program ./starconf.status, which unpacks required files, and allows you to make the link back to the configuration used to build the software. All these are described in detail in Sec. 2.2.

2.1 Overview of the Autotools

The autotools as a group help you create software distributions which will build reliably on a large variety of platforms. They are most obviously associated with unix systems, but in fact support the larger class of POSIX systems which have an implementation of /bin/sh available – a class which includes Windows machines, through the cygwin environment.

The minimum required contact with the autotools is to generate the files which will let you configure a freshly checked-out directory before working on it, since these configuration files are not checked in to the repository. The ./bootstrap script (see Sec. 2.2) will do this for you, but it can only do this if the autotools are installed (see Sec. 3.5).

The aim of the notes below is to give you an overview of autoconf and automake functionality, enough to make it possible for you to roughly understand the existing configuration files, and to make it easier to approach the full autoconf and automake documentation. We start off with a broad overview, and provide a few further details in the subsections which follow.

The two most obvious and important files are configure.ac and Makefile.am. The first tells autoconf how to make the ./configure script which does the actual configuration work, and which will be distributed with your package, the second (assuming you’re using automake rather than hand-maintaining the input Makefile) tells automake how to create the template makefile Makefile.in, which is also distributed with your package, and which is edited by the ./configure script to (finally!) produce the Makefile which (finally!) builds the code on the user’s system.

It’s useful to illustrate the inputs and outputs of the various tools. First are autoconf and automake:


pict
Figure 2.1: Inputs to, and outputs from, autoconf



pict
Figure 2.2: Inputs to, and outputs from, automake


As you can see, although configure.ac is most associated with autoconf, automake reads it, too, to discover for example whether you have invoked AC_PROG_LIBTOOL or STAR_MONOLITHS, and so whether it needs to add the appropriate support in the generated Makefile.in.

Once the ./configure script is created, it is able to interrogate the system (the user’s system, that is, not the developer’s), and based on that edit the various .in files to produce their corresponding outputs.


pict
Figure 2.3: Inputs and outputs to configure


This diagram for ./configure shows a file config.h.in being edited. This file – which we will have more to say about in Sec. 2.1.1 – is a template list of C preprocessor definitions, which can be #included in a C source file, and so used to configure the use of functions and definitions there. Like any of the other .in files, it is possible to maintain this by hand, but it is more reliable to maintain it using another application autoheader, which looks as follows.


pict
Figure 2.4: Inputs and outputs to autoheader


The other file we have not mentioned yet is starconf.m4. This file contains macros (in the implementation language of autoconf, m4) which are used to extend autoconf’s functionality, supplying the Starlink-specific autoconf support. You will not typically have anything to do with this file, and we mention it only to indicate schematically where (some of) the Starlink magic comes from. This file is not in your working directory, but is installed in the place the autotools expect to find it, by the starconf application described in Sec. 2.2. You may also notice a file aclocal.m4 in your directory. This is a cache of autoconf macros, maintained by the autoreconf program; it should not be checked in.

Of the applications mentioned above, automake, autoconf and (usually implicitly) autoheader are run on the developer’s machine before or while making a distribution; thus the products of these programs, ./configure, config.h.in and Makefile.in, are included in the distribution, and the .in files edited by the ./configure script into the config.h and Makefile which actually control the build.

We have elided some details here, for simplicity, and we could possibly have elided more, since you generally don’t have to concern yourself with autoheader or aclocal.m4. You don’t even have to worry about the dependencies between these files and applications, since the Makefile.in which automake generates knows about these dependencies and will generally keep things up to date for you. If you wish to be sure things are up to date, then you can simply run application autoreconf (see Sec. 2.1.4), which knows enough about the relationships between them to run each of the required ones in the correct order. This is likely the only one of the autotools commands which you will run yourself.

For more details on the relationships between these applications, and fuller diagrams, see the online copy of the book ‘GNU Autoconf, Automake, and Libtool’, and in particular Appendix C of that book, ‘Generated file dependencies’.

Comprehensive details of the autotools are to be found in their respective manuals, which are available online at

The autoconf manual is relatively clear, once you have a basic idea of what it’s trying to do, but it’s more effective as a reference manual than as a tutorial. The automake manual should be regarded as a reference manual only: you might not guess from the manual, that automake will make your life simpler, rather than full of anguish, suffering and confusion. It doesn’t really matter how good or bad the libtool manual is, since if you discover you have to look at it, you already know your future holds pain, and the manual isn’t going to make things better or worse.

It’s also worth looking at the ‘Release Process’ section of the GNU Coding Standards document. Though we are not necessarily following these standards, this section both describes what is conventional packaging in the open-source world, and outlines the conventions which the autotools are designed to support.

2.1.1 Autoconf

We use a slightly extended version of autoconf. See autoconf –version for the base version number.

The goal of autoconf is to help you make your program portable, by allowing you, at build time, to adapt the program to the facilities available on the machine on which the program is being built.

You control this through a script called configure.ac (which was called configure.in in older versions of autoconf). The autoconf program produces from this a very portable /bin/sh script called configure, which is distributed as part of your package, and which the person building the package runs as the first part of the ./configure; make; make install incantation.

The ./configure script probes the system it is running on, finding compilers, testing the behaviour of the local unix system, testing whether specific include files exist or not, testing whether required functions are available and working as expected, and managing some configuration by the user, such as allowing them to specify where the package is to be installed.

This information is only useful if it can be communicated to your program in some way. This is done by the configure script by editing the values of certain ‘substitution variables’ into a list of template files.

For example, you might have a version header file version.h.in, containing the line

  const char version[] = "@PACKAGE_VERSION@";

The configure variable PACKAGE_VERSION is one of those substituted by default, and if this file were listed as one of those to be substituted (by mentioning it in the autoconf macro AC_CONFIG_FILES(version.h)), then a file version.h would be created containing the contents of the version.h.in file with the @PACKAGE_VERSION@ substituted by the version number declared within the configure file.

Although this substitution process can be done for any template file, there are two template files which are used particularly often.

The first is Makefile.in, which is a skeleton makefile which you might write by hand (though see the discussion of automake in Sec. 2.1.2). There is an example of this at the top level of the Starlink build tree. A typical Makefile.in might include lines like

  LN_S = @LN_S@
  CC = @CC@
  myfile.o: myfile.c
          $(CC) -o myfile.o myfile.c

That, combined with autoconf macros AC_PROG_CC and AC_PROG_LN_S, would allow you to produce a file Makefile which was customised to use the C compiler appropriate to the local environment, and which had, in the makefile variable {$(LN_S) a command which makes links between files if the local platform supports that, or makes hard links or simple copies if that is all that is possible.

As well as configuring the makefile, you may also want to configure the source code, so that you can take different actions in your code depending on whether certain functions or headers are available. That is is most often done via a particular configured file, conventionally named config.h. This second way of communicating with your source code has the config.h file substituted with a number of C preprocessor #define statements. For example, if you included in your configure.ac file the lines:

  AC_CHECK_HEADERS([sys/wait.h])
  AC_CHECK_FUNCS([strchr])

then the configure script generated from that source would include a test of whether the sys/wait.h header file was available, and whether the strchr function was available in the C library. If so, the resulting config.h file would include the lines

  #define HAVE_SYS_WAIT_H 1
  #define HAVE_STRCHR 1

Whereas most configured files are substituted as a result of being mentioned in a AC_CONFIG_FILES() macro, the config.h.in input file is configured through AC_CONFIG_HEADERS(config.h), which does its configuration in a slightly different way.

After including this file into your source code with:

  #include <config.h>

you can adjust the compiled code with suitable #if preprocessor conditionals, such as

  #if HAVE_SYS_WAIT_H
  #include <sys/wait.h>
  #endif

It is possible to maintain the config.h.in file by hand, but generally better to generate it by using the autoheader application which is part of autoconf. This scans your configure.ac and extracts into config.h.in mentions of all those preprocessor defines it finds. Autoheader knows about macros such as AC_CHECK_HEADERS above; if you wish to add further switches to the config.h.in file, you should do so by calling the autoconf macro AC_DEFINE. See the section 7.1 Defining C Preprocessor Symbols in the autoconf manual for further details. Autoheader is one of those applications run on your behalf by autoreconf (see Sec. 2.1.4).

Note the angle brackets round config.h: this is preferred to double quotes, as it gives the makefile the option of controlling where the source code is; in the simplest case this is handed by simply adding -I. to the compile line. We don’t take take advantage of this particular flexibility, but it is a good habit to get into.

An illustrative configure.ac might look like this (with line numbers inserted):

  1   AC_INIT(mypack, 1.2, starlink@jiscmail.ac.uk)
  2   AC_CONFIG_SRCDIR(src.c)
  3   STAR_DEFAULTS
  4   AC_PROG_CC
  5   AC_PATH_PROG(PERL, perl)
  6   AC_CHECK_HEADERS([sys/time.h])
  7   STAR_MESSGEN(mypack_err.msg)
  8   AC_CONFIG_HEADERS(config.h)
  9   AC_CONFIG_FILES(Makefile)
  10  AC_CONFIG_FILES(myscript.pl, [chmod +x myscript.pl])
  11  AC_OUTPUT

Line 1: This line is required. It declares the package name, version number, and bug-report address. Each of these is available for substitution via the substitution variables PACKAGE_NAME, PACKAGE_VERSION and PACKAGE_BUGREPORT.See autoconf 4.1 Initializing configure

Line 2: This is largely a sanity check, and produces an error if the named file (in this case src.c) is not in the source directory. The source directory is typically the current directory, but you can specify a different one using the –srcdir option to ./configure, if you have a good reason for doing that. See autoconf 4.3 Finding configure Input

Line 3: This macro is one of the Starlink extensions, and the only required Starlink macro. It sets up the defaults required for building Starlink applications, and assures the starconf program that it’s being run in the correct directory. See Sec. 2.2 for a description of the starconf application, and Sec. A for details of the associated macros.

Line 4: This finds a working C compiler, and prepares to substitute it in the substitution variable CC. Presumably the Makefile.in has a line CC=@CC@. See autoconf 5.10.3 C Compiler Characteristics

Line 5: Find a binary called perl in the current PATH and assign the substitution variable PERL the full path to it. The most common way of using this would be for the file myscript.pl.in to start off with the line

  #! @PERL@ -w

so that the script ends up with the correct full path. See line 10<span media="all"> and autoconf 5.2.2 Generic Program and File Checks

Line 6: check that the system has an include file sys/time.h in the include path, and if it has, make sure that the cpp variable HAVE_SYS_TIME_H is defined. If this were a real configure file, you would likely have several other header tests here (in a space-separated list, surrounded by square brackets), and cpp branching inside your source code to handle the various cases. See line 8 and autoconf 5.6.3 Generic Header Checks.

Line 7: Another Starlink extension. It declares that this package has a set of ERR messages in the given file, and that autoconf should check the location of the messgen application. The argument is in fact optional (it merely causes the files to be declared as pre-distribution files – see Sec. A.27; this line should be partnered by a declaration of the variable include_MESSAGES in the corresponding Makefile.am. See Sec. A.23 for fuller details.

Line 8: This is the macro that makes cpp configuration information available, by editing the header file config.h.in (this file name is conventional, but not required). See autoconf 4.8 Configuration Header Files. If you want to put extra information into this file, use the AC_DEFINE macro: a declaration like AC_DEFINE(SPECIALCASE,1) would insert #define SPECIALCASE 1 into the config.h; autoheader also spots this and puts a suitable template into config.h.in. See the autoconf manual, section Defining C preprocessor symbols, for further details.

Line 9: This does the work of substituting the results of the various tests into the files being configured. For each of the files named in the (space-separated list) argument, which most typically includes Makefile, autoconf expects to find a file of the same name but with .in appended, and this is the file which has the substitutions edited in. (Automake also looks at this line, and if it sees a Makefile mentioned here, it looks to see if there is a corresponding Makefile.am already present, and if so, recurses)

Line 10: This is a variant of line 9. The AC_CONFIG_FILES macro takes a second argument consisting of one or more lines of shell script to post-process the file in question, in this case making sure that the generated file is executable.

Line 11: This is the line which really does the work.See 4.4 Outputting Files.

Of this script, it is lines 1, 2 and 11 which are absolutely required, along with something like line 9 to make the configure script do something useful.

It is useful to think of the configure.ac file as being the template for the final ./configure script, with the autoconf macros simply expanding to (large chunks of) shell script during translation by the autoconf program. This isn’t the whole truth, but it suffices for almost all purposes. About the only place where this view might deceive you is if you wished to modify a file after it was generated by AC_CONFIG_FILES for example; if you did that immediately after the AC_CONFIG_FILES line it would fail, since the code which generates the files is in a different place from the AC_CONFIG_FILES line – that is why AC_CONFIG_FILES has its second argument. With this view it is natural that you can add any other shell scripting to your configure.ac, adding any tests and switches you fancy. You communicate the results of your scripting to the output files by making shell variables into substitution variables, and the way you do that is by calling AC_SUBST on them. Thus, after

  wibble=’’
  ... # lots of clever scripting
  AC_SUBST(wibble)

any occurrences of the string @wibble@ in the substituted files will be replaced by the value of wibble at the end of the ./configure script.

2.1.1.1 M4 syntax and traps

Autoconf does its work by running the configure.ac file through the text processor GNU m4, and this can cause occasional surprises.

M4 is a macro language, intended for very much the sort of rewriting that autoconf does. When m4 processes a file, anything at all that looks like an m4 macro is substituted, so that there is no macro invocation character like the backslash in TEX;. Macros can take arguments, delimited by round brackets and separated by commas, as illustrated above.

The first oddity is to do with comment characters. The default m4 comment character is #, but since this is also the shell script and makefile comment character, autoconf makes it an ordinary character. Thus to add comments to a configure.ac file which won’t make it into the ./configure file, you should prefix them with the m4 macro dnl (which means ‘delete to next newline’), as in

  dnl   This is a comment
  AC_INIT(...)

There’s no harm in including #-style comments in your configure.ac file and allowing these to be passed through to the output file, since ‘got-here’ type comments can sometimes help you debug the ./configure script.

The default m4 quote characters are the left and right single quotes, but autoconf changes these to left and right square brackets. You need to use these in two circumstances, firstly when you have a multi-word argument to a macro, and particularly when that argument contains a comma, and secondly if a piece of text looks like a current m4 macro. In general, if you need to care about m4 quoting rules, you’re in trouble, but see section 8.1 M4 Quotation in the autoconf manual, for some advice.

2.1.2 Automake

The most typical use of autoconf is to configure a file Makefile.in. You can write this yourself, as described in the discussion of autoconf, but since so much of a typical makefile is boilerplate, automake exists to write this boilerplate for you. This has the additional advantage that we can support and enforce Starlink-specific conventions with a customised installation of automake. We (currently) use an adapted version of automake – see automake –version for the version number, and Sec. B for a summary of the differences.

The file which controls automake is Makefile.am. Automake reads both this file and configure.ac, and based on these emits a Makefile.in. It is this latter file which you distribute, and which is substituted at build time by the ./configure script which autoconf generates in turn.

The resulting Makefile has rules to do all the required building and installation in a very portable fashion, as well as targets to make distributions (make dist), do tests (make check), clean up (make clean, make distclean and make maintainer-clean) and, in the case of Starlink automake, do an install along with an installation manifest.

The Makefile.am script consists, in its simplest form, of a sequence of declarations of the relationships between the source files in your distribution and the programs and libraries they are intended to produce. For example, here is the Makefile.am for the PAR library, slightly edited:

  ## Process this file with automake to produce Makefile.in
  
  lib_LTLIBRARIES = libpar_adam.la
  libpar_adam_la_SOURCES = $(F_ROUTINES)
  
  include_HEADERS = $(PUBLIC_INCLUDES)
  
  F_ROUTINES = \
       par1_menu.f \
       par_cancl.f \
       [blah...]
  
  PUBLIC_INCLUDES = \
      PAR_ERR \
      PAR_PAR \
      par.h \
      parwrap.h \
      par_err.h \
      par_par.h
  
  BUILT_SOURCES = PAR_ERR par_err.h

Overall, you can see that this automake source file is syntactically a makefile – the statements in this example look like makefile variable settings, and it is possible to put makefile rules in the Makefile.am file, though that is not illustrated here. This is slightly deceptive, however, and while it was useful to think, above, of the configure.ac file as being the template of the eventual ./configure script, for automake you should think of the match between the input syntax and the makefile as a happy coincidence. You provide information to automake through the Makefile.am file, and based on that it then emits the Makefile.in it thinks you want (or need, at least).

The first line is a conventional comment. It starts with a doubled hash mark ##, which causes automake to discard the text after it; lines beginning with a single comment character are copied verbatim into the generated Makefile.in.

The next stanza declares that there is to be a library libpar_adam.la, and that the sources for this file are in the ‘makefile variable’ F_ROUTINES.

Though the variables F_ROUTINES and PUBLIC_INCLUDES are specific to this makefile, and arbitrary, the other variable names have both structure and meaning for automake.

The variable lib_LTLIBRARIES consists of the ‘prefix’ lib and the ‘primary’ LTLIBRARIES. The primary tells automake that you wish to build the libtool library libpar_adam.la, and the prefix indicates that this is to be installed with the other libraries (the variable pkglib_LTLIBRARIES, for example, would tell automake that you wanted to install the result in a package-specific library). For each of the libraries listed in this variable, separated by spaces, there must be a corresponding _SOURCES variable, which has the ‘primary’ SOURCES and a prefix formed from the library name, which lists the set of source files for that library. The prefix must be canonicalised by replacing with underscores everything other than letters and numbers. As well as declaring the files which are to be compiled into the library, this indicates to automake that these source files are to be distributed as part of the final tarball, and that it must emit makefile rules to install the library in the correct place, with the correct installation commands. For the list of such associated primaries, see section Program and Library Variables of the automake documentation.

By default, libtool builds both static and shared libraries. You can control this if necessary with the –enable-shared and –disable-shared options to ./configure, and with the AC_DISABLE_SHARED autoconf variable. For further details see the documentation for AC_PROG_LIBTOOL in the Libtool manual.

If we only wanted to build static libraries, we would replace this line with lib_LIBRARIES = libpar_adam.a, and the given library would be built in the usual way, without involving libtool.

The LIBRARIES and LTLIBRARIES primaries can have any one of the prefixes libdir, pkglibdir, check or noinst, with the latter indicating that the library should be built but not installed. Having non-installed libraries can be useful when you are building a library conditionally, or in stages. Libtool refers to these as ‘convenience libraries’, and they are discussed in section Libtool Convenience Libraries of the automake manual.

The other very important primary (not shown here) is PROGRAMS, which describes programs which are to be built and installed (the special Starlink case of monoliths is described below). This can have the prefixes bin, sbin, libexec, pkglib, check or noinst: check is for tests, and is described in Sec. 4.4; noinst indicates that the program should be built but not installed, and is useful for programs which are used during the build – for generating header files for example – but which are not part of the final product. There is no single standard list of prefixes, since each primary supports only a subset of them (you cannot declare bin_LIBRARIES, for example), but several are mentioned in automake’s What Gets Installed, and the directories in question are discussed in autoconf’s 4.7.2 Installation Directory Variables.

The include_HEADERS line is similar: it indicates that the listed files are to be distributed, and are to be installed in the correct place for include files.

The final line which is significant to automake is the BUILT_SOURCES line. This says that, even though PAR_ERR and par_err.h are to be installed and distributed, they are not actually genuine source files, but must be built; adding the BUILT_SOURCES line forces automake to add a dependency for these files at an artificially early stage. It is only rather rarely necessary to include a line like this. If one of the source files were in this category, then it would naturally be built when it was required, without any need to add it to BUILT_SOURCES. As it happens, there is no rule in this file for building these targets; that is added automatically by automake when it spots that the STAR_MESSGEN macro has been used in the configure.ac file which partners this Makefile.am. In a more general case, however, you would add a make-style rule to the Makefile.am file to build these files from their (presumably undistributed) sources. See also Built sources in the automake manual.

For a second example, we can look at the Makefile.am for the sst application (this is a non-distributed application for building documentation).

  ## Process this file with automake to produce Makefile.in
  
  bin_SCRIPTS = start
  
  bin_MONOLITHS = sst_mon
  sst_mon_SOURCES = \
      sst_mon.f \
      $(sst_mon_TASKS:=.f) \
      $(SUBSRC) \
      $(SUBCSRC) \
      $(PRIVATE_INCLUDES)
  sst_mon_TASKS = forstats procvt prohlp prolat propak prohtml
  sst_mon_LDADD = $(LDADD) ‘fio_link‘
  
  
  SUBSRC = sst_clean.f sst_fwild.f sst_latex.f sst_puts.f  sst_trcvt.f \
           sst_cntac.f sst_get.f sst_latp.f sst_rdad1.f sst_trhlp.f \
           [blah...]
  
  SUBCSRC = find_file.c
  
  PRIVATE_INCLUDES = SST_PAR SST_SCB
  
  # special installation directory (but see discussion below)
  sstsupportdir = $(bindir)
  sstsupport_DATA = sun.tex sst.tex layout.tex forstats.dat html.sty
  
  # The ‘start’ script needs to have installation locations edited into it
  edit = sed \
      -e ’s,@bindir\@,$(bindir),g’ \
      -e ’s,@VERSION\@,$(VERSION),g’
  start: start.in
      rm -f start.tmp start
      $(edit) \
          -e ’s,@edited_input\@,start: produced from start.in by Makefile.am,’ \
          $(srcdir)/start.in >start.tmp
      mv start.tmp start
  
  EXTRA_DIST = start.in $(sstsupport_DATA)

This makefile configures and installs a script, builds and installs a monolith, and adds some supporting data files in a non-standard place.

The SCRIPTS primary indicates to automake where and how to install the start script. The start script must be generated, since it is to include the version number and installation location. Since it includes the installation location, it should not be generated at configure time or install time, but instead at make time, so that the user is free to specify one installation prefix at configure time (through ./configure –prefix=www), override this with another prefix at build time (through make prefix=xxx) and specify a different one – presumably a staging location or something similar – at installation time (through make prefix=yyy install). It is the prefix at build time that is to be baked into the code, if any such baking has to be done. This is one of the GNU conventions mentioned in Sec. 2.1, and is discussed in a little more detail in section 4.7.2 Installation Directory Variables of the autoconf manual. This is why we have to include a makefile-style rule for deriving the file start from its template start.in. This substitutes in the values of the makefile variables bindir and VERSION; these get their values, in the resulting Makefile, by having those values substituted in to the generated Makefile.in by the generated ./configure script; the careful escaping of the @-signs in the sed command is to match the @...@ in start.in while not matching this in Makefile.am or the resulting Makefile.in (yes, this can get a little confusing).

This Makefile.am also declares a single monolith (this and the TASKS primary below are obviously part of the Starlink extensions to automake) and its associated SOURCES, along with its component tasks. For fuller details of the monoliths support, see Sec. 3.4. This incidentally illustrates that automake allows variable reuse and rewriting very similar to that supported by make.

When we are linking this monolith, we need to add a couple of link options, and we do this with a LDADD primary associated with the sst_mon prefix. The extra link options we declare here are added to the eventual link command, replacing the default set of option options $(LDADD), so we include that variable in our version. We also add ‘fio_link‘, because this monolith needs to be linked against the FIO library (automake constrains what can appear in the LDADD variable, and the ‘fio_link‘ is permissable only in Starlink automake – see Sec. B for details).

The variables SUBSRC, SUBCSRC and PRIVATE_INCLUDES are purely user variables, with no special meaning to automake.

The next bit of magic in this file is sstsupport_DATA. In this particular case, we want to install data files in the same location as the binaries (why, and whether this is a good thing, is not for us to worry about here). The obvious thing would be to say bin_DATA = sun.tex ..., but automake forbids that particular combination of prefix and primary, precisely because it wouldn’t generally make sense (see Architecture-independent data files in the automake manual). Instead, we can take advantage of an escape-hatch that automake provides: a given prefix, such as sstsupport, is valid if you also define a variable of the same name, but with dir appended. Thus we define sstsupport_DATA and sstsupportdir, and define the latter to have the same value as the $(bindir) makefile variable, as eventually substituted (see section The Uniform Naming Scheme in the automake manual).

If you look at the real sst component’s Makefile.am file, you will see that it does in fact use bin_DATA rather than the sstsupport_DATA illustrated above. This is because the configure.ac in the sst component declares the option STAR_DEFAULTS(per-package-dirs) (see Sec. A.18) which, amongst a couple of other magic features, causes the variable bin_DATA to become legal.

This is a rather special case, and you should not have to do this sort of semi-licensed hacking very often. In particular, you do not need to do it in the case of the Starlink standard directories /star/docs, /star/etc, /star/examples and /star/help, since you can give these directories as prefixes to the DATA primary. As an example, the HDS makefile installs a file containing the information it has determined about the build machine’s floating point model, and this is declared there as follows:

  noinst_PROGRAMS = hds_machine
  starhelp_DATA = hds_machine.txt
  
  hds_machine_SOURCES = hds_machine.f
  hds_machine_LDADD = libhds.la ‘ems_link‘ ‘chr_link‘ ‘cnf_link‘
  
  hds_machine.txt: hds_machine
          ./hds_machine >$@

This is enough to build the hds_machine application at install time, run it, and install the results in /star/help (with the obvious changes for prefixes stardocs, staretc and starexamples).

The remaining magic in this file is the EXTRA_DIST variable. For all its cleverness, automake cannot work out that the start.in source, nor any of the sstsupport_DATA files are to be distributed, and you need to declare that explicitly by setting the EXTRA_DIST variable. Though it is of course deterministic (see section What Goes in a Distribution of the manual), I find the most straightforward way to work out whether anything needs to go here is to build a distribution and spot what was missed out.

2.1.3 Libtool

We are using unmodified Libtool. See libtool –version for the actual version.

Libtool contains the darkest magic of the autotools, so it is fortunate that we need barely concern ourselves with it. This is because all of the technicalities of calling libtool are handled for us by the makefile generated by automake.

The function of libtool is to build libraries. While this is generally straightforward for static libraries, usually involving little more than working out whether ranlib is necessary, doing the same thing for dynamic libraries is extremely platform dependent. Libtool consists of a large body of platform-dependent code, ported to a broad range of operating systems, which implements a single platform-independent interface.

For more details, see the libtool manual at http://www.gnu.org/software/libtool/manual.html.

Unfortunately, libtool’s library magic introduces a minor complication when you wish to run a program under a debugger : plain gdb myprog won’t work, and you must instead use libtool –mode=execute gdb myprog. See section ‘Debugging executables’ of the libtool manual for discussion.

2.1.4 Autoreconf: why you don’t need to know about aclocal

You don’t have to know anything very much about autoreconf, other than that, if you change one of the autotools files Makefile.am or configure.ac., you should probably run autoreconf to bring everything up to date. In fact, you probably don’t even need that, since the generated makefiles have these dependencies explicit. The problem that autoreconf addresses is that when one of these files is updated, there are several commands which might need to be re-run, including aclocal, autoheader, libtoolize and others, and it’s a headache trying to remember which ones are which.

2.1.5 Running ./configure

Running the configure script is basically very simple: ./configure. There are, however, options and arguments which can help you, or catch you out. You can see the full list of options with the command ./configure –help.

You set the place where the configured component will be installed with the –prefix option. This names a directory which will end up with the standard directories bin, manifests and so on. The default is reported by ./configure –help, and is ultimately controlled by starconf (see Sec. 3.5 and Sec. 2.4; or Sec. 2.2.1 for a means of setting it to a per-directory default, which can be useful in certain circumstances).

Because configure’s tests potentially take quite a long time to run, it is possible to cache the results between runs. If you add an option -C, then these results will be cached in a file config.cache in the current directory. When you do the tree-wide configure from the top level, it is important to give this option, otherwise the configure step takes an unconscionably long time. When this is happening, the configure runs in subdirectories use the cache file in the top-level directory, and when you are running configure in subdirectories, you can use this global cache, too. If you are in a directory two levels down from the top (say), then you can configure slightly faster using the command

  % ./configure --config-cache=../../config.cache

This reads and updates the given cache file. Sometimes, when things get a little confused, you might need to delete this cache file – this is always safe.

The Starlink autoconf adds a couple of extra standard options.

–with-starlink[=location]
If no location is provided, this does nothing. This is effectively the default.

If a location is provided, it overrides the STARLINK environment variable which will be used. Very rarely needed.

Option –without-starlink causes the STARLINK shell variable to be unset. Some packages might with to configure themselves differently in this case.

–without-stardocs
Controls whether documentation is built and installed. You might want to turn this off, since it can take quite a long time. The default is –with-stardocs, and so you can disable this with the configure option –without-stardocs.

A few components have extra options on the ./configure command. For example, the component docs/ssn/078 (the component which holds this document) has a –with-pstoimg option, the value of which must be the full path to a copy of the pstoimg program, to help in the cases where this cannot be discovered automatically (this may be a temporary feature of this component).

All of the components have “Some influential environment variables” listed in the help message. This will include at least STARLINK, and in the (very common!) case of components which include a compiler, an environment variable such as CC or FC which allows you to override the compiler which the configure script will find. This is useful if you want to avoid a default compiler and use a specific one instead. For example, if you wished to use the Sun C++ compiler specifically (while on a Sun, obviously), you would put /opt/SUNWspro/bin in your path, and set the CXX variable:

  % export CXX=CC   # best avoided
  % ./configure

or

  % env CXX=CC ./configure   # best avoided

where the latter is probably preferable, inasmuch as it does not leave this important variable set, in such a way that it can make a difference unexpectedly.

Better than either of these is

  % ./configure CXX=CC

This doesn’t set CXX as an environment variable, but sets it in a similar-looking way as one of the ./configure arguments. This way is preferable to either of the above for two reasons: firstly, it does not leave the variable set; secondly, this way ./configure ‘knows’ that the compiler has been overridden, so that if you are using a configure cache, and you fail to do this when the directory is reconfigured, ./configure can warn you of this, in case this is not deliberate.

If you change one of these variables between runs, and are using a configure cache, then the ./configure script will warn you like this:

  % ./configure -C
  configure: loading cache config.cache
  configure: error: ‘STARLINK’ has changed since the previous run:
  configure:   former value:  /somewhere/else
  configure:   current value: /export3/sun
  configure: error: changes in the environment can compromise the build
  configure: error: run ‘make distclean’ and/or ‘rm config.cache’ and start over

To deal with this, simply remove the config.cache file, check that the environment variable is indeed set as you wish, and rerun ./configure.

You can pass options to the compiler using environment variables, but you will not need to do this in general, other than perhaps to set CFLAGS=-g to turn on debugging in the code you are building. The variables CFLAGS and LDFLAGS are variables you might potentially set and export in the environment, for example to point ./configure to software installed in non-standard places (perhaps you are on a Mac and have installed Fink in /sw or OpenDarwin in /opt/local, or are on a Sun and have extra software in /opt). In this case you might set CFLAGS=-I/opt/local/include and LDFLAGS=-L/opt/local/lib in the environment to help configure and the compiler find libraries. Note that this is rather a blunt instrument, and because you cannot really control where the respective flags appear in generated compiler commands, you can end up picking up inconsistent versions of software. That is, this is a mechanism for getting yourself out of a fix, not a recommended way of building the software.

The important point of this is that these environment variables do matter, and implicitly change the behaviour of ./configure. You should not have them set in your environment if you don’t want this to happen.

The values you set here act as defaults in the Makefile, and can be overriden at make time by giving arguments to make:

  % make ’CFLAGS=-g -I/somewhere/else’

2.2 Starconf, and the Starlink autoconf macros

The starconf application does some of the work of ensuring that your build directory is correctly organised. It does the following:

The ./bootstrap script is ‘owned’ by starconf, and you should not change it, since it will be overritten by starconf if a newer release of starconf includes an updated bootstrap script. If you do have some pressing reason to change it, then remove the word ‘original’ from the second line of the bootstrap file, which signals to starconf that it should leave the file untouched. The standard bootstrap script:

You need to run starconf explicitly only once, when you first prepare a directory for configuration. The ./bootstrap file which this creates itself runs starconf, so that each time you run the bootstrap script, you run starconf also. This has no effect unless either the bootstrap script or the macro set has been updated in the starconf installation, in which case starconf will propagate the updates to your directory. The ./starconf.status script should not be checked into the repository. The command starconf-validate (which is invoked by starconf in passing but which may be invoked explicitly also) will tell you what should and shouldn’t be checked in.

You might on other occasions run the ./starconf.status script. You will do this firstly to query important locations, such as the location of the starconf templates:

  % ./starconf.status --show buildsupportdata
  /export3/sun/buildsupport/share

See ./starconf.status –show –all or ./starconf.status –help for the list of all the things which you can show in this way (though be warned that this list is not yet completely stable, and may yet change).

These variables are fixed for a particular installation of starconf (you can in principle have more than one installation of starconf, and choose which one to invoke, but there is unlikely any need for that).

Two very important variables are STARCONF_DEFAULT_PREFIX and STARCONF_DEFAULT_STARLINK, discussed in Sec. 2.2.1 below.

A companion to the starconf application is the starconf-validate application. When run, this examines the current directory, checking that all required files are present, and checking that you have the correct files checked in to the repository. The command starconf-validate –help shows which files are in which category. Note that this applies only to directories which are fully Starlink applications – those in the libraries and applications directories; components in the thirdparty tree, on the other hand, have some starconf features such as a component.xml file, but are not valid according to starconf-validate; also ‘bundle’ directories such as libraries/pcs, which have no actual code in them, are not valid in this sense.

There are templates available for the most important starconf files. See Sec. 2.2.3.

The final component of the starconf system is the file component.xml. This is an XML file containing information about the particular component in the directory. The information in this file is redundant with some of the information you specify in configure.ac, and so the best way to ensure the two are consistent is to configure component.xml from configure.ac. To this end, there is a template component.xml.in file in the starconf ‘buildsupportdata’ directory. When you are preparing a directory to be build with the Starlink tools, copy this template-component.xml.in into the current directory under the name component.xml.in, and fill in those field which are not configured. Remember to uncomment the elements you fill in! See Sec. 2.2.4 for details. The files component.xml and component.xml.in should both be checked in.

2.2.1 STARCONF_DEFAULT_PREFIX and STARCONF_DEFAULT_STARLINK

Two very important variables are STARCONF_DEFAULT_PREFIX and STARCONF_DEFAULT_STARLINK. The first is the default installation prefix of the software you are building; the second is the location of a currently built Starlink tree, which will be used for applications and include files if they are not found under STARCONF_DEFAULT_PREFIX.

The value for each of these was frozen when starconf was itself built and installed, most typically at the time of the tree-wide bootstrap, and you can see the values for these with the command starconf –show STARCONF_DEFAULT_PREFIX for example. You can also see the results of this configuration in the ./configure script itself, as the command ./configure –help indicates within its output where material will be installed by default. It is not unreasonable to have more than one starconf installation, depending on your path, if you wish to have different frozen-in defaults here. The value of each of these two variables is typically something like /star.

It is important to emphasise that these parameters are frozen when starconf is built, and their values are ignored when a component is itself bootstrapped. If you need to change either value, then there are two ways you can do this.

The first, much more common, way is to provide the –prefix option when you run ./configure in a component (this overrides the frozen-in value of STARCONF_DEFAULT_PREFIX), or you can set the STARLINK variable as a ./configure argument line (or less securely default it from the environment, see Sec. 2.1.5 for discussion), overriding the frozen-in value of STARCONF_DEFAULT_STARLINK.

Specifying –prefix or STARLINK each time you run ./configure might be troublesome when you are working on a component’s configuration script, especially as doing this inconsistently would produce very confusing results. You might want to do this if you are working on a repository branch, and so want built material from the current directory to be installed in one branch-specific place, while using a Starlink tree based on the trunk. You can default this on a per-directory basis by using a m4 back-door. Create a file acinclude.m4 in the directory in question, and include in it a line like:

  m4_define([OVERRIDE_PREFIX],[/my-branch/star])

(or OVERRIDE_STARLINK to override that variable). Then run ./bootstrap, which invokes ‘autoreconf’ in turn, and the generated ./configure file will have the named directory as its default prefix. This method is in principle fragile, and uses a partly-deprecated autoconf interface, and so is not guaranteed to work for all time. It is at present adequately robust in practice, however, and so is a respectable technique as long as you are aware that it is to some extent a rather special effect.

In general, however, we recommend that you do not adjust –prefix, and that you leave the STARLINK variable unset. Also, since they are ignored at all times except when the whole tree is being configures, it might be wise not to have the STARCONF_DEFAULT_... variables set, which could trick you into believing that they are (not) having some effect.

2.2.2 Components

Throughout this documentation, the term ‘component’ is used rather than ‘package’. The distinction is that the components are the units of organisation within the CVS tree, and the packages are the units which are distributed as tarballs. These will generally overlap, but the mapping may not be one-to-one, so that the components within the CVS tree might not be reflected in ultimately distributed packages.

A ‘component directory’ is a directory which has a component.xml.in file in it. All component directories will have a manifest file created and installed in .../manifests; non-component directories will not have manifest files. Everything that’s installed must be installed as part of some component or other. This helps when you want to remove a component (a very primitive de-installer is therefore rm -f ‘sed ’1,/<;files>;/d;/<;files>;/,$d’ .../manifests/XXX‘), and packaging for distribution should be easy.

The starconf ./bootstrap scripts, which are installed by the starconf application, recurse into the directories listed in a configure.ac file’s AC_CONFIG_SRCDIR macro, and will stop recursing when they find a component.xml.in file. They’ll warn if they find a component.xml.in file in any AC_CONFIG_SUBDIRS directory, but ignore it, and exit with an error if they do not find a component.xml.in file and there are no AC_CONFIG_SUBDIRS directories in which to search further. That is, the tree of directories which the top-level bootstrap traverses should have component.xml.in files at or above all its leaves.

This further implies that a component ‘owns’ all the tree beneath the component directory, and thus that you cannot have one component located within another.

This means that bootstrap files only have to appear within component directories, but not their children, and starconf/autoreconf and starconf-validate only have to be run in component directories. Macros like STAR_DEFAULTS are still usable in the subdirectories’ component.ac files (because auto(re)conf in the parent handles those macros when it remakes the child configures).

Automake spots component.xml.in files and will only arrange to install a component manifest if it does fine a component.xml.in file. It also demands that the STAR_SPECIAL_INSTALL_COMMAND macro (see Sec. A.29) appear only in configure.ac files in a component directory (that macro is a sufficiently magic blunt instrument that it shouldn’t be allowed to lurk where it can’t have an eye kept on it).

The starconf-validate script checks these various assertions. As usual, starconf-validate should be regarded as pickily authoritative about how things are ideally configured, and if it objects to something, that means either that the the directory in question is non-ideally configured (and should be fixed), or the part of SSN/78 that suggested you set things up that way is out-of-date or unclear and should be clarified.

2.2.3 File templates

There are three templates available as part of the starconf system. They are located in the starconf ‘buildsupportdir’: given that the starconf application is in your path, you can find this location with starconf –show buildsupportdata, and if you have a starconf.status script in your local directory, you can find this directory with ./starconf.status –show buildsupportdata.

The three templates present are template-Makefile.am, template-configure.ac and template-component.xml.in.

2.2.4 The format of the component.xml file

The meaning of the elements in the component.xml DTD is as follows.

component: attribute ‘id’
The ID for this component. This must match the package name in configure.ac, and is the name under which the manifest file is installed. Let this remain under the control of configure.ac
component: attribute ‘status’
The status of this component, which can be current (the default, and assumed if no such attribute is present), or obsolete. When the Makefile.dependencies file is regenerated, the program doing that will warn if any component depends on a component marked obsolete.
version
The version number of this component. This must match the version number in configure.ac.
path

This is the path to this package within the CVS repository, without any leading slash. Make sure this is correct, or else the top-level build will probably fail.

description
A brief description of the component.
abstract
A fuller description of the component.
dependencies
This is the set of component IDs which this component depends on. This is managed by the STAR_DECLARE_DEPENDENCIES macros, and should not be adjusted by hand.
developers
This is a non-exhaustive list of those who have worked on this package. Little use is made of this at present, but its use is likely to extend, for example to be the source of recipients for CVS commit messages. The fields within it are <name>, which is the real name of the individual; <uname>, which is the username of the developer on the CVS system; and <email>, which is the person’s email address. The last two elements are not required.
documentation
A list of SUNs and the like. At this stage the format of this element is not completely finalised, and it is best to leave its maintenance to the STAR_LATEX_DOCUMENTATION macro.
bugreports
The email address for bug reports.
notes
Any other notes that might be of interest or utility.

Note that, since this is an XML file, you will have to escape the ampersand, left- and right-angle-bracket characters as &amp;, &lt; and &gt; respectively (well, you don’t actually have to escape the right angle bracket character, but it looks marginally neater if you do). There are no other entities defined in this context.

The DTD for this file is included in the repository, at buildsupport/starconf/componentinfo.dtd, with a RelaxNG version at buildsupport/starconf/componentinfo.rnc.

2.3 Version numbers

The only place where you should specify a version number is as the second argument to the AC_INIT macro; if you need the version number anywhere else, such as a bit of code which reports the package version number, you should put it there with the @PACKAGE_VERSION@ substitution variable, or one of the variants of this defined by STAR_DEFAULTS (see Sec. A.18).

Starlink follows the usual practice of indicating release versions with a three-part version number, major.minor-release. We have no particularly precise rules controlling these, but the major version number is typically incremented for a release which contains a major change in functionality or implementation, the minor is incremented for changes of functionality less significant than this, and the release number is incremented for bugfix releases, involving no functionality changes at all.

If a component contains a shared library, then the maintainer may elect to maintain shared library versioning information [XXX: should we make this a requirement for libraries?]. Since we use libtool, we can use its -version-info interface to maintain library versions. This is documented in section ‘Updating library version information’ of the libtool manual, but the explanation is repeated and elaborated below.

The crucial thing to remember is that the versioning information encoded in this -version-info option is not the component version number, but instead an encoding of which interface versions a given shared library implements, in the form of a triple current:revision:age. ’Current’ is the interface version, as a monotonically increasing integer, and ’age’ is a statement of how many interfaces are compatible with it: if a library has a triple c:r:a, then it implements all the interfaces between versions c-a and c, inclusive. When you’re making a new release of a library which had the -version-info c:r:a,

(1)
If the interface is unchanged, but the implementation has changed or been fixed, then increment r.
(2)
Otherwise, increment c and zero r.
(a)
If the interface has grown (that is, the new library is compatible with old code), increment a.
(b)
If the interface has changed in an incompatible way (that is, functions have changed or been removed), then zero a.

This is illustrated in Sec. 2.3.


pict
Figure 2.5: Updating -version-info specifications


The libtool documentation suggests starting the version-info specifications at 0:0:0, and this is the default if no -version-info option is present. Since only some Starlink libraries have this versioning maintained, it is best if you start the specification at 1:0:0, to distinguish this from those libraries which have no version-info defined.

You add the -version-info specification to the libtool command using the library’s LDFLAGS variable, as follows. First the Makefile.am (from the prm component):

  libprm_la_SOURCES = \
      $(F_ROUTINES)
      $(PUBLIC_INCLUDES) \
      $(PRIVATE_INCLUDES) \
      PRM_PAR
  libprm_la_LDFLAGS = -version-info $(libprm_la_version_info)
  
  libprm_a_la_SOURCES = \
      $(PLATFORM_C)
  libprm_a_la_LDFLAGS = -version-info $(libprm_a_la_version_info)
  [...]

and then the corresponding section from configure.ac:

  AC_INIT(prm, 1.3-1, starlink@jiscmail.ac.uk)
  # Version-info specifications.  See SSN/78 for guidelines, and update the table
  # below for ANY change of version number.
  #
  # Release    libprm.la      libprm_a.la
  #  1.3-1       1:0:0          1:0:0
  AC_SUBST(libprm_la_version_info,   1:0:0)
  AC_SUBST(libprm_a_la_version_info, 1:0:0)
  
  [...]

There is no automake magic to the libprm_la_version_info variable name – it is just mnemonic, and you are free to change it if you have a good reason to do so. Since there is no fixed relationship between the component version number and the -version-info specification, it is important to maintain a simple table of associations between the two, and the configure.ac file is a sensible place to do that.

2.4 A few remarks on state

There is quite a lot of state held within the starconf build tree. This section contains a few notes on where this state is, and how to handle it.

Since most of the details here are rather intricate, you might save time by looking at the list of FAQs on state in Sec. 5.3.

Autotool state: The most obvious state within the tree is the existence of the configure and Makefile.in files which are the generated by autoconf and automake. Files config.h.in and aclocal.m4, are also part of this process, inasmuch as they are generated by autoreconf: config.h.in is based on the declarations within configure.ac; aclocal.m4 is a local cache of m4 macros, drawn from the starconf, autoconf, automake and libtool installations. Other objects generated by the autotools are the cache directory autom4te.cache and the tool links config.guess, config.sub, depcomp, install-sh, ltmain.sh, missing and stamp-h1. All of this can be happily blown away by hand, and a lot of it is removed by make maintainer-clean, though this doesn’t remove things like config.h.in which are required for ./configure to run. Most of this state is maintained reliably by the dependencies within Makefile.in – for example if you update Makefile.am, then a simple make will remake Makefile.in and regenerate Makefile before using the new Makefile to do the build. If you have removed this state or suspect that it may be out of date, then autoreconf will always regenerate it – it’s always safe to rerun autoreconf.

Configuration state: When ./configure runs, its most obvious effect is to transform the .in files into their corresponding configured results. However at the same time, it creates a log file in config.log (which can be handy to examine if a configure script fails or appears to give wrong answers), a cache in config.cache if you ran ./configure with the -C option, it creates the directory-specific ./libtool script, and it creates a file config.status. This last file holds most of ./configure’s state, and can be used to either regenerate a particular file (./config.status Makefile) or else to update itself (./config.status –recheck has the effect of rerunning ./configure with the same options such as –prefix which were given to the ./configure script last time). You won’t often have to run ./config.status yourself, but it’s reassuring to know what it’s doing when the makefile runs it for you. It is always regenerated afresh by running ./configure.

Build state: The compiled files are obviously part of the build state, and can be removed by make clean and regenerated with plain make. Less obviously part of this state are the directory .libs, which is libtool’s’ workspace and holds different types of objects and libraries, and .deps, which holds dependency information gleaned as part of the last build. Less obviously still are those (few) .in files which are configured as part of the build. As mentioned in Sec. 2.1.2, so-called ‘installation directory variables’ should be substituted at make time rather than build time, with a hand-written makefile rule in Makefile.am.

Starconf state: There is essentially no starconf state, since the starconf system’s only role is to manage the bootstrap file and provide the extra autoconf macros (installed in the buildsupport part of the tree when starconf itself is installed). The starconf.status file is purely a cache of starconf variables, and allows you to locate the starconf installation which was used most recently. At one time, the starconf.status file did hold state, and allowed you to manipulate it, and was even checked in; this is no longer the case.

Build tree and installation state: The last noteworthy state is that in the build tree as a whole. The top-level makefile requires some state in order to manage its bootstrapping ‘make world’ build. This build is organised by representing the build and link dependencies between components by makefile dependencies between the components’ installed manifests, which are installed in the .../manifests directory alongside, and at the same time, as the component is installed by the install Makefile target. Thus this is state held outside of the build tree. If the ‘make world’ build sees that a manifest file is up-to-date with respect to the dependencies expressed in Makefile.dependencies , it makes no attempt to build it, even if the component itself has been updated and itself needs rebuilding and reinstalling. A slight wrinkle here is that the ‘buildsupport’ tools – namely starconf and the autotools – are not explicitly listed as being a dependency of anything, since they are in fact a dependency of everything. Since they are rather a special case, these are built within the  top-level bootstrap script, and the best way to run that ‘by hand’ is via the command ./bootstrap –buildsupport, noting that the remarks about dependencies above apply to this process also. Thus, if you update a component – including one of the buildsupport components – and wish the top-level ‘make world’ build to notice this, the best way to do this is to first delete that component’s manifest file from the installation tree. This process might change slightly with starconf developments (see Sec. C).

CVS state: This isn’t really part of the build system as such, but this seems a good place to point out, or reassure you, that all the state for a CVS checkout directory is in the files in the CVS subdirectory, and that all the repository state for the directory is in the files in the corresponding directory within the repository.

2.5 Preprocessable Fortran

The Starlink build system includes support for preprocessable Fortran, in both autoconf and automake.

As described in Sec. B, the installed version of autoconf is extended with support for preprocessable Fortran, by adding the two macros AC_PROG_FC and AC_PROG_FPP.

You should use AC_PROG_FC in preference to the macro AC_PROG_F77 described in the autoconf manual, since the ‘FC’ support in autoconf is more flexible and more developed than the ‘F77’ support, and the older macro is likely to be deprecated in coming autoconf releases. However a potential problem with the AC_PROG_FC macro is that it searches for Fortran 9x compilers before Fortran 77 ones. Fortran 9x incorporates all of strict Fortran 77 as a subset, but no more, so if you have used any of the common Fortran extensions (which is sometimes unavoidable), you might find the Fortran compiler objecting. In this case, you should use the second optional argument to AC_PROG_FC to specify Fortran 77 as the required dialect:

  AC_PROG_FC([], 77)

See Sec. A.12 for full details.

Unlike C, there is no preprocessor defined as part of the Fortran standard, and this is inconvenient if you wish to use a Fortran extension if it is available, but not have the compilation fail if it is absent. This most commonly applies to the READONLY keyword on the Fortran OPEN statement. It is possible to use the C preprocessor, cpp, with Fortran, though not completely reliably, since the underlying syntaxes of C and Fortran are so different that cpp is capable of emitting invalid Fortran in some circumstances.

Both the Sun and GNU Fortran compilers can handle cpp-style preprocessor constructs in a Fortran file, avoiding any separate preprocessing stage. Sun have produced a Fortran preprocessor, fpp, which is available at http://www.netlib.org/fortran/: it’s freely available, but not open-source. And more often than not we can in fact fall back on simple cpp, as long as we have confined ourselves to the #include, #define and #if... constructs. If it comes to that, such simple preprocessing could be mostly handled with a simple Perl script.

Starlink autoconf and automake between them add support for preprocessable Fortran. Autoconf supplies the AC_PROG_FPP macro described in Sec. A.13, to investigate what compiler and preprocessor are available, and automake works with the results of that test to add appropriate support to the generated Makefile.in.

For example, you might have in your configure.ac file the lines

  AC_PROG_FC
  AC_PROG_FPP
  AC_FC_OPEN_SPECIFIERS(readonly,[access=’sequential’,recl=1])

and have in your my_io.F the lines

  #include <config.h>
        SUBROUTINE MYIO(...)
  
  * blah...
  
        OPEN ( UNIT = LUCON, FILE = IFCNAM, STATUS = ’OLD’,
  #ifdef HAVE_FC_OPEN_READONLY
       :       READONLY,
  #endif
  #ifdef HAVE_FC_OPEN_ACCESSSEQUENTIALRECL1
       :       ACCESS=’SEQUENTIAL’,RECL=1,
  #endif
       :       FORM = ’UNFORMATTED’, IOSTAT = ISTAT )

The file my_io.F is listed in the Makefile.am as one of the _SOURCES of whatever library or application it contributes to, alongside the other Fortran files. If the compiler discovered by the ./configure script can support preprocessor statements itself, then the .F file is treated like any other; if not, and a separate preprocessing stage is required, then the Makefile handles this itself.

Note that we are using .F here as the file extension for preprocessable Fortran: see Sec. A.13 for discussion.

In those cases where you wish the preprocessing step alone, such as when you are generating an include file from a template, you should name the input file with a .F extension also. You will need to include a preprocessing rule in the makefile. The following idiom might be helpful:

  # Run a .F file through the fpp preprocessor, to produce a file with
  # no extension.
  # The following deals with case-insensitive filesystems, on which
  # foo.f and foo.F would be the same file.  FPP_OUTPUT is
  # either "" (in which case the preprocessor writes to foo.f, and
  # the filesystem is presumably case-sensitive) or ">$@".
  foo: foo.F
          rm -f foo
          $(FPP) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) \
             $(FPPFLAGS) $(CPPFLAGS) foo.F $(FPP_OUTPUT)
          test -f foo || mv foo.f foo

or the following, for a suffix rule

  SUFFIXES = .F
  .F:
          rm -f $@
          $(FPP) $(DEFAULT_INCLUDES) $(FPPFLAGS) $(CPPFLAGS) $< $(FPP_OUTPUT)
          test -f $@ || mv $(<:.F=.f) $@

In this second example, in which we have elided the useful comment, we have declared the .F suffix (we wouldn’t have to do this if there were ‘real’ .F sources in this directory), and are using the $< and $@ magic variables.

In second example, we elided most of the flag variables we included in the first one, since we probably don’t need them. In case of doubt, copy the .F.f: rule in one of the generated Makefile.in files.

There is a more elaborate example of this in the prm component.

2.6 Using the Starlink CVS repository

The Starlink source code is held in a CVS repository, which has public readonly access.

If you have an account on the repository machine, you have read and write access to the full source set. When you are making a fresh checkout, or giving other commands not within the context of a local check-out, you should refer to it as

  % cvs -d :ext:\emph{username}@cvs.starlink.ac.uk:/cvs <cvs-command>

where username is your username on that system; you will need to set the environment variable CVS_RSH to ssh to connect to the repository.

There is also anonymous CVS access to the repository. Use

  % cvs -d :pserver:anonymous@cvs.starlink.ac.uk:/cvs login
  # password starlink
  % cvs -d :pserver:anonymous@cvs.starlink.ac.uk:/cvs <cvs-command>

There is also anonymous web-based access to the repository at http://cvsweb.starlink.ac.uk. See http://dev.starlink.ac.uk/ for news.

There are a few more details about CVS, including a link to a CVS Primer, in Sec. 2.7.

Part of the point of the Starlink CVS repository is to give users ready access to the fully up-to-date sources so that more sophisticated users can build the most recent application versions for themselves and even, if they find a bug, offer fixes. Anyone who finds a bug is invited to report it through the Project’s bug-reporting system at http://dev.starlink.ac.uk/bugzilla/, but if this report comes with fixes, it will be particularly welcome.

If you do have bugfixes to offer, then you should get in touch with the ‘owner’ of the component to discuss how to give them to the project. You will find the owner of a component by looking at the file component.xml in the component’s checkout directory; this should list those who have worked on the component, along with someone nominated as the component’s ‘owner’.

If you make quite a few fixes, then it might be best to give you committer access to the repository, by giving you an account on the repository machine. Talk to someone from the project about setting that up.

2.7 CVS

There is a compact CVS Primer on the web at http://purl.org/nxg/note/cvsprimer, and that includes pointers to fuller documentation. This section includes a few tips on using CVS which go beyond that document, or which are more specific to the Starlink CVS repository.

2.7.1 CVS and tagging

Tagging is very important, as it is through tagging that you create branches, and mark certain sets of files as a set, so that they can be recovered as a set in future. The current tagging policy is at http://wiki.starlink.ac.uk/twiki/bin/view/Starlink/CvsTagging.

You make a tag with the cvs tag command, while you are in a directory with a part of the repository checked out:

  cvs tag <tag-name>

This applies the tag to the repository versions indicated by the files in the current directory. In the most common case, you have just finished preparing the set of files for a release, so all the files in the directory are as you want them to be, and committed. There’s a slight trap here: if there are any files which are not committed, then it is the repository version which corresponds to the modified file which is tagged, not the modified file itself (this is never useful; it is simply a warning to use the tag command only in a fully-committed directory). If you tag a set of files which are on a branch, then it is (probably unsurprisingly) the branched files which are tagged.

There is also a cvs rtag command which is similar, but operates only on the repository. You won’t need to use rtag; don’t confuse the two.

2.7.2 CVS and recursion – checking out only a subtree

There are two traps in the way that CVS recurses into subdirectories. The first is that if a subdirectory is present in the repository but not in your checkout (most commonly because it has been recently added to the repository by someone else), then CVS will not by default add that subdirectory when you do an update, and will give no indication that it is there. It is not clear to me why this is a sensible default, but it is the case nonetheless. Only if the -d option is present on the cvs update command will the ‘missing’ directory appear.

The other ‘trap’ is that CVS does recurse by default when you do a checkout. This is almost always the right thing, but it can be inconvenient when you want just the top level of the repository. If you wanted to check out only the top level, or only the buildsupport tools, then the command cvs -d ??? checkout . would not be the right thing, since it would check out the top level and everything underneath it. A better set of commands would be

  cvs -d username@cvs.starlink.ac.uk:/cvs co -l .
  cvs -d username@cvs.starlink.ac.uk:/cvs co buildsupport
  cvs -d username@cvs.starlink.ac.uk:/cvs co thirdparty/fsf

The first line checks out the top level directory but, because of the -l (‘local’) flag, does not recurse and check out the complete repository. The following lines check out particular directories, usefully recursing into their children.

If you want to check out just the applications directory, but none of its children, use

  cvs -d username@cvs.starlink.ac.uk:/cvs co -l applications

while in the top-level directory. Don’t do ... co -l . in the applications directory – you’ll get the top-level directory again.