Share Email Print

Proceedings Paper

The common pipeline library: standardizing pipeline processing
Author(s): Derek J. McKay; Pascal Ballester; Klaus Banse; Carlo Izzo; Yves Jung; Michael Kiesgen; Nick Kornweibel; Lars K. Lundin; Andrea Modigliani; Ralf M. Palsa; Cyrus Sabet
Format Member Price Non-Member Price
PDF $14.40 $18.00

Paper Abstract

The European Southern Observatory (ESO) develops and maintains a large number of instrument-specific data processing pipelines. These pipelines must produce standard-format output and meet the need for data archiving and the computation and logging of quality assurance parameters. As the number, complexity and data-output-rate of instrument increases, so does the challenge to develop and maintain the associated processing software. ESO has developed the Common Pipeline Library (CPL) in order to unify the pipeline production effort and to minimise code duplication. The CPL is a self-contained ISO-C library, designed for use in a C/C++ environment. It is designed to work with FITS data, extensions and meta-data, and provides a template for standard algorithms, thus unifying the look-and-feel of pipelines. It has been written in such a way to make it extremely robust, fast and generic, in order to cope with the operation-critical online data reduction requirements of modern observatories. The CPL has now been successfully incorporated into several new and existing instrument systems. In order to achieve such success, it is essential to go beyond simply making the code publicly available, but also engage in training, support and promotion. There must be a commitment to maintenance, development, standards-compliance, optimisation, consistency and testing. This paper describes in detail the experiences of the CPL in all these areas. It covers the general principles applicable to any such software project and the specific challenges and solutions, that make the CPL unique.

Paper Details

Date Published: 16 September 2004
PDF: 9 pages
Proc. SPIE 5493, Optimizing Scientific Return for Astronomy through Information Technologies, (16 September 2004); doi: 10.1117/12.551214
Show Author Affiliations
Derek J. McKay, European Southern Observatory (Germany)
Rutherford Appleton Lab. (United Kingdom)
Pascal Ballester, European Southern Observatory (Germany)
Klaus Banse, European Southern Observatory (Germany)
Carlo Izzo, European Southern Observatory (Germany)
Yves Jung, European Southern Observatory (Germany)
Michael Kiesgen, European Southern Observatory (Germany)
MBA Michael Bailey Associates (Germany)
Nick Kornweibel, European Southern Observatory (Germany)
Lars K. Lundin, European Southern Observatory (Germany)
Andrea Modigliani, European Southern Observatory (Germany)
Ralf M. Palsa, European Southern Observatory (Germany)
Cyrus Sabet, European Southern Observatory (Germany)
Tekom GmbH (Germany)

Published in SPIE Proceedings Vol. 5493:
Optimizing Scientific Return for Astronomy through Information Technologies
Peter J. Quinn; Alan Bridger, Editor(s)

© SPIE. Terms of Use
Back to Top