Chandra standard data processing involves hundreds of different types of data products and pipelines. Pipelines are initiated by different types of events or notifications and may depend upon many other pipelines for input data. The Chandra automated processing system (AP) was designed to handle the various notifications and orchestrate the pipeline processing. Certain data sets may require "special" handling that deviates slightly from the standard processing thread. Also, bulk reprocessing of data often involves new processing requirements. Most recently, a new type of processing to produce source catalogs has introduced requirements not anticipated by the original AP design. Managing these complex dependencies and evolving processing requirements in an efficient, flexible, and automated fashion presents many challenges. This paper describes the most significant of these challenges, the AP design changes required to address these issues and the lessons learned along the way.

Date Published: 30 June 2006
PDF: 12 pages
Proc. SPIE 6270, Observatory Operations: Strategies, Processes, and Systems, 62701W (30 June 2006); doi: 10.1117/12.672293

Published in SPIE Proceedings Vol. 6270:
Observatory Operations: Strategies, Processes, and Systems
David R. Silva; Rodger E. Doxsey, Editor(s)