Share Email Print

Proceedings Paper

Towards dynamic light-curve catalogues
Author(s): Bart Scheers; Fabian Groffen
Format Member Price Non-Member Price
PDF $14.40 $18.00
cover GOOD NEWS! Your organization subscribes to the SPIE Digital Library. You may be able to download this paper for free. Check Access

Paper Abstract

Time-domain astronomy is becoming a fundamental aspect of the next generation of astronomical instruments. The timing properties will revolutionise the studies of all kinds of astronomical objects. Consequetially, the huge complex data volumes and high cadences of these facilities will force us to overhaul and extend current software solutions. LOFAR, laying the groundwork for this, will produce a continuously updated spectral light-curve catalogue of all detected sources, with real-time capabilities to cope with the growth of 50 - 100TB/yr, making it the largest dynamic astronomical catalogue. Automated pipelines use the column-store MonetDB as their key component. We exploit SciLens, a 300+ node, 4-tier locally distributed cluster focussed on massive I/O. Introduction of the new array-based query language, SciQL, simplifies data exploration and mining. I will demonstrate how MonetDB/SQL & SciQL on its SciLens platform manages the millions of lightcurves for LOFAR. Initial benchmark results confirm the linear scale-up performance over tens of TBs using tens of nodes.

Paper Details

Date Published: 24 September 2012
PDF: 8 pages
Proc. SPIE 8451, Software and Cyberinfrastructure for Astronomy II, 84510M (24 September 2012); doi: 10.1117/12.926069
Show Author Affiliations
Bart Scheers, Astronomical Institute Anton Pannekoek, Univ. of Amsterdam (Netherlands)
Ctr. Wiskunde & Informatica (Netherlands)
Fabian Groffen, Ctr. Wiskunde & Informatica (Netherlands)

Published in SPIE Proceedings Vol. 8451:
Software and Cyberinfrastructure for Astronomy II
Nicole M. Radziwill; Gianluca Chiozzi, Editor(s)

© SPIE. Terms of Use
Back to Top