Share Email Print

Proceedings Paper

The MWA archive infrastructure: archiving terabytes of data over dedicated WAN connections
Author(s): Andreas Wicenec; Dave Pallot; Alessio Checcucci; Slava Kitaeff; Kevin Vinsen; Chen Wu
Format Member Price Non-Member Price
PDF $17.00 $21.00

Paper Abstract

The Murchison Wide Field Array (MWA) is being upgraded from 32 tiles to 128 tiles of 16 dual-polarization dipole antennas. In the course of this project the software and the data infrastructure are also undergoing a major overhaul in order to cope with a more continuous and remote operational model; and the substantial increase in data rate (400 MB/s from the correlator plus 160 MB/s from the real time imaging pipeline). During the course of 2012/13 the data collected by the MWA will be transported via a dedicated 40 Gbit WAN network link between the Murchison Radio Observatory (MRO) and Perth (700 km). However, this network will not be available for some time; and until then, the data will be transported using disk arrays instead. Once in Perth, the data will be ingested into a tape library. The archiving process itself consists of various steps executed either at the MRO site or in Perth, and makes use of a modied version of the archiving system from the Atacama Large Millimeter Array (ALMA). This includes the extraction of meta-data from the original raw data and the ingestion and generation of the appropriate data links for the MWA archive. The MWA correlator generates a collection of small les all belonging to the same observation. In order to optimise the network transfer and the storage of those les they are transparently packed into larger containers at the MRO site and subsequently handled as one big le. This paper describes the setup of the MWA archiving infrastructure.

Paper Details

Date Published: 24 September 2012
PDF: 7 pages
Proc. SPIE 8451, Software and Cyberinfrastructure for Astronomy II, 845118 (24 September 2012); doi: 10.1117/12.925901
Show Author Affiliations
Andreas Wicenec, The Univ. of Western Australia (Australia)
Dave Pallot, Curtin Univ. (Australia)
Alessio Checcucci, The Univ. of Western Australia (Australia)
Slava Kitaeff, The Univ. of Western Australia (Australia)
Kevin Vinsen, The Univ. of Western Australia (Australia)
Chen Wu, The Univ. of Western Australia (Australia)

Published in SPIE Proceedings Vol. 8451:
Software and Cyberinfrastructure for Astronomy II
Nicole M. Radziwill; Gianluca Chiozzi, Editor(s)

© SPIE. Terms of Use
Back to Top