Share Email Print
cover

Proceedings Paper

The NOAO Data Lab virtual storage system
Author(s): Matthew J. Graham; Michael J. Fitzpatrick; Patrick Norris; Kenneth J. Mighell; Knut Olsen; Elizabeth B. Stobie; Stephen T. Ridgway; Adam S. Bolton; Abhijit Saha; Lijuan Wendy Huang
Format Member Price Non-Member Price
PDF $14.40 $18.00
cover GOOD NEWS! Your organization subscribes to the SPIE Digital Library. You may be able to download this paper for free. Check Access

Paper Abstract

Collaborative research/computing environments are essential for working with the next generations of large astronomical data sets. A key component of them is a distributed storage system to enable data hosting, sharing, and publication. VOSpace1 is a lightweight interface providing network access to arbitrary backend storage solutions and endorsed by the International Virtual Observatory Alliance (IVOA). Although similar APIs exist, such as Amazon S3, WebDav, and Dropbox, VOSpace is designed to be protocol agnostic, focusing on data control operations, and supports asynchronous and third-party data transfers, thereby minimizing unnecessary data transfers. It also allows arbitrary computations to be triggered as a result of a transfer operation: for example, a file can be automatically ingested into a database when put into an active directory or a data reduction task, such as Sextractor, can be run on it. In this paper, we shall describe the VOSpace implementations that we have developed for the NOAO Data Lab. These offer both dedicated remote storage, accessible as a local file system via FUSE, and a local VOSpace service to easily enable data synchronization.

Paper Details

Date Published: 26 July 2016
PDF: 8 pages
Proc. SPIE 9913, Software and Cyberinfrastructure for Astronomy IV, 99131I (26 July 2016); doi: 10.1117/12.2233306
Show Author Affiliations
Matthew J. Graham, National Optical Astronomy Observatory (United States)
California Institute of Technology (United States)
Michael J. Fitzpatrick, National Optical Astronomy Observatory (United States)
Patrick Norris, National Optical Astronomy Observatory (United States)
Kenneth J. Mighell, National Optical Astronomy Observatory (United States)
Knut Olsen, National Optical Astronomy Observatory (United States)
Elizabeth B. Stobie, National Optical Astronomy Observatory (United States)
Stephen T. Ridgway, National Optical Astronomy Observatory (United States)
Adam S. Bolton, National Optical Astronomy Observatory (United States)
Abhijit Saha, National Optical Astronomy Observatory (United States)
California Institute of Technology (United States)
Lijuan Wendy Huang, National Optical Astronomy Observatory (United States)


Published in SPIE Proceedings Vol. 9913:
Software and Cyberinfrastructure for Astronomy IV
Gianluca Chiozzi; Juan C. Guzman, Editor(s)

© SPIE. Terms of Use
Back to Top