Share Email Print

Proceedings Paper

Toward sustainable deployment of distributed services on the cloud: dockerized ODI-PPA on Jetstream
Author(s): Raymond W. Perigo; Arvind Gopu; Michael D. Young; Yuanzhi Bao
Format Member Price Non-Member Price
PDF $17.00 $21.00

Paper Abstract

The One Degree Imager - Portal, Pipeline and Archive (ODI-PPA) - a mature and fully developed product - has been a workhorse for astronomers observing on the WIYN ODI. It not only provides access to data stored in a secure archive, it also has a rich search and visualization interface, as well as integrated pipeline capabilities connected with supercomputers at Indiana University in a manner transparent to the user. As part of our ongoing sustainability review process, and given the increasing age of the ODI-PPA codebase, we have considered various approaches to modernization. While industry currently trends toward Node.js based architectures, we concluded that porting an entire legacy PHP and Python-based system like ODI-PPA with its complex and distributed service stack would require too significant an amount of human development/testing/deployment hours. Aging deployment hardware with tight budgets is another issue we identified, a common one especially when deploying complex distributed service stacks. In this paper, we present DockStream (, an elegant solution that addresses both of the aforementioned issues. Using ODI-PPA as a case study, we present a proof of concept solution combining a suite of Docker containers built for each PPA service and a mechanism to acquire cost-free computational and storage resources. The dockerized ODI-PPA services can be deployed on one Dockerenabled host or several depending on the availability of hardware resources and the expected levels of use. In this paper, we describe the process of designing, creating, and deploying such custom containers. The NSF-funded Jetstream led by the Indiana University Pervasive Technology Institute (PTI), provides cloud-based, on-demand computing and data analysis resources, and a pathway to tackle the issue of insufficient hardware refreshment funds. We briefly describe the process to acquiring computational and storage resources on Jetstream, and the use of the Atmosphere web interface to create and maintain virtual machines on Jetstream. Finally, we present a summary of security refinements to a dockerized service stack on the cloud using nginx, custom docker networks, and Linux firewalls that significant decrease the risk of security vulnerabilities and incidents while improving scalability.

Paper Details

Date Published: 6 July 2018
PDF: 11 pages
Proc. SPIE 10707, Software and Cyberinfrastructure for Astronomy V, 107072X (6 July 2018); doi: 10.1117/12.2313647
Show Author Affiliations
Raymond W. Perigo, Indiana Univ. (United States)
Arvind Gopu, Indiana Univ. (United States)
Michael D. Young, Indiana Univ. (United States)
Yuanzhi Bao, Indiana Univ. (United States)

Published in SPIE Proceedings Vol. 10707:
Software and Cyberinfrastructure for Astronomy V
Juan C. Guzman; Jorge Ibsen, Editor(s)

© SPIE. Terms of Use
Back to Top
Sign in to read the full article
Create a free SPIE account to get access to
premium articles and original research
Forgot your username?