Share Email Print
cover

Proceedings Paper

An efficient framework for Java data processing systems in HPC environments
Author(s): Aidan Fries; Javier Castañeda; Yago Isasi; Guillermo L. Taboada; Jordi Portell de Mora; Raül Sirvent
Format Member Price Non-Member Price
PDF $17.00 $21.00
cover GOOD NEWS! Your organization subscribes to the SPIE Digital Library. You may be able to download this paper for free. Check Access

Paper Abstract

Java is a commonly used programming language, although its use in High Performance Computing (HPC) remains relatively low. One of the reasons is a lack of libraries offering specific HPC functions to Java applications. In this paper we present a Java-based framework, called DpcbTools, designed to provide a set of functions that fill this gap. It includes a set of efficient data communication functions based on message-passing, thus providing, when a low latency network such as Myrinet is available, higher throughputs and lower latencies than standard solutions used by Java. DpcbTools also includes routines for the launching, monitoring and management of Java applications on several computing nodes by making use of JMX to communicate with remote Java VMs. The Gaia Data Processing and Analysis Consortium (DPAC) is a real case where scientific data from the ESA Gaia astrometric satellite will be entirely processed using Java. In this paper we describe the main elements of DPAC and its usage of the DpcbTools framework. We also assess the usefulness and performance of DpcbTools through its performance evaluation and the analysis of its impact on some DPAC systems deployed in the MareNostrum supercomputer (Barcelona Supercomputing Center).

Paper Details

Date Published: 12 October 2011
PDF: 14 pages
Proc. SPIE 8183, High-Performance Computing in Remote Sensing, 818306 (12 October 2011); doi: 10.1117/12.898217
Show Author Affiliations
Aidan Fries, Univ. of Barcelona (Spain)
Institute for Space Studies of Catalonia (Spain)
Institute of Cosmos Sciences (Spain)
Javier Castañeda, Univ. of Barcelona (Spain)
Institute for Space Studies of Catalonia (Spain)
Institute of Cosmos Sciences (Spain)
Yago Isasi, Univ. of Barcelona (Spain)
Institute for Space Studies of Catalonia (Spain)
Institute of Cosmos Sciences (Spain)
Guillermo L. Taboada, Univ. of A Coruña (Spain)
Jordi Portell de Mora, Univ. of Barcelona (Spain)
Institute for Space Studies of Catalonia (Spain)
Institute of Cosmos Sciences (Spain)
Raül Sirvent, BSC-CNS Barcelona Supercomputing Ctr. (Spain)


Published in SPIE Proceedings Vol. 8183:
High-Performance Computing in Remote Sensing
Bormin Huang; Antonio J. Plaza, Editor(s)

© SPIE. Terms of Use
Back to Top