The improving resolution of remote sensing (RS) images is causing the amount of data in such images to increase with a geometric progression. A very large application system can now contain terabytes, even petabytes. Furthermore, these large quantities of data are generally dispersed in different places for storage. Research on how to store and manage massive distributed RS images is vital.
Three main modes of RS image storage and management currently exist: file-based mode, where images are stored as files in network storage devices; database mode, where a special large database manages the images; and a combined mode, where the metadata and index information is stored in a database while the images themselves are stored as files in a file server. All three modes use traditional network storage technologies: direct attached storage (DAS), which is attached to a single server; network attached storage (NAS), a server dedicated to file sharing; and storage area networks (SAN), networks of shared storage devices. When a very large amount of data needs to be accessed in an application system using one of these technologies, metadata operations become the bottleneck. As such, it is difficult to design an integrated storage and management system that offers both high performance network storage and secure data sharing across platforms.
To enhance the performance of NAS and to enable data sharing across SAN platforms, object-based storage (OBS) was recently developed. In traditional storage models, the user and storage components of a file system are centralized in a server. In the object-based storage model, however, the file system is divided into two parts.1,2 The file system user component contains such functions as hierarchy management, naming, and user access control. The file system storage management component is offloaded to an OBS device (OBSD) as the OBS storage management component and focuses on mapping logical constructs, such as files or database entries, to the physical organization of the storage media.
We have constructed an OBS model for the storage and management of distributed images. In this model, the images are organized as objects according to their parameters and then stored in the OBSD, and their attributes are stored in the metadata server (MDS). The database system and file system user components still stand in the RS applications server. The database and file system storage management components are offloaded to the OBSD as the RS object storage management component.
The architecture of an OBS-based application system for distributed RS images is composed of several RS application servers, MDSs, and OBSDs. All of these are independent, so the data, control, and management paths are separate. The MDSs manage the attributes of the RS objects stored in the OBSDs, which can be directly accessed by RS application servers using multiple SCSI initiator ports. In this architecture, storage space is managed and allocated by a storage controller: as such, there's no need to operate the file system in the host computer. In comparison to traditional storage architectures, the OBS architecture for distributed RS images has the characteristics of a high-performance storage service: extendibility, distributed metadata, intelligence, and security.1,2
We ran our prototype implementation on several computers with Pentium-4 2.6GHz CPUs and 1GB memory running Linux RedHat with kernel version 2.4.21. We found that the write performance of the OBS model is far better than that of a network file system based on traditional network storage models, especially for large files, and is linearly enhanced by increasing the number of OBSDs. Single RS images are often huge, and a very-large-application system needs a high data bandwidth storage system to adapt to frequent data access. The OBS model for distributed RS images fits this requirement.
The OBS model for distributed RS images provides a feasible storage framework for the high-performance parallel data access and distributed management of metadata of massive, distributed RS images. Further research will explore a smart storage plugin based on OBS to enhance the storage and management of distributed RS images.