dealing with data

The past several decades of remote sensing hardware development have yielded a range of high-resolution and multispectral systems for airborne and satellite deployment. Such systems regularly deliver an unprecedented quantity of Earth-observation data a very short time after acquisition, so that the engineering challenge has shifted from image acquisition to data management. Extensive (and expensive) data-gathering efforts in government and industry will be in vain if the information is not easily retrieved, carefully preserved, and secured for use by the intended audience only.

There is an increasing recognition, in both public and private sectors, of the importance of managing all of this digital imagery and associated derived products, including geographical information systems (GIS) applications. Stephen Cambone, undersecretary of defense for intelligence and a keynote speaker at the GeoIntel conference (October 2003), discussed the need to transform the use of geospatial assets in the military and intelligence establishments. In the current environment, he said, we must "know something about everything all the time" rather than know everything about a few things (as in the Cold War era). At the same time, more and more commercial enterprises are using geospatial information as an essential element of their business strategy. And, of course, the concern with homeland security has led to a critical need for geospatial databases that can be accessed effectively and efficiently in the event of an emergency. These major paradigm shifts in how government and commercial enterprises do business imply a critical need for intelligent management of digital assets of all kinds, and particularly of geospatial digital assets.

Another major driver in the geospatial digital asset management (G-DAM) marketplace is the growth of so-called location-enabled solutions, i.e., the use of geospatial information in industries for which there is no fundamental geographical content. An example would be the insurance industry, in which location, including imagery of the surroundings, may be used to identify risk exposures.

Satellite collection capability has in many ways outstripped the ability to handle the data effectively on the ground. A single high-resolution satellite image of the type currently being acquired by commercial satellites can be as large as 700 MB. Tools to convert imagery to information play a crucial role in remote sensing. Equally important for the commercial marketplace and for many government applications is the ability to store and distribute the data and the products in a timely manner. Indeed, the continued growth and health of the remote sensing industry and related geospatial enterprises depend on broad adoption of this technology.

Figure 1. A G-DAM system provides processing and product-generation capabilities, as well as sophisticated search/retrieval and data archiving capabilities.

G-DAM systems are crucial to handling the petabytes of data and associated information products generated by remote sensing systems (see figure 1). G-DAM systems use automation techniques to speed the archiving and image processing of geospatial data. They must be capable of integration with exploitation tools, GIS layers, and other metadata in order to move beyond GIS to spatial decision support systems (see table). The concept of a geospatially oriented DAM system is relatively new. Some are deployed in sensitive government installations in the United States and abroad. The news media have made some use of these systems. Rapid broadcast of remotely sensed imagery in the wake of recent events is an illustration of the utility of intelligent archiving.

In the geospatial context, a G-DAM package ingests imagery, stores it intelligently, preserves it, and incorporates interfaces to a broad variety of image manipulation tools. It provides searchable, secure archives and addresses all aspects of the digital media processing chain. Effective G-DAM solutions go beyond incorporating software and hardware for automated asset processing, management, retrieval, and preservation. They provide optional integrated software solutions for imagery analysis and a specialized joystick for smooth imagery manipulation.

The user needs efficient search and retrieval and an easy-to-handle catalog system. G-DAM solutions must incorporate a variety of techniques to search and retrieve imagery from the facility archive. The archive should be searchable by location, date, and subject matter, and the process should be able to be implemented by drawing on an onscreen map or entering parameters. To achieve this full range of capability requires intelligent storage.

If a system has stored data intelligently, it is easy and quick to search and retrieve by date, location (especially critical in the geospatial context), and other parameters relevant to the application at hand. For example, in an agricultural application, both time of day and season of the year may be important parameters. For cases in which multiple sets of data, acquired in different ways, exist for the same area, users need to be able to search using metadata related to the cameras used for the various images.

Data that is more immediate in some sense (whether temporal or otherwise) should be more readily accessible. If the application is time-critical (e.g., the news media), then it is essential that images acquired most recently be in the online storage areas.

The variety of remote sensing applications demands that the original archive be able to be enhanced with derived products. The search mechanism implemented in the DAM must allow for this—a user should be able to retrieve original imagery and associated derived product. A combination of hardware and software architecture will enable this functionality. The hardware architecture is typically a traditional three-level archive—storage is online, near line, or offline.

The data collector (typically the owner of the data) will nearly always have serious concerns about security of the data and its preservation. A capable asset-management system incorporates comprehensive multi-level security schemes, including control and monitoring of user privileges, workstation access, system administration, and product distribution. Access to the data should be password controlled; in addition, the system must be capable of supporting a hierarchy of users with more or less privileged access (see figure 2).

Figure 2. A complete security system provides protection in the archiving and output portions, as well as user password protection.

The output of the system should also be controlled—various products may be viewable by different sets of users. In other words, rules limiting export of files control distribution of products according to predetermined categories of recipients. In addition to the measures outlined above, the administrative controls include security logs that track and permanently record each and every transaction, file retrieval, or distribution.

Data security and preservation are different sides of the same coin. Geospatial data is expensive and older data is irreplaceable. G-DAM systems typically allow the user to indefinitely preserve digital assets, even if they are seldom or no longer used. Modern, sophisticated systems automatically scan the archive periodically to determine the condition of the data. Action to preserve data or change the modality on which it is stored can be taken on the basis of several factors.

Typical G-DAM systems are what is sometimes referred to as custom COTS (commercial off the shelf) to allow for customer needs. The set of formats supported by a G-DAM system may differ depending on where it is installed, for example. It is likely that an installation at a major international ground station will need to support a more complex set of formats than one installed at a small mapping company.

Automated immediate backup is just one element in the approach to asset preservation. In addition, a G-DAM system carefully monitors both the age and utilization of all media, automating the performance of tape maintenance, for instance, or dictating the transfer to fresh media to ensure quality preservation. Coupled with the ability to accept future storage technologies and formats, this results in an unlimited life span for the system and its content.

The developer, unlike the user, is involved in generating new and saleable products from the archive. Developers present requirements to the G-DAM that are similar to those of the user, but that also include the ability to interface to and integrate a variety of tools for image processing and manipulation and for GIS use. The developer needs easy and extensive access to tools while the user and data owner do not.

Many different image processing and GIS tools exist. Software for applications as diverse as imagery analysis, demographic analysis, route planning, report writing, or file editing and repurposing should be integrated and accessible through the user interface. The G-DAM system should provide an easy interface to currently available tools while remaining adaptable to other tools as they are developed. A related requirement is that the system be capable of ingesting a wide range of imagery formats.

Automated workflow, to ensure efficient use of the system, is another concern of the developer. A G-DAM system should be capable of a high degree of automation in the installation's workflow. The developer can then customize the automation of distribution and the assignment of tasks to speed data and tasks to the appropriate desktops. The automation of the distribution, assignment, and approval processes permits tracking of work in progress and enables ongoing prioritization adjustments among different projects or customers. Depending on the current and forecast workflow requirements of an organization, this type of automation can yield significant returns in efficiencies.

G-DAM development has been rapid. Today's systems are capable of capturing live satellite imagery at a rate of up to terabytes a day, eventually storing millions of images for easy and rapid retrieval.

Current systems are highly automated but also rely on user interaction. Systems are scalable and modular, allowing for growth of an installation. Scalability is an essential attribute, as typical geospatial enterprises will have rapidly growing storage requirements; a satellite receiving station may receive dozens of multi-megabyte images per day, for example. Modern G-DAMs are capable of ingesting a wide variety of formats, including motion imagery (MPEG) and various still image formats (JPEG and TIFF). They are designed with customizable browser-based graphical user interfaces and modular architectures. DAMs can manage more than 10,000 TB of digital assets in any combination of online, near-line, and offline storage.

In a typical geospatial installation, data ingest and archive functions are integrated with imagery analysis software on dozens of workstations. Imagery analysts using typical modern G-DAMs can call up a variety of applications, including GIS, from the same interface used to search and retrieve imagery. New software programs can be readily added to the suite of applications.

Today's image-based geospatial data management systems address crucial user issues such as efficient data ingest and retrieval, cataloging, data security, data integrity, intellectual property protection, and others. Systems incorporate extensive retrieval strategies, including geographic search techniques. Open architecture ensures easy integration of multi-functional software (e.g., third party viewers and imagery manipulation software) to meet the needs of the individual user. oe