Share Email Print

Proceedings Paper

Leveraging human oversight and intervention in large-scale parallel processing of open-source data
Author(s): Enrico Casini; Niranjan Suri; Jeffrey M. Bradshaw
Format Member Price Non-Member Price
PDF $17.00 $21.00

Paper Abstract

The popularity of cloud computing along with the increased availability of cheap storage have led to the necessity of elaboration and transformation of large volumes of open-source data, all in parallel. One way to handle such extensive volumes of information properly is to take advantage of distributed computing frameworks like Map-Reduce. Unfortunately, an entirely automated approach that excludes human intervention is often unpredictable and error prone. Highly accurate data processing and decision-making can be achieved by supporting an automatic process through human collaboration, in a variety of environments such as warfare, cyber security and threat monitoring. Although this mutual participation seems easily exploitable, human-machine collaboration in the field of data analysis presents several challenges. First, due to the asynchronous nature of human intervention, it is necessary to verify that once a correction is made, all the necessary reprocessing is done in chain. Second, it is often needed to minimize the amount of reprocessing in order to optimize the usage of resources due to limited availability. In order to improve on these strict requirements, this paper introduces improvements to an innovative approach for human-machine collaboration in the processing of large amounts of open-source data in parallel.

Paper Details

Date Published: 27 May 2015
PDF: 6 pages
Proc. SPIE 9499, Next-Generation Analyst III, 94990K (27 May 2015); doi: 10.1117/12.2177264
Show Author Affiliations
Enrico Casini, Florida Institute for Human and Machine Cognition (United States)
Niranjan Suri, Florida Institute for Human and Machine Cognition (United States)
U.S. Army Research Lab. (United States)
Jeffrey M. Bradshaw, Florida Institute for Human and Machine Cognition (United States)

Published in SPIE Proceedings Vol. 9499:
Next-Generation Analyst III
Barbara D. Broome; Timothy P. Hanratty; David L. Hall; James Llinas, Editor(s)

© SPIE. Terms of Use
Back to Top
Sign in to read the full article
Create a free SPIE account to get access to
premium articles and original research
Forgot your username?