Share Email Print

Proceedings Paper

Application of data mining techniques to identify data anomalies: a case study in the oil and gas industry
Author(s): Jenifer S. McCormack; Brian Wohlschlaeger; Bryan Lanier
Format Member Price Non-Member Price
PDF $17.00 $21.00

Paper Abstract

This paper presents the application of the AgentMinerTM tool suite to improve the efficiency of detecting data anomalies in oil well log and production data sets, which have traditionally been done by hand or through the use of database business rules. There was a need to verify the data sets, once cleansed and certified to ensure that the existing data certification process was effective. There was also a need to identify more complex relational data anomalies that cannot be addressed by simple business rules. Analysis techniques including statistical clustering, correlation and 3-D data visualization techniques were successfully utilized to identify potential complex data anomalies. A data-preprocessing tool was also applied to automatically detect simple data errors such as missing, out of range, and null values. The pre-processing tools were also used to prepare the data sets for further statistical and visualization analyses. To enhance the discovery of data anomalies two different data visualization tools for the data clusters were applied.

Paper Details

Date Published: 12 March 2002
PDF: 10 pages
Proc. SPIE 4730, Data Mining and Knowledge Discovery: Theory, Tools, and Technology IV, (12 March 2002); doi: 10.1117/12.460241
Show Author Affiliations
Jenifer S. McCormack, Science Applications International Corp. (United States)
Brian Wohlschlaeger, Science Applications International Corp. (United States)
Bryan Lanier, Science Applications International Corp. (United States)

Published in SPIE Proceedings Vol. 4730:
Data Mining and Knowledge Discovery: Theory, Tools, and Technology IV
Belur V. Dasarathy, Editor(s)

© SPIE. Terms of Use
Back to Top
Sign in to read the full article
Create a free SPIE account to get access to
premium articles and original research
Forgot your username?