TY - GEN
T1 - Incremental anomaly detection approach for characterizing unusual profiles
AU - Fang, Yi
AU - Omitaomu, Olufemi A.
AU - Ganguly, Auroop R.
PY - 2010
Y1 - 2010
N2 - The detection of unusual profiles or anomalous behavioral characteristics from sensor data is especially complicated in security applications where the threat indicators may or may not be known in advance. Predictive modeling of massive volumes of historical data can yield insights on usual or baseline profiles, which in turn can be utilized to isolate unusual profiles when new data are observed in real-time. Thus, an incremental anomaly detection approach is proposed. This is a two-stage approach in which the first stage processes the available historical data and develops statistics that are in turn used by the second stage in characterizing the new incoming data for real-time decisions. The first stage adopts a mixture model of probabilistic principal component analyzers to quantify each historical observation by probabilistic measures. The second stage is a chi-square based anomaly detection approach that utilizes the probabilistic measures obtained in the first stage to determine if the incoming data is an anomaly. The proposed anomaly detection approach performs satisfactorily on simulated and benchmark datasets. The approach is also illustrated in the context of detecting commercial trucks that may pose safety and security risk. It is able to consistently identified trucks with anomalous features in the scenarios investigated.
AB - The detection of unusual profiles or anomalous behavioral characteristics from sensor data is especially complicated in security applications where the threat indicators may or may not be known in advance. Predictive modeling of massive volumes of historical data can yield insights on usual or baseline profiles, which in turn can be utilized to isolate unusual profiles when new data are observed in real-time. Thus, an incremental anomaly detection approach is proposed. This is a two-stage approach in which the first stage processes the available historical data and develops statistics that are in turn used by the second stage in characterizing the new incoming data for real-time decisions. The first stage adopts a mixture model of probabilistic principal component analyzers to quantify each historical observation by probabilistic measures. The second stage is a chi-square based anomaly detection approach that utilizes the probabilistic measures obtained in the first stage to determine if the incoming data is an anomaly. The proposed anomaly detection approach performs satisfactorily on simulated and benchmark datasets. The approach is also illustrated in the context of detecting commercial trucks that may pose safety and security risk. It is able to consistently identified trucks with anomalous features in the scenarios investigated.
KW - PPCA
KW - Transportation security
KW - chi-square statistics
KW - incremental knowledge discovery
KW - radioactive materials
UR - http://www.scopus.com/inward/record.url?scp=77957923838&partnerID=8YFLogxK
U2 - 10.1007/978-3-642-12519-5_11
DO - 10.1007/978-3-642-12519-5_11
M3 - Conference contribution
AN - SCOPUS:77957923838
SN - 3642125182
SN - 9783642125188
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 190
EP - 202
BT - Knowledge Discovery from Sensor Data - Second International Workshop, Sensor-KDD 2008, Revised Selected Papers
T2 - 2nd International Workshop on Knowledge Discovery from Sensor Data, Sensor-KDD 2008
Y2 - 24 August 2008 through 27 August 2008
ER -