Higher-order Markov Graph based Bug Detection in Cloud-based Deployments

Qing Cao, Haoran Niu

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

2 Scopus citations

Abstract

Detecting execution anomalies is an integral part of building and protecting modern large-scale distributed systems. These systems generate a large volume of system logs to record system state and significant events, which provide a valuable resource to help debug system failures and perform root cause analysis. However, detecting anomalies in log sequences remains a challenge due to reasons including the imbalance of the data, the complexity of relationships between events, and the high dimensionality of log events. Traditional graph-based models may lose important higher-order sequence patterns and result in undetectable higher-order anomalies because they use first-order or fixed-order networks to represent the underlying log data. In this paper, we propose a novel unsupervised graph-based anomaly detection method, called GraphLog, which utilizes a variable high-order network representation. This variable representation enables GraphLog to efficiently learn log patterns from normal logs and detect first-order and higher-order log patterns that deviate from normal data. We demonstrate that the proposed graph-based log anomaly detection algorithm is effective, and it outperforms other baseline methods when trained using two real-world datasets.

Original languageEnglish
Title of host publication2022 IEEE International Performance, Computing, and Communications Conference, IPCCC 2022
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages153-160
Number of pages8
ISBN (Electronic)9781665480192
DOIs
StatePublished - 2022
Externally publishedYes
Event2022 IEEE International Performance, Computing, and Communications Conference, IPCCC 2022 - Austin, United States
Duration: Nov 11 2022Nov 13 2022

Publication series

NameConference Proceedings of the IEEE International Performance, Computing, and Communications Conference
Volume2022-November
ISSN (Print)1097-2641

Conference

Conference2022 IEEE International Performance, Computing, and Communications Conference, IPCCC 2022
Country/TerritoryUnited States
CityAustin
Period11/11/2211/13/22

Fingerprint

Dive into the research topics of 'Higher-order Markov Graph based Bug Detection in Cloud-based Deployments'. Together they form a unique fingerprint.

Cite this