Real-time statistical clustering for event trace reduction

Event tracing provides the detailed data needed to understand the dynamics of interactions among application resource demands and system responses. However, capturing the large volume of dynamic performance data inherent in detailed tracing can perturb program execution and stress secondary storage systems. Moreover, it can overwhelm a user or performance analyst with potentially irrelevant data. Using the Pablo performance environment's support for real-time data analysis, we show that dynamic statistical data clustering can dramatically reduce the volume of captured performance data by identifying and recording event traces only from representative processors. In turn, this makes possible low overhead, interactive visualization, and performance tuning.