Defending against Internet worms using a phase space method from chaos theory

Enterprise networks are facing ever-increasing security threats from Distributed Denial of Service (DDoS) attacks, worms, viruses, intrusions, Trojans, port scans, and network misuses, and thus effective monitoring approaches to quickly detect these activities are greatly needed. In this paper, we employ chaos theory and propose an interesting phase space method to detect Internet worms. An Internet worm is a self-propagating program that automatically replicates itself to vulnerable systems and spreads across the Internet. Most deployed worm-detection systems are signature-based. They look for specific byte sequences (called attack signatures) that are known to appear in the attack traffic. Conventionally, the signatures are manually identified by human experts through careful analysis of the byte sequence from captured attack traffic. We propose to embed the traffic sequence to a high-dimensional phase space using chaos theory. We have observed that the signature sequence of a specific worm will occupy specific regions in the phase space, which may be appropriately called the invariant subspace of the worm. The invariant subspace of the worm separates itself widely from the subspace of the normal traffic. This separation allows us to construct three simple metrics, each of which completely separates 100 normal traffic streams from 200 worm traffic streams, without training in the conventional sense. Therefore, the method is at least as accurate as any existing methods. More importantly, our method is much faster than existing methods, such as based on expectation maximization and hidden Markov models.