Multiscale information theory for complex systems: Theory and applications

The science of complex systems requires a general way to represent and understand structure. We review a mathematical formalism that uses information theory to make precise the intuition that a complex system exhibits structure at multiple scales. We show that structure can be seen as the totality of relationships among a system’s components, and information theory can quantify these relationships. Beginning with fundamental axioms that specify the properties that a function must satisfy in order to be an information measure, we develop quantitative indices that summarize system structure, namely the complexity profile (CP) and the marginal utility of information (MUI). We demonstrate the applicability of our formalism with examples from evolutionary biology, economics and finite geometry.