Metastable Markov chains

Abstract

In this paper, we discuss the dynamics of metastable systems. Such systems exhibit interesting long-living behaviors from which they are guaranteed to inevitably escape (e.g., eventually arriving at a distinct failure or success state). At the heart of this work, we emphasize (1) that for our goals, hybrid systems can be approximated as Markov Decision Processes, (2) that although corresponding Markov chains may include a very large number of discrete states, much of their dynamic behavior is well-characterized simply by the secondlargest eigenvalue, which is directly analogous to a dominant pole for a discrete-time system and describes both the mean and higher-order modes of the escape statistics, and (3) that for many systems, one can accurately describe initial conditions as being rapidly forgotten, due to a significant separation in slow and fast decay rates. We present both theory and intuitive toy examples that illustrate our approach in analyzing such systems, toward enabling and encouraging other researchers to adopt similar methods.