Reinforcement Learning for Elimination of Reentrant Spiral Waves in Excitable Media

Despite recent advancements in understanding the mechanisms underlying sudden cardiac death due to cardiac fibrillation, new defibrillation techniques have been slow to manifest. The reasons for this are manifold, but from a controls perspective, the spatiotemporal behavior exhibited by the electrical activity of the heart during fibrillation is high-dimensional, chaotic, and fundamentally nonlinear making standard control techniques difficult to implement. In this work, we investigate the use of a reinforcement learning framework to identify a control strategy to eliminate reentrant spiral waves that are associated with cardiac fibrillation. We propose a reduced order model that replicates the behavior of an idealized spiral wave core traveling in an excitable medium. We implement the Q-learning method with function approximation using a neural network to learn a control strategy that actively drives a spiral core to the boundary of the domain where it can be absorbed. Results indicate that the reinforcement learning algorithm is able to rapidly learn an effective control strategy for use in the reduced order model. Continued development of this framework for implementation in more realistic models could inform the design of active control strategies to achieve low-energy control of spatiotemporal chaos in the heart associated with cardiac arrest.