CellMigrationGym: a data-driven framework utilizing deep reinforcement learning to unravel mechanisms of cell migration

Cell migration is a fundamental phenomenon in biology that underlies normal development as well as cancer. Recently, a data-driven approach was introduced that uses deep reinforcement learning(DRL) and 3-D live images to study cell migration. This approach formulates the cell migration process as a sequential Markov decision process (MDP), so that hypotheses of the underlying mechanism of the observed migration can easily be incorporated as high-level regulatory rules and constraints for DRL. The application of the approach successfully uncovered a novel mechanism of cell migration in C. elegans embryogenesis that involves a modular organization of cells by using ubiquitous labels of cell nuclei and simple rules based on empirical statistics of the images. This success demonstrates new opportunities to use DRL to infer the biology of cell migration without prior knowledge. This paper presents an open framework, CellMigrationGym, to standardize the DRL approach to study cell migration. Built upon common packages (OpenAI Gym, PyBullet, and DRL libraries), CellMigrationGym provides powerful and flexible functions to investigate cell migration behavior. Through a case study, we demonstrate the critical functions of CellMigrationGym with technical details, such as 1) preparation and standardization of multiple observational data, 2) reward formulation and DRL model configuration appertaining to the hypotheses of migration mechanism (such as gradient-driven and collective cell behavior-driven mechanisms), 3) exploration of migration scenarios under hypothesized mechanisms, and 4) evaluation of neighboring cell’s influence on the cell migration.