TutORial: Optimization of Sequential Decision Making for Chronic Diseases: From Data to Decisions

By Brian T. Denton

Rapid advances in medical interventions for chronic diseases such as cardiovascular disease, cancer, and diabetes have made it possible to detect diseases at early stages and tailor treatment pathways to individual patients based on their risk factors including gender, race/ethnicity, and disease-specific factors. However, the large number of relevant risk factors to be considered, combined with uncertainty in future health outcomes and treatment side effects, makes optimizing these decisions challenging. Randomized trials are the gold standard for selecting treatment interventions but the large number of possible decisions and their high cost makes these trials infeasible. Data-driven operations research methods are showing great promise in helping patients and medical doctors improve decisions about health interventions. Observational data that is now routinely collected in many health systems is a valuable resource for fitting and validating stochastic models for chronic diseases. Moreover, optimization methods for sequential decision making, including Markov decision processes, partially observable Markov decision processes, and reinforcement learning methods, exploit these models to optimize treatment policies that can balance competing criteria such as the harms and benefits associated with treatment of chronic diseases. This tutorial provides an introduction to some of the most commonly used approaches for using raw data for the purpose of optimizing sequential treatment policies. Special attention is paid to the challenges associated with using observational data and the influence of model parameter uncertainty in this context.