Warren Powell
328 Christopher Drive
Princeton, New Jersey 08540 USA
Website: http://www.castlelab.princeton.edu/
Topics:
Sequential decision analytics
Sequential decision problems include any problem that consists of decision, information, decision, information...where each decision is evaluated using some performance metric. Decisions are made with a method called a “policy” and the goal is to find the policy that works best according to some objective function.
Sequential decision problems arise throughout every human process, including transportation and logistics, energy, health, finance, e-commerce, business (from supply chain management to marketing), as well as the laboratory sciences. They exhibit decisions that might be binary (stopping problems), discrete elements (what drug, material, process or product), and an array of sources of uncertainty.
Sequential decision problems have been studied in operations research under many names, including dynamic programming, stochastic programming, stochastic search, simulation optimization, stochastic control, as well as active learning or multiarmed bandit problems. Recently, considerable attention has been attracted to communities using names such as approximate dynamic programming or, more recently, reinforcement learning. There are 15 distinct fields that study these problems, using at least eight different notational systems.
I will demonstrate that a single modeling framework can be used to model any sequential decision problem, covering all 15 different fields. While there is a vast array of methods to make decisions, they can be organized into four classes of policies that span every method in the research literature, as well as any methods used in practice. These are universal.
I can illustrate these ideas using topics from transportation and logistics, or energy, or a sampling of different applications.
Background:
- BSE Princeton University
- MSE MIT
- Ph.D. MIT
Warren B. Powell is Professor Emeritus at Princeton University, where he taught for 39 years, and is currently the Chief Innovation Officer at Optimal Dynamics. He was the founder and director of CASTLE Lab, which focused on stochastic optimization with applications to freight transportation, energy systems, health, e-commerce, finance and the laboratory sciences, supported by over $50 million in funding from government and industry. He has pioneered a new universal framework that can be used to model any sequential decision problem, including the identification of four classes of policies that spans every possible method for making decisions. This is documented in his latest book with John Wiley: Reinforcement Learning and Stochastic Optimization: A unified framework for sequential decisions. He published over 250 papers, four books, and produced over 60 graduate students and post-docs. He is the 2021 recipient of the Robert Herman Lifetime Achievement Award from the Society for Transportation Science and Logistics, the 2022 Saul Gass Expository Writing Award. He is a fellow of Informs, and the recipient of numerous other awards.