Chapter 10

Some Simplifying Conditions for Markov Chain Modeling

Marlin U. Thomas

Department of Operational Sciences, Air Force Institute of Technology, Wright-Patterson Air Force Base, Ohio45433, marlin.thomas@afit.edu

Abstract

Markov chains are commonly applied in operations research modeling due to the rational appeal of the inherent Markov dependence and related properties that make them particularly useful for applications as queuing, inventory, maintenance, and manpower planning decisions. Other properties of Markov chains provide computational convenience as well. Under certain lumpability conditions, the state space of a Markov chain can be partitioned into selected subsets of states to form a smaller chain that retains the Markov property, thus providing even greater simplicity in modeling operational systems. Conditions and application of these methods are discussed in this tutorial, including statistical inference for cases where data are available.\AQ{Please confirm/correct all edits to the sentence beginning ``Examples are provided... .'' Examples are provided throughout the discussions, including the application of lumpability to examine university retirement options and to study DNA testing. Some suggestions for further research in Markov chain lumpability are provided.

Key words: discrete time Markov chains; Markov chain lumpability; statistical inference of Markov chains; Markov manpower models; DNA testing

The 2012 volume of the TutORials in Operations Research series will be available to people who have registered for the 2012 INFORMS Annual Meeting. All INFORMS members will be able to access TutORials after January 1, 2013. Printed TutORials books from this and previous years can also be ordered here, along with CDs from 2005 to 2009.

Download Chapter 10 - Password-protected content.

For login instructions click here.

________________________________________________

Citation information:

Thomas, Marlin U. , Some Simplifying Conditions for Markov Chain Modeling. INFORMS TutORials in Operations Research, Vol. 9. INFORMS, Hanover, MD, pp. 184--203.

http://dx.doi.org/10.1287/educ.1120.0101