Teaching O.R. at the NPS

Learning by solving real problems

At the Naval Postgraduate School, students learn O.R. by doing O.R. that helps enhance the security of the United States and its allies.

By Thomas W. Lucas

Editor’s note:
For more than a decade, OR/MS Today has invited the most recent recipient of the INFORMS Prize for the Teaching of OR/MS Practice to contribute an article to the magazine’s annual special issue on innovative education. We ask for a brief description of the award-winning O.R. program and the reasons for its success, as well as an outline of the recipient’s educational background, teaching philosophy, mentors and advice for their fellow O.R. educators. Following is the story of Naval Postgraduate School Professor Thomas W. Lucas, the 2015 teaching prize recipient.

It is a great honor to receive the 2015 INFORMS Prize for the Teaching of OR/MS Practice. My journey to this point has followed an unusual path, and there are many people who made this possible. Most prominent among them are my parents, who were math teachers who loved learning and sharing knowledge – and fostered a similar state of mind in their children. After earning a B.S. in operations research and industrial engineering, I took a job as a systems engineer at Hughes Aircraft Company. I spent 11 years at Hughes working on a variety of Navy problems, such as bottom-contour navigation, target tracking, data fusion and weapon employment. The job was rewarding and interesting, and inspired me to learn more. So, while at Hughes, I continued my education (M.S. and Ph.D.) by participating in Hughes’ graduate education fellowship program.

Subsequently, I spent six years doing public policy research as a project leader and member of the statistics group at RAND. During this time, John Rolph invited me to teach classes at the University of Southern California’s Marshall School of Business – it was an eye-opening experience that strengthened my desire to be in the classroom.

Rick Rosenthal, the 2000 winner of this award, recruited me to the Operations Research (O.R.) Department at the Naval Postgraduate School (NPS) nearly two decades ago. Since I had a background and interest in national defense, and a love of teaching at the university level, I jumped on this unique opportunity. Rick was a great mentor and role model. More importantly, he charged me with doing my utmost for our students. While at NPS, I’ve had a long and fruitful collaboration with Susan Sanchez. We share research interests that are grounded in practical applications. We have partnered in building a center that provides tremendous resources and opportunities for our students.

Professor Tom Lucas at NPS received the 2015 INFORMS Prize for the Teaching of OR/MS Practice.

Professor Tom Lucas at NPS received the 2015 INFORMS Prize for the Teaching of OR/MS Practice.

It is easy to get motivated to teach our students. As officers, they show up to class on time, call you sir or ma’am, and really try to do all the homework – well, most of the time. Moreover, since what they are studying directly affects their profession, the students are driven. Sadly, many of us have lost former students operating in the service of their country. We hope that what our students learn at NPS saves lives, and we strive to make it so.

NPS students are mostly mid-level military officers, and while we are a U.S. Navy graduate university, we have students from all the services and more than 40 countries. Much is expected from our faculty, as scores of our graduates have grown into senior military leaders – including former O.R. graduates Adm. Mike Mullen (recent Chairman of the Joint Chiefs of Staff), P.C. Lui (longtime Singapore chief defense scientist) and James Roche (former Secretary of the Air Force). Because of our graduates’ success, new NPS professors are advised to do right by their students, since one day they will almost certainly be working for some of them!

NPS students have jobs waiting for them upon graduation. Graduates of the O.R. Department typically go on to help their service and country determine how best to equip, staff, organize, employ, train, supply, and maintain their forces. The United States Department of Defense (DoD) is more than a half-trillion-dollar-a-year enterprise – making several tens of billions of dollars in new investments per annum. When you couple the breadth of decisions that must be made with the inherent uncertainty involved, there is no shortage of topics with potentially great impact, which is good since every student needs to complete a master’s thesis.

The SEED Center for Data Farming

The SEED Center for Data Farming (http://harvest.nps.edu) was created (with Susan Sanchez) to involve and support our graduate students in applied and theoretical research projects, primarily sponsored by the DoD. SEED is an acronym for “simulation experiments and efficient design.” Data farming refers to using high-performance computing to “grow” data. We contrast data farming with data mining as follows. The data miner seeks valuable nuggets of information from a large volume of data. Similarly to how a geographical miner has minimal control over the geology being mined, a data miner typically has less than full control over the raw data he or she analyzes (e.g., Twitter feeds, credit-card swipes, satellite reports, etc.). Conversely, data farmers choose the inputs, outputs and number of computational experiments to conduct. Thus, like agricultural farmers, what they reap depends upon what they sow and cultivate. The enabling technologies for data farming are high-performance computing, new designs of experiments developed specifically for computational models, and high-dimensional data analysis and visualization techniques [1]. The Center’s mission is to imbed students in projects that make modeling and simulation more effective for decision-makers – with an ultimate goal of informing decisions at all levels to ensure that sailors, soldiers, Marines and airmen have what they need to accomplish their missions safely and efficiently.

Nearly 170 students, from all the services and more than a dozen countries, have completed their thesis or dissertation research in the SEED Center. Our students’ theses tend to have a real-world problem underpinning them – ideally with people outside NPS awaiting the results. Researchers and students in the Center are developing new methods and routinely doing computational experiments using designs that were not available only a decade ago – sometimes simultaneously varying hundreds of variables of different types and levels. SEED Center students have won more than 20 major thesis awards. Their success follows from the breadth of support that the Center provides, including the SEED Lab (a work area with desks, computers, printer and a smart screen that facilitates collaboration among students doing similar research), laptops, a high-performance computing cluster, research assistants, weekly seminars and international workshops. The more than a dozen international data-farming workshops that the SEED Center has hosted or participated in have allowed more than 60 students to work in teams of subject matter experts, modelers and analysts from around the world in week-long, model-based studies. We are able to provide this support because our sponsors recognize the value received on their investment.

Problem-Driven Research and Sponsored Applications

The U.S. Navy has long understood the value of operations research. Indeed, military applications, such as finding submarines and protecting supply convoys, led scientists to formalize their efforts into the science now known as operations research [2]. Our research in the SEED Center is also problem-driven. Student applications have included support to ongoing operations (such as in Afghanistan), systems analysis (e.g., trade-off studies), humanitarian assistance and disaster relief, critical infrastructure protection, tactics development, manpower planning, organizational behavior, logistics and more. What most of these have in common is that there are a large number of potentially important variables that may interact in nonlinear ways, multiple outcomes of interest, and lots of uncertainty. Consequently, stochastic simulation is our most-used O.R. analysis technique.

Thomas W. Lucas

Simulation enables our students to build models that are better representations of the complicated systems or phenomena being studied than is typically feasible using analytically tractable models derived by invoking classic academic assumptions such as linearity, deterministic relationships, declaring independence, normality, heteroscedasticity, memorylessness, etc. Hence, their insights are usually of enhanced practical value. The simulations we use tend to be of high dimension with significant a priori uncertainty. Fortunately, the science of exploring these models is rapidly advancing, with our students making important contributions [3].

SEED Center students have spent the last dozen years collaborating with faculty to build a portfolio of designs that provide computational researchers with design and analysis flexibility. Design flexibility allows us to readily generate designs for a broad set of input factors (numbers and types) and sampling budgets. Analysis flexibility provides the ability to fit many diverse meta-models to multiple outputs and generate a wide variety of visual relationships. Our most-used design is the family of space-filling nearly orthogonal Latin hypercubes (NOLHs) developed by Lt. Col. Tom Cioppa [4]. Since extensive computation is used to generate these NOLHs, a catalogue of ready-to-use designs was made available in an online spreadsheet – this has become our killer app, and is used by scientists around the world. A subsequent student, Col. Andy Hernandez, used mixed integer programming to increase greatly the dimensions in which NOLHs are available [5]. Brazilian Lt. Col. Helcio Vieira extended Hernandez’s approach to develop an algorithm that makes designs that are nearly orthogonal and nearly balanced for mixes of categorical, discrete (with a mix of multiple levels) and continuous factors [6]. Lt. Col. Alex MacCalman used evolutionary algorithms to build NOLHs for a complete second-order model [7]. Each dissertation built on the previous ones, and all of them were driven by needs arising from practical applications.

The above algorithms were developed to help researchers extract more information from their computational experiments. They have been used in more than a hundred diverse applications by students at NPS, usually as part of their master’s theses. Brief recaps of a few exemplary ones follow:

  • The U.S. Army Training and Doctrine Command Analysis Center used simulation to develop an investment strategy regarding unmanned aerial vehicles (UAVs). One such model was ASC-U (short for assignment scheduling capability for UAVs). ASC-U develops a planning schedule that takes into account hundreds of UAVs, of multiple types, trying to fulfill thousands of missions over a 15-day period. As part of a broader effort, we used NOLHs to determine that fewer classes of UAVs were needed. The analysis was credited with helping save billions of dollars [8].
  • A series of students did award-winning theses under the rubric of NPS’s Project Jason investigating how Navy ships can best defend against swarming UAVs and other emerging threats. The robust tactics discovered through millions of simulated engagements, across a breadth of scenarios, have already been transferred to the fleet in the form of tactical bulletins. This research is classified so the very interesting details can’t be shared.
  • The Army’s enlisted specialty (ES) model informs the management of more than 400,000 enlisted soldiers with a seven-year planning horizon. The ES model we explored took into account 859,633 variables and calculated projections against 224,473 constraints. We found that, by adjusting just eight of the 52 fixed coefficients in an optimization portion of the model, the number of misaligned soldiers is reduced by roughly two combat brigades (8,400 soldiers) [9].

Be Passionate and Other Pedagogical Thoughts

My top advice to teachers is simple: excite and energize your students. Learning is a lifelong endeavor, and most learning takes place outside of a classroom. The easiest way to excite your students is with your own enthusiasm, energy and passion. Indeed, you got where you are because you love your subject. Share that passion with your students and help nurture it in them.

Some students who do not see the practical value of what is being taught quickly lose interest. By focusing on solutions to real problems and using real data in your examples, the lectures can motivate as well as teach. Most methods were derived for a reason; tell the story associated with the method’s development and highlight cases where O.R. practitioners and scientists have made a profound difference. Moreover, since classes may contain students with a broad range of capabilities, I give out variable-resolution projects. That is, most all of the students can meet the minimum standard, but motivated students have natural ways in which to “add something extra” and are rewarded for doing so.

Learning how to learn may be the biggest challenge facing students. A real-analysis professor of mine, whose name I have long since forgotten, started each exam with 30 points based simply on writing out definitions and theorems. Well, that was easy, since all I needed to do was memorize the definitions and theorems, so I did. Lo and behold, I found that having a solid foundation in the definitions and theorems made the lectures much more meaningful and the homework considerably easier. To this day I start each weekly lab with a short vocabulary quiz. This professor probably has no idea how his simple technique affected me and the thousands of students I have had in class. Few other professions, if any, can do so much for so many.

Professor Tom Lucas, professor of operations research at the Naval Postgraduate School in Monterey, Calif., received the 2015 INFORMS Prize for the Teaching of OR/MS Practice. He is co-director of the SEED Center
(https://harvest.nps.edu/) at NPS.

References

  1. Lucas, T.W., W.D. Kelton, S.M. Sanchez, P.J. Sanchez, and B.L. Anderson, 2015, “Changing the paradigm: simulation, often the method of first resort,” Naval Research Logistics, Vol. 62, No. 4, pp. 293-303.
  2. Morse, P.M., and G.E. Kimball, 1951, “Methods of Operations Research,” Wiley.
  3. Kleijnen, J.P.C., S.M. Sanchez, T.W. Lucas, and T.M. Cioppa, 2005, “A user’s guide to the brave new world of designing simulation experiments,” INFORMS Journal on Computing, Vol. 17, No. 3, pp. 263-289.
  4. Cioppa, T.M., and T.W. Lucas, 2007, “Efficient nearly orthogonal and space-filling Latin hypercubes,” Technometrics, Vol. 49, No. 1, pp. 45-55.
  5. Hernandez, A.S., T.W. Lucas, and M. Carlyle, 2012, “Constructing nearly orthogonal Latin hypercubes for any nonsaturated run-variable combination,” ACM Transactions on Modeling and Computer Simulation, Vol. 22, No. 4, pp. 20:1-20:17.
  6. Vieira Jr., H., S.M. Sanchez, K.H. Kienitz, and M.C.N. Belderrain, 2013, “Efficient, nearly orthogonal-and-balanced, mixed designs: an effective way to conduct trade-off analyses via simulation,” Journal of Simulation, Vol. 7, pp. 264-275.
  7. MacCalman, A.D., H. Vieira, and T.W. Lucas, 2016, “Second order nearly orthogonal Latin hypercubes for exploring complex stochastic simulations,” Journal of Simulation.
  8. Sanchez, S.M., T.W. Lucas, P.J. Sanchez, C.J. Nannini, and H. Wong, 2012, “Designs for large-scale simulation experiments with applications to defense and homeland security,” “Design and Analysis of Experiments, volume III,” by Hinckleman (ed.), pp. 413-441,Wiley.
  9. Erdman, R.W., 2010, “Using experimental design and data analysis to study the enlisted specialty model for the U.S. Army G1,” MS in Operations Research, Naval Postgraduate School, Monterey, Calif.

Save