Defining analytics through eyes of students

No matter how we define “analytics,” for EMBA students, often simple “analytics” adds value and this defines “success.”

By Peter Bell

Beyond highly quantitative: The case for a broader definition of “analytics.” Image © Kheng Guan Toh | 123rf.com

Beyond highly quantitative: The case for a broader definition of “analytics.” Image © Kheng Guan Toh | 123rf.com

As INFORMS has moved further into “analytics,” considerable interest has surfaced in an attempt to define “analytics.” Most of the discussion within INFORMS has taken the view that “analytics” has to be highly quantitative, but I hope that INFORMS will take a broader view. Here’s why.

I’ve been teaching a core analytics course in Ivey’s Executive MBA (EMBA) program for many years, and part of this course has always been a project done in the student’s workplace where the challenge has been to “do some analytics” that has a positive impact, either financially or organizationally. A key deliverable for this project is a letter from management assessing the impact of the work and its value to the organization. I estimate that I have read and graded almost 1,000 projects over the years.

These are executive student projects (teams of two or three) so they rarely involve any analytical heavy lifting, but the claimed impact has been impressive with about 5 percent of the projects reporting gains in excess of $5 million, and 20 percent reporting gains in excess of $1 million.

Just after I was invited to write an article for this back-to-school issue of OR/MS Today, the following e-mail arrived from a former student:
“I just wanted to follow up on our project from last term. Based on the partial changes we implemented due to the project analytics, we have seen an increase of over $150K, within 30 days of implementation. We took a phased-in approach and it’s been successful with negligible complaints or concerns from our users. We are on track to see an additional revenue lift of approximately $1.5M in the next 12 months if all things remain steady.

“Although it may not have met the exact description of an analytics project, we were aware of the risks and the ‘stickiness’ of our customer base. Demand remains consistent, and is in fact growing with new users.”
The second paragraph is a response to my grade report where I expressed doubt as to whether the project met the course requirement to include some “analytics.” These students were planning a major change in this company’s pricing strategy, but there was no attempt at data collection or “demand modeling,” so I expressed the concern that if they started moving prices around, sales might decline so they would need to monitor demand/sales closely.

Many EMBA projects (such as the one above) over the years have applied simple heuristics derived from analytics to real-world situations without doing any math or statistics, and they have claimed a substantial revenue lift or cost reduction. These student projects provide interesting, and I think valuable, lessons about “analytics.”

Competing With Analytics

If I cover a topic in my “Competing with Analytics” course, then in the mind of the student that becomes “analytics.” For example, in the classroom we cover several cases where an analytical pricing approach proves useful, and I use these examples to emphasize the basic approach to pricing from analytics, which is to segment the market and then price each segment separately so as to maximize revenue (or contribution) while meeting an overall sales objective or constraint. The models that we build in class also happen to illustrate the potential revenue enhancing value of a basic pricing heuristic “when demand is high price high, and when demand is low price low.”

We also discuss a supply chain/pricing case that illustrates the value of pricing decisions in helping out supply chain issues. In this case, the firm greatly benefits from raising the price on products that use bottleneck production processes and reducing the price on products that do not. Again, this leads to a pricing heuristic along the lines that if a product is difficult to schedule, price it high; but if it’s easy to schedule, price it low. The cases we cover all include demand data and lend themselves to some statistical analysis and construction of demand models, and can also be set up as optimizations to find optimal revenue maximizing prices. These are executive students, however, and many find the demand modeling and optimization quite challenging.

When students go back to their organizations to do the course project, they remember the general heuristics, and they apply these without doing any data collection, demand modeling and price optimization. I have seen this heuristic pricing approach applied to e-tailing, ready-mixed concrete, long distance transportation, banking, medical services, professional services, graphic arts, manufactured products and many other situations. In all these examples, the claimed revenue gains (supported by “management”) were significant and in some cases spectacular.

Reasons for Project Success

There are, of course, many possible explanations for the apparent success of these projects. Perhaps the claimed gains are a mirage? I doubt this to be true in all cases since once these managers have proved that gains are possible, they have often hired analytics people to push these ideas forward. I also receive many e-mails from former students long after the course is over updating me on how “pricing analytics” has transformed their organizations. In some cases I have checked organizational websites and seen market segmentation with variable prices by segment live and in color long after the grades were in.

A second explanation might be that the “before” situation was so ugly that spending a bit of time studying the issue and imposing a slightly less ugly solution produced the observed benefits. If this is the case, then surely this fits one of the claimed benefits of our analytics approach; that analyzing and systematically laying out a problem situation will improve the understanding of a complex issue and will enable the situation to be better managed with the associated benefits.

The explanation that I offer is that sometimes the body of research in analytics can be reduced to a set of high-level heuristics, and that applying these heuristics in a sensible and carefully controlled way will capture a high percentage of the gains available from using analytics.

As an example, if you are selling a service and you identify high- and low-demand market segments (either by time or by customer), and then price the high-demand segments high and the low-demand segments low and jiggle prices around fairly sensibly so as to meet sales targets, you will capture a very high percentage of the potential revenue gains. If you push this further by collecting sales/demand data, building demand models, determining “optimal prices” and installing software to calculate/implement/manage the revenue analytics, you will capture additional revenues, but perhaps not as much as you might expect. If this is the case, the majority of the benefit from the transformation of the firm to an analytics-driven firm comes from the adaptation of an analytics-driven thinking about prices, not from the details of the analytics.

Quick and Dirty

This is not a new idea. Gene Woolsey’s book (“Operations Research for Immediate Application: A Quick and Dirty Manual” [1]) advocated and demonstrated that often very simple models applied quickly could produce fast savings and be a hit with management. One difference today is that we have been modeling for some time, and we know that if we model this particular situation the results will generally look like this, so we can implement the result without re-doing the model.

Of course, it’s difficult to charge a million dollars for a piece of paper that has on it “price high when demand is high, price low when demand is low,” although it might be possible to charge that amount for an extensive data collection/modeling effort that produces this same basic advice. The interesting issue is how much value does the advanced analytics add, over and above a heuristic application of the basic concepts?

Student models that perform optimizations in Excel reinforce the idea that the value of analytics often lies in the approach and not the details. When I look at students’ Excel sheets I often find a veritable rats nest of =IF(..), =MAX(..), and /or =VLOOKUP(..) functions inside the optimization model and so the students are trying to optimize potentially highly non-linear problems. Apparently students fiddle around performing multiple runs until they come up with a solution they like, and then they implement the solution leading to a claim of significant benefits. If these students had the advanced analytical skills to properly formulate and optimize their models, I wonder how much the benefits would have increased.

In a similar vein, the project that has claimed the highest benefit (perhaps a one-time saving of $100 million) involved collecting the data necessary to carefully cost out three activities within a major North American company and then optimizing the distribution of workload among these activities. The “advanced analytics” content of this work was a three-variable Excel solver model. The huge benefit of this work clearly came from the analytical approach to issue identification, data collection, careful basic data analysis and costing and the effective implementation of the findings as a new North American “strategy.”

These examples strongly suggest that much of the benefit of analytics arises from the analytical problem-solving approach, and while the “advanced analytics” is the cherry on the top, in some (perhaps many) situations, it might be quite a small cherry.

Often we appear to be focusing on the development and application of advanced theory or algorithms to try to get a few points closer to the true optimum solution when we know that the data we are using is rough and probably out-of-date, so the result is a really good solution to an approximate problem. My data suggests that sometimes we might do better by focusing on cleaning up the data, improving our understanding of the real issue, and implementing much faster heuristics to find a decent answer to a problem that is closer to reality.

We in analytics tend to think that the “answer” we derive to an issue is the end-point, but for management it is often a starting point. The “answer” is usually delivered to a highly intelligent manager (or team) who has a thorough understanding of the business and the issue, and who then merges our analytics work with personal experience and a variety of opinion into planning a path forward. After choosing a course of action, managers implement change, monitor the situation carefully and make corrections. It is common in business strategy to say that the success of a chosen strategy is “all in the implementation.” The same can be said of analytics; successful analytics can be simple models implemented very effectively.

A quote I use every term to introduce students to analytics comes from Daniel Elwing, former president and CEO of ABB Electric, who said [2]: “[Analytics] is not a project or a set of techniques; it is a process, a way of thinking and managing.”

Along with “a way of thinking” and a “process” that starts with data collection, analytics adds value to the data through modeling, which in turns adds value to the decision-maker. Often, very simple models produce substantial benefits.

I encourage INFORMS to seize this broad view of analytics going forward.

Peter C. Bell (pbell@ivey.ca) is a professor of management science at the Ivey School of Business at Western University in Ontario, Canada. He is a past recipient of the INFORMS Prize for the Teaching of OR/MS Practice, served as chair of the 2013 and 2014 INFORMS Franz Edelman Prize Competition and as 2014 and 2015 chair of CPMS: The Practice Section of INFORMS. He is a frequent invited contributor to OR/MS Today, particularly the annual special issue on “Innovative Education.”

References

  1. G. Woolsey and H. S. Swanson, 1969, 1975, “Operations Research for Immediate Application: A Quick and Dirty Manual,” Harper and Row.
  2. D.H. Gensch, N.A. Aversa and S.P. Moore, 1990, “A Choice-Modeling Market Information System that Enabled ABB Electric to Expand its Market Share,” Interfaces, Vol. 20, No. 1 (January/February).

Save