ISSUES IN EDUCATION

The value of open-ended questions, Part 1

By Will Millhiser

Consider two questions one might ask in an introductory management science class:

(a)

education

(b) Find two different linear programs with the same optimal solution x1 = 1, x2 = 6.
Which question is more intriguing? Which requires deeper thought and greater display of linear programming knowledge? Which uses more intuition and less “procedural knowledge,” that is, memorization of basic mechanics? Hopefully you said (b) each time. The questions are related; (b) is the solution to (a), but (a) is one of an infinite number of solutions to (b).

Now consider typical introductory OR/MS and operations management (OM) questions: identify the shadow prices for the two resources; find the project duration and critical activities; determine the reorder point and quantity; compute the average waiting time in the queue, and so on. What do these problems have in common? Each, like (a), requires a single correct answer.

Mathematics education scholars examined whether single-answer problems threaten students, prevent students from showing all they know or deny longer-lasting learning. As an alternative, open-ended questions have been shown to give benefits in classroom teaching, homework and quizzes/tests. What is an open-ended question?

“Open-ended questions are not multiple-choice questions without options. They are not questions that demand a single correct response. Nor are they questions where any response is acceptable. Rather, open-ended questions address the essential concepts, processes and skills that go beyond the specifics of instruction to define a subject area. In general, they require complex thinking and yield multiple solutions [1].”

In OR/MS and OM, open-ended problem solving has three broad areas of relevance: (i) furthering and demonstrating one’s understanding of the mathematical techniques, (ii) interpreting results and sensitivity analysis, and (iii) applying techniques in real world settings where data is often ambiguous and changing.

Why Open-Ended Questions?

There are four primary benefits.

Confidence. Open-ended questions help students build confidence by problem solving in ways that come naturally. Cooney et al. [2] suggest, “Students learn in a variety of ways, and the way they show their knowledge varies as well. The nature of open-ended items allows students to approach problem solving however they choose.” Confidence also comes from rising to the greater challenge of the open-ended question. For example, Cooney et al. add, “Solving a problem for which a solution is not immediately apparent can also give students confidence in their mathematical knowledge.”

Conversing in OR/MS. An open-ended question, like public speaking and journal writing, is especially helpful in developing the students’ ability to communicate in the language of OR/MS and OM and to communicate quantitative information (for more on these benefits, see [3]).

Better assessment. Proponents of open-ended activities say, “Such questions can communicate levels of student achievement more clearly than multiple-choice items and give better guidance for instruction” [1]. I can certainly attest to this; reading student responses to open-ended exam questions tells me more about the students’ reasoning ability, the students’ level of achievement and the quality of my instruction.

Deeper learning. The most important benefit is promoting profound learning. Some argue that such questions promote an intuition called conceptual knowledge that, unlike procedural knowledge, is more readily applied in unfamiliar contexts outside the classroom (e.g., see [4], [5] and [6]). But what level of challenge fosters the most profound learning? Finkel [7], Boss [8] and Bain [9] say make students struggle! Finkel specifically says open-ended questions should be more like parables. In disciplines such as ours, Finkel says a parable is a “puzzle, paradox or perplexing problem” that is tangible and engaging, like a short story, and stimulating so that students are intrigued about the answer.

Other authors agree that open-ended problems should be paradoxical, though some take a more extreme view; occasionally, a problem should be impossible. Why? Nineteenth-Century philosopher John Stuart Mill said it best: “A pupil from whom nothing is ever demanded that he cannot do, never does all he can” [10].

Challenges

Even when open-ended exercises are appropriate, they sometimes go wrong. Here’s why.

Misleading the students. The first is a danger that if not constructed and administered properly, open-ended problems mislead. Wu [11] documented three examples that are so difficult that the intended audience is incapable of understanding the solutions or erroneously led to believe no solutions exist because the problems require advanced skills. Wu further worries that a student might infer from open-ended problem solving that conjectures and experiments are viable substitutes for analytical reasoning and proof. To avoid these risks, Wu suggests carefully narrowing the scope of problems in ways that do not diminish richness. Perhaps the distinction between conjecture and logical reasoning is not as important in an applied fields such as OR and OM, but Wu’s concern is worth noting.

Overwhelming the students. Aside from misinformation, open-ended questions can overwhelm students on two levels: the time required to formulate responses in time-sensitive settings and the obscurity of certain questions. Ask me about the time none of 114 students came close to finishing my mid-term exam in the allotted 75 minutes.

Overwhelming the professor. Let’s not forget the grading burden. More on this later.

To be continued… The next “Issues in Education” will put these ideas into practice with a dozen ways to make any question open-ended and resources for instructors.

Will Millhiser (william.millhiser@baruch.cuny.edu) is an assistant professor in the Operations Management Group at Baruch College, The City University of New York. In an earlier life he taught high school mathematics and studied mathematics teaching methods.

References

  1. Badger, E., B. Thomas, 1992, “Open-ended questions in reading,” Practical Assessment, Res. and Evaluation, Vol. 3, No. 4, pareonline.net.
  2. Cooney, T.J., W.B. Sanchez, K. Leatham, D.S. Mewborn, 2004, “Open-Ended Assessment in Math: A Searchable Collection of 450+ Questions,” Heinemann. Quotations taken from online version, books.heinemann.com/math.
  3. Cooney, T.J., 1996, “Thinking About Being a Mathematics Teacher,” in T.J. Cooney, S.I. Brown, J.A. Dossey, G. Schrage, E.C. Wittmann (eds.) “Mathematics, Pedagogy and Secondary Teacher Education,” Heinemann, pp. 1-26.
  4. Hiebert, J., 1986, “Conceptual and Procedural Knowledge: The Case of Mathematics,” in J. Hiebert (ed.), “Conceptual and procedural knowledge in mathematics: An introductory analysis,” Routledge.
  5. Schoenfeld, A.H., 1988, “When good teaching leads to bad results: The disasters of ‘well-taught’ mathematics courses,” Educational Psychologist, Vol. 23, No. 2, pp. 145-166.
  6. Boaler, J., 1998, “Open and Closed Mathematics: Student Experiences and Understandings,” J. for Res. in Math. Ed., Vol. 29, No. 1, pp. 41-62.
  7. Finkel, D.L., 2000, “Teaching with Your Mouth Shut,” Boynton/Cook Publishers.
  8. Boss, S., 2000, “Teenager or Tyke, Students Learn Best by Tackling Challenging Math,” Northwest Teacher, Vol. 1, No. 1, pp. 8-13.
  9. Bain, K., 2004, “What the Best College Teachers Do,” Harvard University Press.
  10. Twitchell, D., 2002, “The Value of ‘Impossible’ Problems,” posted on ArticlesForEducators.com, www.articlesforeducators.com/dir/mathematics/impossible_problems.asp.
  11. Wu, H., 1994, “The role of open-ended problems in mathematics education,” J. of Math. Behavior, Vol. 13, No. 1, pp. 115-128.

Acknowledgment:

The author is grateful to Armann Ingolfsson and others at INFORMS Transactions on Education for thoughtful comments including the “broad areas of relevance” noted (i), (ii), and (iii) above.