Department of misleading headlines: Student evaluations edition

Study Confirms Correlation Between Student Grades and Ratings on Instructor Evaluations

To those who are super concerned with grade inflation, this seems to be the smoking gun. Low grades lead to bad evaluations from students, and good evaluations are the result of high grades. There is an upward incentive pressure on grades that is degrading the quality of higher education. QED.

Only, that’s not at all what’s going on in the study.

The study is described thusly:

To test her hypothesis that poor teaching evaluations are a form of revenge, Vaillancourt designed three related studies, each involving between 150 and 176 mostly first-year university students.

Students were given up to 20 minutes to write a short essay on an assigned topic, and were told their work would be graded by professors or teaching assistants. The essays were then randomly assigned a high or low grade and, in some cases, comments such as “No suggestions, great essay!” or “This is one of the worst essays I have ever read!”

Students were then asked to provide feedback to their evaluator about his or her marking ability, fairness, helpfulness and general competence.

In the first study, participants given poor marks and negative comments were almost 10 times more likely to rate their evaluator harshly than those given high marks and positive comments.

It seems reasonable to conclude that anonymous scoring with zero useful, actionable feedback generates the results described. But I would also submit that anonymous scoring with zero useful, actionable feedback is crummy teaching.

And students in the study argued this quite strongly. Consider the epigraph from the published research piece:

My evaluator must not mark hastily and should re-read key points to offer better feedback. It is actually the poorest job in terms of evaluation that I’ve ever seen. It is easy for the evaluator to talk crap like “This is the worst essay I have ever read,” but when offering feedback about how it may be improved say nothing except “needs clarification.” How is this supposed to help me improve? This evaluator is a complete dumb ass, excuse my French.

This is far from a call for a better grade. It’s a call for meaningful feedback, and in fact supports a hypothesis that student feedback has a kernel of truth to which we ought to pay attention.

Indeed, Ken Bain and his colleagues argue that:

  1. when we ask students to rate the quality of their learning in a course, they are actually decent judges of this (notice that this is not among the things asked in the study in question), and
  2. the best college teachers sought to understand and meaningfully interpret feedback from students, using it to reflect on and improve their practice.

To restate, in Bain’s study, instructor quality was negatively correlated with instructor dismissiveness of feedback from students. Kernel of truth, people.

Advertisements

4 responses to “Department of misleading headlines: Student evaluations edition

  1. So could my willingness to stop class to argue the finer points of studying and learning (providing feedback to the lackadaisical) be what makes my students say, “I love your class Mr. M…I just hate math”? Year after year, this is what I hear. I do not fear reviews from my students. My D students love my class, but I can’t for the life of me figure out why…I’m harsh on laziness, I challenge students when they fail to meet my expectations…frankly, I can be mean. But they love it…but many of them just don’t want to do math or think and they will tell me that straight to my face…This has given me a lot to think about…anyone know a good shrink? 🙂

  2. How would your students respond, Don, to the question, how much did you learn in Mr. M’s class? That seems to be a question that generates more useful feedback than whether they like our classes.

    You might also—before paying that shrink—ask what students appreciate about your course. I was talking with a colleague about these matters the other day and he agreed that there is information in student evaluations but that we sometimes need to work hard to find it.

  3. I think I will try asking those two questions. It’s been a rough 10 years of teaching. I’m still learning, and I do like reading this column and others like it since I do get to think about my teaching. I really want my students to become better thinkers and enjoy math for its beauty and its usefulness. But evaluations (verbal or written) from students and parents have always bugged me for their lack of clarity. So I will start making my own. If you know of other good questions I should be asking, please, pass them along.

  4. This reminds me of my 8th graders who might not get placed in the ‘next’ math course when they go to high school. Parents can get upset and my stock value can decrease. Although, I try to make it abundantly clear that not every 8th grader is ready for Algebra nor ready to advance to Geometry as a freshman. I believe in placement based on ability, and this seems to conflict with parents who think there is a natural advancement in math. I like your idea. Ask students how much they learned in your class and what they appreciate about your class. This is valuable.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s