Ken Bain and his research team have a perspective on student evaluations of college teaching that makes a tremendous amount of sense to me. Bain outlines two points in his excellent book, What the Best College Teachers Do.
point 1: Ask the right question
If you ask students the right questions, their answers can help you evaluate the quality of teaching…We know, for example, that if you ask students something like, “Rate your learning in this course,” their responses usually have a high positive correlation with independent measures of their learning.
So we shouldn’t ask students whether they like the course, or whether it is easy to take notes from this instructor. We should ask them how much they learned.
Of course, we don’t all agree on what it means to learn in a college course. That’s where Bain encourages us to seek the kernel of truth in student evaluations.
point 2: Skempian analysis
Richard Skemp wrote about the difference between instrumental understanding and relational understanding. This difference comes out (although Skemp is not referenced) in Bain’s writing, referring to a Scottish study,
Students who thought learning meant memorization praised courses that valued recall while those who expected to reason on a higher level reported that they didn’t learn much.
And vice versa. Those who expected to think deeply and explore concepts praised courses in which these things were valued, while these same courses were panned by students who thought of learning as memorizing. This is precisely the conflict Skemp pointed to.
This is where Bain’s research provides some insight. His group’s work identified characteristics and—especially—practices of the best college teachers (identified by a metric Bain spells out nicely). So Bain isn’t worried about what the student evaluations look like for the best college teachers; he is concerned with what the best college teachers do with these evaluations.
One of the teachers in the study exemplifies these practices. Bain quotes him:
I have some students…who come into my class thinking that all they have to do is memorize and regurgitate. The class frustrates them at first because I’m asking them to understand and reason. In the end, if they give me low marks, it’s because I’ve failed to affect their concepts of what it means to learn my discipline
That is, the best college teachers seem to seek the kernel of truth in their student evaluations. It’s not that they take the evaluations at face value; they ask what can be learned from the substance of the evaluations.
I aspire to that.