Become a Patreon!
Abstract
Excerpted From: Roger W. Reinsch, Sonia M. Goltz and Amy B. Hietapelto, Student Evaluations and the Problem of Implicit Bias, 45 Journal of College and University Law 114 (2020) (132 Footnotes) (Full Document)
This article addresses the implicit bias problems inherent in using student evaluations when making employment decisions concerning university faculty members. Research indicates that student evaluations contain implicit bias regarding race, gender, and a variety of other protected categories. We begin by looking at the current use, purpose and structure of student evaluations. We then explore what implicit bias is and the research that demonstrates that most of us have some sort of implicit bias. Once the concept of implicit bias is explained, we examine the research that indicates there is implicit bias in student evaluations. We then discuss the law and implicit bias generally, followed by specific legal issues that are raised. Next, we examine recent trends at some universities which have recognized and begun to address the problems with student evaluations. Finally, we offer recommendations as to how to evaluate faculty members' teaching using alternative methods.
Student Evaluations of Teaching are recognized as a common performance measure used by universities to make employment decisions with regard to faculty, as Emery and colleagues noted: “A current practice among colleges and universities in the USA is for the administration to use a student evaluation instrument of teaching effectiveness as part of the faculty member's performance evaluation.” Faculty at most colleges and universities today in the United States are subject to summative student evaluations. These summative student evaluations are used to make several employment decisions, such as when determining pay increases, tenure and promotion.
Summative student evaluations of teaching regularly use numerical scores to assess whether a faculty member is a good teacher. The intended purpose of summative evaluations is to provide information to administrators about the faculty member's teaching ability. However, student ratings may represent essentially little more than opinions, raising the issue of potential student bias as Hornstein noted: “The validity of anonymous students' evaluations rests on the assumption that, by attending lectures, students observe the ability of the instructors, and that they report it truthfully”. Institutions are typically not unaware of the likelihood of bias coloring evaluations, but use them anyway largely because of their convenience, as noted by Flaherty: “While some institutions have acknowledged the biases inherent in SETs, many cling to them as a primary teaching evaluation tool because they're easy--almost irresistibly so. That is, it takes a few minutes to look at professors' student ratings on, say, a 1-5 scale, and label them strong or weak teachers. It takes hours to visit their classrooms and read over their syllabi to get a more nuanced, and ultimately more accurate, picture.” Therefore, many administrators seem willing to discount or overlook the possibility of bias so they can continue to rely on SETs.
As stated, the evaluations typically use a Likert scale anchored with numbers, often from 1-5. These numbers are often associated with verbal anchors--for example, five usually means a high rating. Student evaluations are then compiled, producing a mean score for each question, and finally an overall mean score for that faculty member in that class. Those making employment decisions generally will rely mostly on the overall mean scores. At most institutions, although a mean of 3 is usually designated as “acceptable” with the scale's verbal anchor, being in the 3 range is considered by administrators not to be very good and, in fact, may be seen as problematic. The expectation is that everyone will be at 4 or above. This expectation is a false one and could be viewed as a manifestation of the Garrison Keillor syndrome, namely that “everyone is above average.”
Because the expectation is that everyone should be rated above average, the presence of implicit bias is even more concerning. Bias is likely to lower a particular faculty member's mean score while at the same time raising another faculty member's score. This makes it difficult for certain groups of people-- usually underrepresented group members--to achieve “above average” ratings while making it easier for members of majority groups to do so. Therefore, if implicit bias involves any of the protected categories under the law and evaluations are used to make employment decisions, then those employment decisions are based on some factors that are discriminatory and therefore illegal. For instance, a lower mean could result in the faculty member receiving lower merit increases or not getting promoted. This would be discriminatory under The Equal Pay Act and Title VII of the Civil Rights Act of 1964 as amended Civil Rights Act of 1991.
For these reasons it is important to look at the types of biases that might be in these evaluations. The issue of likely bias should not be dismissed because it is inconvenient or a challenge to come up with alternative unbiased measures of performance; we believe instead that it should be treated as a critical issue because student evaluations of faculty are used frequently and in a variety of employment related decisions: SETs play a role in the hiring process, tenure decisions, promotion decisions, salary decisions, and other benefits such as faculty awards. This is not a new concern and has been recognized in the education literature, such as by Basow and Martin, who noted: “The question of whether student evaluations can be biased is a critical one for those using them, whether for formative or summative purposes.” Therefore, we will look at some of the research on implicit bias.
[. . .]
Ultimately, it will still be up to a court to decide how much bias is too much bias. So, if the holistic approach or other mechanisms mitigate the bias, but do not eliminate it, is that still too much bias? There are no cases that have tested this issue yet, but we contend that it is a step in the right direction. As Falkoff said “if academic institutions do not take steps to assess teaching more holistically, they run the risk of losing talented faculty members for reasons that are not only inappropriate but may well be illegal.” Essentially, mitigation is a step toward both the retention of faculty who are performing effectively although SETs don't indicate that and toward avoiding possible litigation. In case litigation happens, efforts at mitigation will demonstrate that the university took substantive steps to avoid discrimination.
Roger W. Reinsch, Professor, Labovitz School of Business and Economics, Juris Doctorate from the University of Missouri-Columbia.
Sonia M. Goltz, Professor, School of Business and Economics, Michigan Technological University, Ph.D. Industrial/Organizational Psychology, Purdue University.
Amy B. Hietapelto, Professor and Dean, Labovitz School of Business and Economics, Ph.D. in Business Administration, University of Minnesota-Twin Cities.
Become a Patreon!
