Washington, December 10, 2020 — By not adjusting for school and classroom factors outside the control of educators, classroom observation scores for Black teachers in Chicago Public Schools unfairly penalize them for being more likely to teach in schools in low-income neighborhoods with students who are academically disadvantaged, according to a study published today in Educational Evaluation and Policy Analysis, a peer-reviewed journal of the American Educational Research Association.
The study found that the typical Black teacher in Chicago ranked at the 37th percentile in classroom observation scores, compared to the 55th percentile for the typical White teacher. Once the researchers controlled for differences in school and classroom factors, including student poverty, misconduct rates, and incoming academic achievement, the gap statistically disappeared. This race gap in scores is important because the evaluation of teacher performance through classroom observations is a common practice in schools across the country, often paired with other measures of teaching effectiveness, such as those based on student test scores. Teachers’ classroom observation scores are used by school districts nationwide to determine teacher tenure and promotion decisions.
The study, conducted by Matthew P. Steinberg of George Mason University and Lauren Sartain of the University of North Carolina at Chapel Hill, analyzed student data from Chicago Public Schools for academic years 2013–14 and 2014–15, the first two years of a newly implemented teacher evaluation system. More than 5,500 K–5 teachers from 411 CPS elementary schools were examined.
The study found that 89 percent of the explained Black-White gap in classroom observation scores was driven by differences between the characteristics of schools where Black and White educators typically worked, such as the proportion of students receiving free or reduced-price lunch or average student achievement on end-of-year standardized exams. The other 11 percent of the explained gap was related to classroom-level differences within individual schools, including student poverty, misconduct, and academic achievement. None of the race gap was explained by differences in teachers’ measured effectiveness in improving student achievement, by school culture, or by the race of the teachers’ evaluators.
“Our findings indicate that these classroom observation scores do not equitably compare the performance of teachers who taught in very different classroom and school settings,” said Steinberg, an associate professor of education policy at George Mason University. “The race gap in teacher scores does not reflect real differences in teacher performance.”
“Left unadjusted, these scores may lead to disproportionate and incorrect identification of Black teachers for remediation and dismissal, and may have serious implications for the diversity of the teacher workforce,” Steinberg said. “Our study, which focused on Chicago, raises questions about how classroom observation scores are being analyzed and used by school leaders across the United States. School leaders everywhere need to account for the potential impact of school and classroom factors on teacher scores.”
The authors noted that policymakers and school leaders should encourage the type of teacher sorting that increases opportunities for minority students to be exposed to minority teachers, but that minority teachers might not seek out teaching assignments in some of the country’s most economically and racially segregated schools if they believe their evaluation ratings will suffer.
“Across school districts in the United States, and in particular, urban school districts, there is a widening demographic and racial gap between teachers and their students,” said Steinberg. “At the same time, prior evidence shows that minority students experience both short- and long-term educational benefits when taught by minority teachers. The potential labor market consequences which we observe in the context of Chicago’s teacher evaluation system should be of concern to students, families, and school and policy leaders nationwide.”
“Our findings underscore the importance of policymakers refining personnel evaluation systems in ways that ensure greater equity and improve the educational circumstances of teachers and students in the most disadvantaged schools,” Steinberg said.
The authors noted that during the COVID-19 pandemic, how teachers are evaluated, particularly in remote environments, is top of mind for school districts and unions.
“While it is perhaps more important than ever that teachers receive feedback on their practice in order to best serve students in these less-than-ideal classroom contexts, teachers raise valid concerns about how observations and student test scores will be used to judge their effectiveness,” said Sartain, an assistant professor of education at UNC-Chapel Hill. “The issues raised in our study may be of even greater concern now, in light of COVID-19, when some of the inequities in students’ home environments are even more relevant.”
Funding: Sartain received funding from the Spencer Foundation in support of this research.
Study citation: Steinberg, M. P., & Sartain, L. (2020). What explains the race gap in teacher performance ratings? Evidence from Chicago Public Schools. Educational Evaluation and Policy Analysis. Prepublished December 10, 2020. http://doi.org/10.3102/0162373720970204
###
About AERA
The American Educational Research Association (AERA) is the largest national interdisciplinary research association devoted to the scientific study of education and learning. Founded in 1916, AERA advances knowledge about education, encourages scholarly inquiry related to education, and promotes the use of research to improve education and serve the public good.