Report: Lowest-scoring teachers concentrated in poorest schools

Print More
A new report shows teachers with the lowest scores on evaluations are overrepresented in the district's highest-poverty schools.

Photo illustration by Grace Donnelly

A new report shows teachers with the lowest scores on evaluations are overrepresented in the district's highest-poverty schools.

Teachers who score the lowest under the district’s relatively new evaluation system are overrepresented in schools with the highest concentration of poor students, according to a new report that looks for the first time at how teachers’ scores correlate with characteristics such as student poverty, teacher race and school climate.

Issued Tuesday by the University of Chicago Consortium on School Research, the report shows that 30 percent of teachers with the lowest observation scores teach in the district’s highest-poverty schools, while just 9 percent teach in lower-poverty schools. Conversely, more than a third of the best-scoring teachers work in lower-poverty schools, and just 6 percent are in the district’s poorest schools.

That means students who attend the poorest schools are not only “more likely to be taught by teachers with the bottom scores on observations, they are also the least likely to be taught by teachers who have the top scores.”

Teachers at higher-poverty schools who scored in the top 20 percent on their observations also scored “substantially less” than teachers in the top 20 percent at low-poverty schools. And differences in evaluation scores persisted even after controlling for teacher experience levels and credentials.

Researchers say, however, that they don’t know whether the discrepancies reflect true differences in the quality of teaching or that it’s just harder to be effective or get high marks in higher-poverty schools.

“They may reflect that it is harder to get high scores in unorganized, chaotic schools or in schools with few resources or instructional supports for teachers, as other studies have shown,” the report notes.

Past research has shown students in higher-poverty schools often receive more scripted instruction and test prep — instead of the kind of deeper learning that earns higher evaluation marks — and that teachers prefer to work in schools where they’re more likely to be effective.

The researchers expressed concern that the rating system could create disincentives for working with disadvantaged students and have ramifications for teachers of color.

“If minority teachers are more likely to work in contexts where it is difficult to get high ratings, the composition of the workforce itself could be affected by the personnel decisions based on evaluation scores,” the report states. “This is especially true for African American teachers, who disproportionately teach in higher-poverty schools.”

Similar findings elsewhere

Chicago’s teacher evaluation system, called REACH, incorporates multiple teacher observations and a so called “value-added” component that measures student growth on math and reading tests. In an attempt to keep a level playing field, the growth ratings are adjusted for student characteristics and prior test scores.

Chicago developed REACH to comply with a 2010 state law. Previously, the district used a 1970s-era checklist.

While the report identifies trends, it says more research would be needed to explain the causes. But co-authors Jennie Jiang and Susan Sporte say the strong correlation between poverty and teacher evaluation scores is consistent with recent findings elsewhere.

“Chicago is not the only one,” Jiang says. “You see this in many, many states and districts across the nation.”

In a written statement, CPS Chief Education Officer Janice Jackson said while REACH represents “a dramatic improvement over past evaluation systems,” the district is working to improve it.

CPS and the Chicago Teachers Union have discussed taking into consideration school context — such as poverty levels, special education needs and administrative changes — on teacher evaluations and Jackson wants to make that a higher priority, according to a recent letter she sent to the joint committee reviewing REACH.

“The challenges highlighted in this report are complex and are not faced by CPS alone, but we are working with the CTU, researchers and technical experts to better understand their root causes and address any improvements we can make to better support our teachers and students,” Jackson said in a statement.

The report analyzes teacher evaluation data from 526 Chicago schools — excluding those that are privately run — during the 2013-14 school year, the second year the district used REACH.

Researchers looked at observation data from some 18,000 teachers — both tenured and non-tenured — as well as value-added data from 4,900 teachers who work with 3rd- to 8th-graders in math and reading. (They excluded high school teachers from the value-added analysis because that part of the evaluation is still in what they considered to be a pilot phase.)

Experience, race, gender

Among the report’s other findings were:

  • Teachers with more years of experience scored better on their observations and in student growth than newer ones, and teachers with credentials like National Board certification and advanced degrees did better on their observations. First-year teachers scored “significantly” lower on their observations and in student growth, and scores dipped slightly for teachers who had more than 15 years of experience.
  • On average, white teachers scored better than non-white teachers on their observations. But much of that difference was due to the “substantial relationship” between observation scores and school characteristics. Teachers of color are overrepresented at the highest-poverty schools, where teachers were more likely to get lower observation scores.
  • There were no significant differences by teacher race or ethnicity on the student growth components of the evaluation.
  • Male teachers had lower observation and student-growth scores than female teachers with the same level of experience and credentials — and the results changed little when adjusted for the school they taught in. This finding surprised researchers at both the Consortium and the CTU.
  • Teachers in schools with better organization and learning environments, as measured by the Consortium’s 5Essentials and supplemental surveys, tend to score better on observations and student growth.

When the teacher evaluation system was put in place, Sporte says, many believed teachers could be ranked without regard for where they taught.

“This work would seem to indicate that’s just not the case, that… you can’t just pick that teaching part out from that interactivity that happens between teachers and students in the class,” she says.

Carol Caref, a CTU researcher, says the report substantiates much of the anecdotal evidence the union has collected on REACH.

She says it’s not surprising that teachers at the highest-poverty schools have low ratings. In challenging schools with poor leadership, it’s harder to develop practices consistent with high ratings, she notes, such as helping students take responsibility for their own education.

“When the school is kind of portrayed as a more struggling school, that interferes with the culture and climate of the school,” she says. “You’re not teaching in a vacuum.”

While teachers and principals play a role, she says, the highest-poverty schools would benefit from accommodations like more social workers and smaller class sizes.

Findings can point toward improvements

The study didn’t look at how a teacher performed in front of different students — say in honors geometry versus a general education math class — to see if students affected the teacher’s score. It also doesn’t address possibly systemic differences in how students are assigned to teachers.

“For example, minority or male teachers may be assigned more students with discipline issues, or with previous low achievement or special education status,” the report notes. “If so, it is possible this might negatively impact what an observer sees in their classroom and therefore what score they are assigned.”

The report also doesn’t address whether the evaluator’s characteristics, for example race and gender, had an impact on the teacher’s observation. (REACH observations are conducted by trained principals and assistant principals who must pass a state certification test.)

And while the report continues to raise questions about whether observations and value-added measurements “accurately capture teacher quality,” its authors say the new data give a better idea of what’s going on in the classroom and how to improve it.

“You’re never going to be done with teacher evaluation,” Jiang says. “This whole process is really an evolution.”

But the CTU’s Caref has her doubts. She says it’s hard to separate the impact of REACH from the many other changes that have happened in the district in recent years — from a new state standardized assessment to CEO turnover — and “the fact schools are so underfunded and under-resourced.”

“How can we fix REACH so it would do a better job? I’m not sure it’s possible,” she says. “We would have to fix the school system.”

  • Front Row

    Principal ratings should be discussed in this article too. Most of the most pathetic and inept administrators find their way to high poverty school.

  • Concerned Parent

    Poor or no training of principals on this evaluation process now and when it all started. There has been much money spent on central office’s
    instructional effectiveness people – a failure. There has not been one workshop this year on this process for principals. When there was one last year, principals were not allowed to ask questions.

    One principal shared three workshops he was required to attend in the past. One was taught by a 5 year retired principal who had never used this process. That shameful supes academy gave a workshop with wrong information from a person from out of state who never used it. There was one in the summer, but the presenters were unsure how the teacher evaluation process was going to work for that school year.

    It is possible that a few principals purposely keep teacher ratings low so that teachers will not be able to leave their school. This forces teachers to stay as they have low ratings. This also prevents a low poverty school having to hire a poor teacher to replace a good one. Teachers are very hard to find for CPS, especially for high poverty schools. New teachers take much more work to evaluate.

    Some network chiefs tell principles to give lower ratings to teachers if student scores are low and to use grades given by the teacher to students
    as a measurement in this process.

    Wonder what would happen if white teachers were in high poverty schools?

    So disappointed to see male teachers are rated low.

    When there is no support from the special education office and they send students to schools that are in need of hospitalization or intensive programs, the lack of student success and the crisis students disrupting the rest of the class, become teacher responsibility. This makes for a lower teacher evaluation. Male teachers may be in these rooms on a wing and a prayer, as special education office doesn’t help and seems to cause this problem.

    Principals must keep the 5 essential teacher survey results high. This survey was created by the same people who did this study. It is possible that at much lower poverty schools, they may have some poor performing teachers, but to make teachers happy and for a good survey results on 5 essentials, principals evaluate those teachers more positively.

    CPS tends to punish high poverty schools by making them use canned curriculum and lots of testing. Children do better at a school that doesn’t
    have a canned curriculum and all that testing. Talented teachers want to be away from that and don’t want to work in schools like this. New teachers and their students may not do well at high poverty schools but these schools are the ones hiring. New teachers are observed more often too.

    In reading this report, a network chief is listed as a committee member. The union I think filed a grievance against him for his reference to principals against red-shirted teachers.

    A vicious cycle it seems to be.

  • Northside

    It doesn’t take much training to see a teacher who looks a like he/she is having a bad day and say “hey you know what, I will come back another day because we are all human” and we are all in this together. instead, many like to come and do the classic “gotcha”…..while they do the exact opposite to the ones they are “sweet” on or have political connections……Not ALL…but some …