Reshaping student assessment in CPS

Print More

For four years, Chicago Public Schools has been retaining students who failed to meet prescribed minimums on the Iowa Tests of Basic Skills (ITBS), directing them to summer school to boost their scores or face retention. This has constituted the Board of Education’s attack on social promotion—with the Iowa tests as the prime gatekeeper.

The policy has borne fruit in rising test scores, greater success in summer school and glowing public notice, notably a mention by President Clinton in his 1999 State of the Union message. “The Iowa scores are a good predictor, and parents and people in general are comfortable with them,” insists Philip Hansen, the schools’ chief accountability officer.

But critics have pummeled the policy for its reliance on the ITBS as the only cause for retention and for its punishing use of retention to deal with failing students. A 1999 study by the National Research Council in Washington, D.C., using Chicago as a case in point, questioned using any one standardized test as the sole factor in determining promotion, in part because “the validity of test and retest scores depends in part on whether the scores reflect students’ familiarity with actual test items or a particular test format.”

Robert Hauser, a social statistician at the University of Wisconsin, further faults the Chicago approach because its tests are not linked to its curriculum, “a violation of professional standards.”

Even Riverside Publishing Co., the firm in suburban Itasca that produces the ITBS, admonishes districts not to use its product as a one-shot assessment tool. “The test should not be used as the sole determinant in making educational decisions,” remarks Margaret Sherry, a spokesperson for Riverside.

While the Board of Education is sticking with the ITBS, it also is introducing an adjustment that brings in other measures for some marginally performing students—a move that’s being hailed by some observers and found wanting by others.

Going into effect for summer school this year, the change permits teachers to consider grades, attendance, conduct, homework and test results in recommending advancement. Students who fall within certain limits below the board’s cut-off scores on the ITBS are entitled to an automatic review following summer school.

“I felt that we’d gone high enough with the test standards, that now we’d rest and bring in the classroom aspects and codify them,” says Schools CEO Paul Vallas. “Students who have trouble with tests now have the insurance policy of being able to do well in class and get to move on that way. This becomes a carrot-and-stick approach that will promote academic behavior.”

Hansen says that grades have become more reliable, too. “There aren’t just A’s and B’s anymore,” he says. “There’s more discussion about grading among the faculty—the 3rd-grade teacher is talking to the 2nd-grade teacher.” He adds that the Consortium on Chicago School Research “assured us that grades had begun to mean something.”

The change in the promotion policy immediately paid off for summer school students. This year a total of 5,708 students were held back, compared with 11,725 last summer. School officials could not immediately say how many hit test score targets.

The promotion relaxation brings Chicago more in line with other cities that have moved to end social promotion but are relying on multiple measures of student performance. Los Angeles is phasing in a retention policy based on grades in reading and (soon) math, plus other class work and performance on the Stanford 9 Achievement Test.

At the insistence of Mayor Rudolph Giuliani, New York City schools this year began retaining students in grades 3 and above on the basis of city and state standardized tests, class work and attendance. Boston now predicates advancement on a range of measures, but at 3rd and 8th grades, students must pass standardized math and reading tests.

While Hansen insists Chicago’s policy is evolving on its own and not in response to outside pressure, its critics aren’t so sure. “The changes they have made—and they might not admit it—are a response to all this concern over all the retentions,” says Donald Moore, executive director of the research and advocacy group Designs for Change. “The public reaction is having an effect.”

The relaxed criteria for retention fails to impress Moore. “The basic problem for us is the widespread retention of students,” he says. “Those students are harmed in that most don’t catch up, and they will drop out of school later on. It [the relaxation] is a small step forward that they aren’t being retained by using tests.”

But applause rings from other quarters. “This is an enormous step in the right direction,” says RaeLynne Toperoff, executive director of the Teachers’ Task Force, a reform group. “This allows for major input from the person closest to a learner’s progress—and that’s the teacher.”

“We have to look at more than a test score,” agrees Doris Sams, president of the Chicago Region PTA. “We’re living in a real world, not a testing world.”

“We know it’s important to have standard measures,” says Pam Massarsky, recording secretary of the Chicago Teachers Union, “but they should never take the place of what a teacher knows about a child. The board ignored the professionals. We expressed our disapproval during strategic bargaining.”

Massarsky says the CTU would like teacher input to be a factor in all cases, not just those where students score within a certain range.

John Ayers, executive director of the business-based Leadership for Quality Education, is comfortable with the relaxation, but he also warns: “I want to make sure that principals and teachers don’t abuse this. You’ve got to be careful that too many principals don’t just push low-performing kids over into the next grade, and we’ll be back at social promotion again.”

Ayers also voices the perception that standardized tests matter today: “The skills measured on the ITBS, while they are narrow and about only certain parts of cognition, are important to survive in the world.”

Beverly Tunney, president of the Chicago Principals Association, could not be reached for comment, and other association officials contacted by Catalyst declined to comment.

Multiple measures

Before Vallas announced the proposed change in the retention policy, the LSC Summit, composed of Designs for Change, the Teachers’ Task Force, Parents United for Responsible Education and other reform groups, began circulating a proposal to scrap the Iowa-based accountability system and replace it with one using multiple measures of student performance, including portfolios and parent input.

“The system needs to reduce what it’s investing in oversight and punishing tests and put more into teacher training and authentic assessment,” says Toperoff.

“Authentic” or performance-based assessment refers broadly to ways of judging students through open-ended questioning and actual classroom tasks. It is advocated by education progressives for its ties to actual curriculum, whereas the Iowas, standardized tests keyed to national averages, are generally reviled.

From the beginning, reliance on the Iowas to spell summer school and retention for students and probation, reconstitution and, now, intervention for schools, appalled members of the LSC Summit, “People had a problem with basing such decisions on one instrument,” says Toperoff.

“We took our concerns to the Board of Education and the City Council Education Committee, and people told us, ‘If the schools aren’t using the right approach, what is the right one?,” she relates.

In early 1998, the LSC Summit held a conference on authentic assessment addressed by Harvard University education professor Vito Perrone and Ernest House, a University of Colorado educator. It gathered input from Monty Neill of the Boston-based advocacy group Fair Test and from University of Wisconsin education professor Fred Newman. Soon Summit planners put their heads together to devise an alternative, which has developed over time.

The latest Summit proposal (called the ERA Plan) calls for each school to come up with its own assessment plan, tied to its school improvement plan, to judge students through multiple measures, including portfolios, performances and exhibitions of work. Enfolded into the plan would be results from Illinois Standards Achievement Test (ISAT), a statewide standardized exam administered in 3rd, 5th and 8th grades (reading, math and writing) and in 4th and 7th grades (science and social studies).

The Board of Education and the state would evaluate each school’s approach but allow schools to set their own promotion requirements, based on “clear expectations for student learning at each grade, aligned with state learning standards.” Schools themselves would decide whether to flunk or advance children, with those foundering given “intensive educational support.” Students may be retained “only with parental agreement in response to teachers’ recommendations,” the proposal reads.

The ISAT and city and state review of schools would provide external accountability for maintaining high standards throughout the system.

Alternative costly

One downside to performance-based assessment is that it’s more expensive, given that it demands more teacher time and training and, optimally, outside judges.

Lauren Resnick, a testing expert at the University of Pittsburgh, says that direct assessment can cost four or five times the amount required to administer a paper-and-pencil standardized test.

The board’s Hansen says cost even was an issue in the grading of the board’s Chicago Academic Standards Exams (CASE). “We could have either hired people at $8 an hour to grade them over a weekend or have teachers do them,” he says. “We went with the teachers.”

“No, performance assessment isn’t cheap to do, but in the big picture you’re not talking about a huge amount of expenditure,” observes Tony Bryk, senior director of the Consortium on Chicago School Research and a proponent of multiple measures.

Samuel Meisels, a University of Michigan education professor who helped originate an authentic-assessment method called “work sampling,” is affronted at reservations citing expense: “That’s an intolerable excuse. There’s nothing more valuable than teaching children well. How expensive is it to retain a child in Chicago? What’s the cost to remediate, or the cost to self-esteem?”

The Summit would fund its proposal by diverting the $100 million it says the board spends annually on summer school, after-school programing and tutoring geared to the ITBS, as well as the $7,000 it spends annually for every retained student.

Hansen believes that portfolios and the like will inch their way into grades that will now influence promotion. “Any good teacher should be looking at projects, experiments, things like that, and should use them in learning,” he says.

Hansen also notes that the new “miscue analysis” tests now being used in the primary grades, where pupils read aloud from books to establish their reading levels, are just the kind of things that his critics want.

The CASE exams, end-of-semester tests in 11 core subjects tied to Chicago standards, are another step toward authentic assessment, he adds.

While pushing for a whole new accountability system, critics believe that switching from the ITBS to the Illinois Standards Achievement Test (ISAT) would constitute progress.

“They would be better than the Iowas,” says Moore of Designs for Change, underlining that the ISAT is based on defined standards of learning and that it features longer, more contextual material, notably in reading.

Hansen says that given the newness of the state’s revised testing program, “it’s a bit premature” to consider a switch. “We want all the kinks out,” he says.

Vallas predicts the Chicago School Board may link its accountability program to the ISAT in “two or three years. Those tests are very exacting, geared to standards, and I would prefer that students take one test instead of two.”

But for local accountability, he says, the ISAT should be given in the same subjects at every grade level, which would require the development of more tests.

Some authorities who voice misgivings about the ITBS as an accountability tool nonetheless counsel sticking with them, at least for a while. “You always want a time-line,” says Lauren Resnick. “If you use a measure halfway through—and then throw it out—the public won’t know what it’s getting.”

Hansen agrees: “One of the problems with urban school systems is that they keep changing their tests. We’re showing improvement, and for us to change now wouldn’t be appropriate.”

Hansen says that although the ITBS doesn’t gauge learning standards, there is a close correlation between ITBS and SAT reading scores for the students who take both tests, namely 3rd-, 5th- and 8th-graders. The accountability officer concedes that the two tests aren’t aligned in critical thinking, writing and oral skills, nor in math — the ITBS looks heavily at arithmetic ability, where the ISAT focuses on algebra and geometry.

The School Board has turned a deaf ear to the Summit’s call for public hearings on its plan.

Toperoff thinks some at the board may be sympathetic to the Summit’s pleading. “I have had many conversations with Phil [Hansen], as have many others in our group,” she says. “Phil in his heart of hearts agrees with us. But I think his understanding doesn’t extend beyond the fact that [the Iowa-based system] is easy to manage and cost-effective.”

Julie Woestehoff, executive director of Parents United for Responsible Education, predicts a complaint PURE filed with the U.S. Office of Civil Rights last October charging discrimination against African-Americans and Latinos in the retention policy will force an adjustment.

Other observers say the Summit proposal is dead in the water. “They [the Summit members] have little ability to force change, and less if they haven’t done their homework about how much it would cost to implement,” says Fred Hess, director of the Center on Urban School Policy at Northwestern University. “People like Phil Hansen pay attention to cost. The summit is asking the board to buy a pig in a poke. Also, people who are behind the LSC Summit are not people that leaders of the system take seriously. That isn’t to say they shouldn’t, but they don’t.”

“The reform groups stood by while Rome burned, when there were no standards or intervention in failing schools,” says Vallas of the days before his administration. “They make money off school reform. They have no credibility—I don’t deal with them.”

Vallas takes particular exception to the idea of parents determining retention, a linchpin of the Summit plan. “Retention would be subjective, and the LSCs would have influence,” the CEO asserts. “There wouldn’t be any standards.”

Moore promises that Designs for Change and other groups will organize parents at schools with the largest number of retained students to protest the promotion policy and force a change.