Special report: Your current society’s actual values!

PART 2—WHATEVER HAPPENED TO STANDARDS (permalink): How much cheating-by-erasure may have occurred in DC’s public schools?

That question will be quite hard to answer, but the story to date has a comical aspect. But first, a basic distinction:

Although the District isn’t a state, it does have a “state education department”—the Office of the State Superintendent of Education (OSSE). That office supervises the DC public schools. In 2008, when this story began, Deborah Gist was DC’s “state superintendent of education.”

Let’s get on with our story:

In the spring of 2008, Michelle Rhee was finishing her first year as chancellor of the DC Public Schools But uh-oh! In Monday’s report in USA Today, Gillum and Bello say that Gist “recommended that the scores of many schools be investigated because of unusually high gains” (see THE DAILY HOWLER, 3/29/11).

When test scores jump in implausible ways, serious people will want to check twice. A bit later, the USA Today reporters describe Gist’s action in a bit more detail:

GILLUM AND BELLO (3/28/11): In 2008, the Office of the State Superintendent of Education (OSSE)…asked McGraw-Hill to do erasure analysis in part because some schools registered high percentage point gains in proficiency rates on the April 2008 tests.

If USA Today’s report is accurate, this would suggest that the OSSE was concerned by the size of some schools’ score gains. At any rate, 96 schools were flagged by McGraw-Hill for excessive wrong-to-right erasures. In at least three schools, 85 percent of the classrooms were flagged for excessive erasures.

But uh-oh: “Although all of the experts consulted by USA TODAY said such aberrations should trigger investigations at the school level, that did not happen in D.C. in 2008.” As they continue, Gillum and Bello report what happened in more detail:

GILLUM AND BELLO: In November 2008, Deborah Gist, then the state superintendent of education, recommended that D.C. public schools and several charter schools investigate why their erasure rates were so high. "It is important to note that these (data) analyses do not suggest reasons for the high erasure rates," Gist wrote to the schools. "However, it is important that all procedures available to us are employed to guarantee the validity of the state assessment system."

Seven charter schools responded to OSSE and carried out probes. Gist's proposal met resistance from Rhee's staff, documents obtained by USA TODAY show. Memoranda flew back and forth for five months as D.C. school officials questioned the methodology and the rationale for an investigation.

[…]

In April [2009], state superintendent Gist left Washington to take a job as head of Rhode Island's state school system. Her successor, Kerri Briggs, then dropped the request for D.C. public schools to investigate its schools. Both Gist and Briggs, now director for education reform at the George W. Bush Institute in Texas, declined to comment.

Seven charter schools did conduct probes. (USA Today doesn’t report the results.) But for better or worse, the DC Public Schools rejected Gist’s request.

The earth continued to turn; before long, it was time for testing again. But in the spring of 2009, the annual testing produced a new set of excessive erasures. This time, though, the DC Public Schools got off its duff and conducted a probe. Or at least, they gave the appearance.

“After the 2009 tests, the school district hired an outside investigator to look at eight D.C. public schools,” the reporters note. In the following passage, we get a bit more detail:

GILLUM AND BELLO: The tests administered in April 2009 produced another round of score improvements for D.C. schools. The proficiency rate districtwide in reading for elementary schools rose 3 percentage points over 2008; the math rate jumped 7 points.

Data obtained by USA TODAY show that, after those tests, 46 D.C. public schools were flagged by McGraw-Hill for having classrooms with high rates of wrong answers changed to right ones.

[…]

OSSE chose eight D.C. public schools plus four charter schools for investigation. District officials would not identify the eight D.C. public schools, but USA TODAY was told by a former official that Noyes was one of them.

[John] Fremer, president of Caveon Consulting Services, the Utah company hired by D.C., acknowledges the investigations were limited and focused mainly on process.

In this, the second year of Rhee’s reign, the District schools did respond to the OSSE’s request. But that is where the mordant humor starts to enter the story.

Please note: The number of schools flagged for excessive erasures dropped in this second year of Rhee’s tenure, from 109 in 2008 to 46 in 2009. That said, it’s hard to tell what this drop might have meant. In this chart, USA Today notes that a new methodology was used in 2009 (see note at bottom); the newspaper makes no attempt to say if this methodological change might account for the lower number of schools getting flagged. At any rate, the mordant humor enters our tale when we read USA Today’s account of what these District-funded probes turned up.

In this passage, Gillum and Bello describe the limitations placed on the probes. After that, they describe what the probes turned up:

GILLUM AND BELLO: Fremer, president of Caveon Consulting Services, the Utah company hired by D.C., acknowledges the investigations were limited and focused mainly on process. "Did everyone who should have received training (on how to give tests) receive training? Was there a mechanism in place for checking out the test booklets? How were they stored?" he says in describing the questions.

When Caveon interviewed individual teachers, Fremer says, an official from the school district was always present and occasionally a principal sat in. Teachers were asked about why erasure rates were so high, Fremer says, but he adds: "We didn't ask if teachers cheated."

D.C. school officials did not ask Caveon to do its own analysis of the test data, Fremer says. For other investigations, he says, Caveon has gone to the testing company to examine the tapes of the scanning machines that detected which wrong answers were erased and changed to right. It is helpful, he says, to examine each student's answers to determine, for example, whether students got hard questions right but missed easy ones. That unlikely outcome can indicate tampering.

After Caveon's investigation, D.C. school district officials cleared all but one of the eight public schools originally on the list. OSSE approved those findings, according to documents USA TODAY obtained.

At Burrville Elementary, where half of the school's classrooms had been flagged for high wrong-to-right erasure rates by McGraw-Hill, the conclusion was that one teacher had wrongly cleaned up stray pencil marks on student answer sheets. That was not allowed, OSSE said in a letter to Rhee. In that classroom, students' math and reading scores were invalidated.

At another school, Stanton Elementary, where wrong-to-right erasures in one fourth-grade class were about 10 times the district average, no violation was found. But an unidentified teacher was banned from administering future tests. The letter sent to Rhee by OSSE did not explain why.

Ted Trabue, president of the State Board of Education, agrees the 2009 investigation was limited. But he credits OSSE, which sets test security policy, for tightening the rules since 2009.

From the outside, there is no way to know what may have occurred in these various schools. But on the basis of this rather circumscribed probe, the DC Public Schools determined the following:

They found that one (1) well-intentioned teacher had cleaned up stray pencil marks on some answer sheets. And not only that! At a second elementary school, one (1) teacher was barred from administering future tests, though “no violation was found.”

And sure enough, that was it!

It’s hard to miss the mordant comedy found in this second-year probe. Meanwhile, excessive erasures were still running wild at various schools, including at the Noyes Education Center, the suddenly top-notch school on which Gillum and Bello have focused. In 2008, passing rates had soared at Noyes—and McGraw-Hill flagged 75 percent of its classrooms for excessive erasures. But that pattern never changed at Noyes, despite the probe Rhee finally launched. In 2009 and 2010, at least 75 percent of the school’s classrooms were still getting flagged for excessive erasures, including some classrooms whose erasure rates truly went through the roof.

Michelle Rhee has always been all about high standards. That said, the standards for her erasure probe seem to have been a bit lax. Tomorrow, we’ll look at some of the ways this story has progressed this week—and we’ll journey to Atlanta. But plenty of humor can be found in this tale.

This kind of thing is decades old. Sometimes, you just have to laugh.