David Hoppe

David Hoppe is available
for freelance writing and editing assignments; and consulting with commercial and nonprofit cultural organizations. Resume and references available upon request.

Site managed by
Owl's Head Business Services

:: Educational testing

Just another job

By David Hoppe

I had a real Wizard of Oz moment the other day - make that a Wicked moment, if you prefer. You know the Wizard's story: he's a fake who hides behind a curtain, dishing meaningless bromides to his subjects in the Emerald City so they'll think he's got some sort of special mojo.

The moment I'm talking about came courtesy of a writer named Dan Dimaggio, whose article, "The Loneliness of the Long-Distance Test Scorer," was published in December's Monthly Review magazine ( www.monthlyreview.org ).

For the past three years, Dimaggio has worked as a test scorer, reading and scoring the written-response portions of the kind of standardized tests given to millions of American third to twelfth graders. The work is seasonal, usually lasting around four months of any given year.

Dimaggio, who lives in the Twin Cities, says he has personally read tens of thousands of papers, for which he has been paid at a rate of 30 to 70 cents per paper. That means he has to score 40 papers every 60 minutes in order to make $12 an hour.

Dimaggio's article was a revelation to me. Although I've never been a great fan of standardized testing and am particularly skeptical about the increasing faith politicians, business leaders and bureaucrats seem to have in it, I also understand that these tests can, in fact, serve to provide us with one, very general, indicator about how kids are doing in school.

What never occurred to me was that rather than being an objective and impartial reflection of a student's educational attainment, these tests are yet another form of boiler room toil, subject to the flaws and foibles of our growing population of over-educated, underpaid American schmoes.

I worked in a boiler room once. I was a telephone solicitor, trying to sell people Time-Life books about the Wild West, or some such thing. A bunch of us of sat behind heavily bruised steel desks in an office above Market St. in San Francisco. On those desks were a hunk of phone book for the area code each of us was supposed to call and an ashtray. The only other furnishing in the room was a Coke machine. The job involved calling over 100 phone numbers every hour.

It turns out something similar happens with the tests our kids take at school. "In test-scoring centers, dozens of scorers sit in rows, staring at computer screens where students' papers appear.each scorer is expected to read hundreds of papers," writes DiMaggio. "So for all the months of preparation and the dozens of hours of class time spent writing practice essays, a student's writing probably will be processed and scored in about a minute."

Since scorers are paid per paper - and since there are only so many papers overall - scorers are in a race against their coworkers to do as many papers as they can. DiMaggio says this resulted in contradictory messages from the testing company, warnings, for example, that he was scoring too fast, with simultaneous messages that his group was way behind.

DiMaggio writes: "Unfortunately, after scoring tests for at least five states over the past three years, the only truly standardized elements I have found are a mystifying training process, supervisors who are often more confused than the scorers themselves, and a pervasive inability of these tests to foster creativity and competent writing." According to DiMaggio, testing companies' "ultimate goal is to present acceptable numbers to the state education departments as quickly as possible, beating [the departments'] deadlines."

It's a numbers game. And the thing about numbers is that if they're too low or too high they won't be considered reliable. "Usually in a day or two, when the scores we are giving are inevitably too low (as we attempt to follow the standards laid out in training), we are told to start giving higher scores," writes DiMaggio. "For some mysterious reason, unbeknownst to test scorers, the scores we are giving are supposed to closely match those given in previous years."

Meanwhile, some kid, or some teacher, is going to be judged based on whether or not those test scores are up to par. DiMaggio writes that scorers can never know how students are affected by the scores they get. "Whether Marissa will be prevented from going to seventh grade with her friends because one of us, before our first cup of coffee kicked in, decided that her paper was 'a little more like a 3 than a 4.' Whether Marissa's school will be closed or her teachers fired (to be reborn as test scorers next spring?) remain mysteries to the test scorers."

We've substituted testing, the illusion that numbers never lie, for the hard work of deciding what an education should be. Rather than reform, our fixation on educational testing is a default setting, the cultural failure to agree on whether our kids should be taught or merely trained. That's why the next time I hear our schools described in terms of test scores, I'll think of the Wizard of Oz - the test scorer behind the curtain.