August 29, 2003
The teachers were teaching to the test but they guessed wrong

This story from the New York Daily News illustrates the dilemmas of the worldwide debate about exam standards very nicely:

An independent panel appointed to look into why an alarming number of high school kids bombed out on the June Math A Regents Exam has found the statewide test was too hard.

So the scores will be bumped up so that many of those who failed will get passing grades.

"In short, students in June 2003 were held to a higher standard than their counterparts a year earlier," the commission of math experts said in its report.

But the panel, appointed by state Education Commissioner Richard Mills, also found that teachers messed up by trying to anticipate subject matter.

"The teachers were teaching to the test," said Assemblyman Steve Sanders (D-Manhattan), chairman of the Education Committee. "The problem was, they guessed wrong."

The panel found that teachers, believing the test would be heavy on trigonometry, drilled students on their sines and cosines. But the test didn't have a single trig problem, the report said.

I'm starting to have heretical thoughts about exams, which can be summarised by me saying that I think this "panel" may well have done the right thing.

After all, the purpose of exams is to arrange people in order of merit. It must make distinctions. If they all get A*, which is apparently what is happening with children in England doing their A levels just now, the result is that the Universities don't know who's the best, so they interview them all, and go by how eloquently they talk or how politely they suck up to the interviewers. (I read that point made better by someone else recently, but I've lost the link. Sorry.) And if they all get F– you get the same kind of effect. So the logical thing might very well be to do what these people have in effect done, which is: first mark all the papers, and then decide which numbers get you into which grade.

The obvious objection to this procedure is that it fails to make any distinction between this year and last year and next year, or this decade and last decade and next decade. If the standards lurch around from year to year, who is to say whether this fifteen year old is any brighter than that seventeen year old?

Okay, cards on the table, I don't know the answer any more (I suggest) than you do. I don't see how you can have an exam system which separates the smartness of the examinees from the skill with which they were prepared for their exams by the teachers. After all, presumably, some teachers guessed right about what was going to be in these particular exams, and as a result, their pupils will presumably get a higher grade than they "deserve". Or, more simply, some pupils were relatively lucky in having teachers who did not "teach to the test" (to quote Assemblyman Sanders' words). They too presumably did rather better than rivals who actually "deserved" to do as well as they did.

On the other hand, pupils who have been better taught, are pupils you'd rather have at your university, regardless of how much worse they might have done with worse teaching, or how much better other pupils might have done with better teaching.

Okay I give up. I've failed. Micklethwait: F–.

UPDATE: According to the bit at the end of this I think it must have been GCSEs rather than A Levels where they got all those A*s. F––.

FURTHER UPDATE: ... and what is being said about A levels is that too many people are getting A, so could they please introduce A*s for those also. See this.

Posted by Brian Micklethwait at 10:35 PM
Category: Higher education