Evaluations in the Era of Standardized Testing

In her book In Schools We Trust, Deborah Meier discusses many of the problems with standardized tests, and thus the inherent problems in our current education system, where "success," "achievement," and "failure" are based on those test scores.

She describes "norm-referenced tests" whose scores will never actually show overall improvement in students' learning:

By design, 50 percent of the students' scores will fall below the median--and 50 percent above. This would be true no matter what the competence of the population actually is. By definition, scores on these kinds of tests can never inform the public as to whether better learning and better teaching are taking place. (104)

When scores do rise, the test makers simply "renorm" the test, and determine a new median score, which will once again keep half the students above and half below the new norm.

It seems crazy that such tests, where half of students are automatically going to "fail," are the tools with which we gauge student, teacher, and school performance--yet that is exactly what happens.

As more educators have become aware of the inherent flaws of norm-based tests, the movement toward "new" tests has arisen. The MCAS, New York Regents, Virginia SOL, and others have led the way in providing a test that can be gauged against a state curriculum and be explicitly taught to -- something that doesn't rely on a supposed level of inherent intelligence or aptitude (like an IQ test or the SAT). How different are these "new" tests, though, in relation to the old norm-based ones?

Largely the answer is, not a lot--except that the absence of the much-maligned bell curve complicates deciding what items to include and how to set expectations, scores, and cutoffs. (122)

The revolution of these "new" tests, then, is really more of an image makeover of the old ones: although there is no longer a need to distribute scores along a curve and automatically score half as failing, they are still designed not as much to see what students know, but to see if they know what the test makers think they should know.

That is one of the fatal flaws of standardized tests (and, really, most evaluations). By trying to see what students know, there has to be a standard in place so it can be determined whether students know "enough." As Meier describes the process, it seems like a free-for-all: every contributor pushing for the inclusion of their beloved material, debate about how to pose the questions, potentially controversial political considerations (should the Armenian Genocide be called a "genocide?"). Just from this short snapshot, it is easy to see that test-making is not a science, and there is no one right way to do it. In fact, these tests that are supposed to measure "standards" become whatever they need to be:

...a minimum competency test in some states (as in Texas and North Carolina), or "tough" (as in Massachusetts, Virginia, and New York, although now a student can eventually pass the MCAS with just 33 percent of the answers right on tests designed to measure math standards). (126)

With these descriptions, it is hard to understand how there is not more uproar about standardized tests. The current high-stakes tests are "put together much faster, require less techinical validation and fewer reliability checks, are much longer, include more detailed factual questions... [and] the scores are no longer a mere artifact of the bell curve but are instead a mere artifact of the judgment of state commissioners" (127). Anyone looking at this description should be shocked, but it is more common to get reassured by the "objective" quality of the tests, because in the popular mindset, standardized = no room for error or bias. Teachers, administrators, parents--most everyone--sees these tests as fair assessments of what students should know, and fail to see the unintended negative effects.

These negative effects reach far and wide. They affect the direct classroom instruction (low-performing districts may mandate scripted lessons for their teachers), curriculum development (subject areas that are not tested, like music and art, are often cut), and student success (high stakes mean more students are being held back and/or failing, reducing their likelihood to go to college). Why is it that we trust standardized tests so much, hailing them as the path to school reform, when all they do is dig us into deeper holes?

I would propose a movement away from all of these so-called "standardized" tests, and toward a more well-rounded approach to evaluation. Meier writes that "the alternative to standardization is real standards" (132). In Horace's Compromise and its follow-up, Horace's School, Ted Sizer outlines some alternatives to standardized testing. He acknowledges that:

Tests, or any exhibition of mastery, are troublesome mechanisms. But the alternative to them--no basis to describe or assess what school is for--simply is worse. (68)

Sizer offers a variety of means to evaluate students: timed tests, essays, oral exams, portfolios of work. These forms of assessment would certainly take much longer to prepare, administer, and evaluate, but he argues that "those hours are better so spent than continuing with the known inadequacies of the status quo" (68).

I think he is absolutely correct. From what Meier described, it seems quite clear that the current system of high-stakes standardized testing is not working, since, by its nature, it sets a large number of students up to fail. I would propose a system more in line with Sizer's, which would let students show off what they have learned in a way that works best for them.

While some body of policy makers/administrators/teachers would still be deciding, in a way, what students should know, it would be much more flexible. A student presenting on Abraham Lincoln's presidency and how the emancipation of slaves has affected present-day race relations would have a much greater opportunity to show off his knowledge (and also his public speaking, researching, and critical thinking skills) than he would on the test question "Was Lincoln the thirteenth or sixteenth president of the United States?" which, like Meier says, is hard and is arguably not important to know (125). There would have to be some loose standard in place--a student presenting on geometry, for instance, should be expected to know how to find angle measurements--but it would not be nearly as strict as tests, where just the wording of a question could lead to a wrong answer.

A system of student exhibitions, portfolios, theses, or something similar would be much more effective in evaluating a student's knowledge and achievement. Each school or district would be able to determine which skills it deemed important, but Sizer's list of reading, writing, speaking, listening, measuring, estimating, calculating, and seeing seem like a good starting point. With this type of system, students could really show off what they know -- and not what they don't know. Students could choose areas of interest to them and take ownership of their work, and evaluators would not have to be confined to a "right/wrong" binary grading system.

It would definitely be a challenge for schools to adopt a more holistic assessment process. Tests are cheap to produce and administer (another reason they are so popular), and a portfolio or similar system would take hours and hours of student and teacher investment. But the negative effects of the standardized tests, as laid out by Meier, are simply too huge to ignore. A test like the MCAS, which 70 percent of Fenway High School students failed even though 90 percent of seniors had gone on to college for years before its introduction, cannot be an effective measurement of student learning by any means.

Deborah Meier and Ted Sizer, through their own schools, seem to have found some good alternatives to the traditional testing model. As long as programs and laws like No Child Left Behind remain in their current state, though, the culture of testing will persist. A broader movement away from standardized testing needs to take place for true education reform to take place.