Comments on: Working Backwards to Assessment http://erichoefler.com/2007/02/09/working-backwards-to-assessment/ Notes on education, writing, litracy, and culture Sat, 23 Jan 2010 03:56:20 +0000 http://wordpress.org/?v=2.9.2 hourly 1 By: Doug Noon http://erichoefler.com/2007/02/09/working-backwards-to-assessment/comment-page-1/#comment-77 Doug Noon Sat, 10 Feb 2007 05:49:55 +0000 http://edublog.erichoefler.com/?p=49#comment-77 I've mentioned elsewhere recently that I had the opportunity to spend a day writing test items for our state's new science assessment. A group of teachers from various parts of the state, and from both secondary and elementary levels, worked in grade-level teams with representatives of a large test development company. It's an industry - test item writing. They hire retired teachers and psychometricians to do this work. They wanted us to help them understand the regional culture so that they could write questions that would minimize bias. Not easy to do (maybe impossible for a state the size of Alaska - with hundreds of geographically isolated and culturally distinct communities.) I asked, Did you know that many communities don't have <em>trees</em>? Or that they hunt entirely different animals? They hadn't thought of that. Many of the high school science teachers were dissatisfied with the multiple choice options for the more complex concepts. A couple of things that stand out for me: The test company folks agreed that multiple choice questions were not ideal, but those were the best we could do with a test that would be administered on a large scale. One of the "experts" also said, This isn't a reading test. Yet it was a paper and pencil test that we were developing. You are right that we invest a lot of faith in technologies that are far from perfect, and which may in fact be misleading. I’ve mentioned elsewhere recently that I had the opportunity to spend a day writing test items for our state’s new science assessment. A group of teachers from various parts of the state, and from both secondary and elementary levels, worked in grade-level teams with representatives of a large test development company. It’s an industry – test item writing. They hire retired teachers and psychometricians to do this work.

They wanted us to help them understand the regional culture so that they could write questions that would minimize bias. Not easy to do (maybe impossible for a state the size of Alaska – with hundreds of geographically isolated and culturally distinct communities.) I asked, Did you know that many communities don’t have trees? Or that they hunt entirely different animals? They hadn’t thought of that.

Many of the high school science teachers were dissatisfied with the multiple choice options for the more complex concepts. A couple of things that stand out for me: The test company folks agreed that multiple choice questions were not ideal, but those were the best we could do with a test that would be administered on a large scale. One of the “experts” also said, This isn’t a reading test. Yet it was a paper and pencil test that we were developing.

You are right that we invest a lot of faith in technologies that are far from perfect, and which may in fact be misleading.

]]>