Here is a link to some excellent reporting from Gary Stern of the Lohud Journal News on the creation of the cut scores for the latest round of testing, about to be released. To all intents and purposes, these cut scores are the way in which the results will support the desired outcome. As it is currently asserted that teachers are inept, have failed to implement the Common Core standards, and therefore, among other things, should have their tenure rights removed and their union protections whittled away in order to be replaced by technology (well, you get my drift) the outcome that only about a third of students met the minimum standards will come as no surprise. The same numbers and tests could just as easily have been calibrated to prove the opposite – if that were the mandate given to the committee members. When will 95 educators evaluate the merit and efficacy of the tests themselves, and the test providers?
Gary Stern wrote in the LoHud Journal yesterday, July 26th, that: “The state Education Department did not want The Journal News to know which 95 educators set cut scores for its new grades 3-8 tests.The department on Jan. 7 denied a Freedom of Information request from the news organization. The response took over 90 days, while state law requires a response in about 30 days.
Nevertheless, eventually 18 of the 95 took part in phone, email or questionnaire responses.
As the series continues today, July 27th 2014, the comments of some of those panelists are revealed - and I quote: “David Dickerson, an associate professor of mathematics at SUNY Cortland who took part, said: "It was a contentious process. I think we came up with something that made us all equally unhappy but that we could live with." But Maria Baldassarre Hopkins, assistant professor of education at Nazareth College in Rochester, said the process was driven by the introduction of outside research about student success. "I question how much flexibility and freedom the committee really had," she said. "The process was based solely on empirical data, on numbers. ... There are ways to make the numbers do what you want them to do." And Eva Demyen, superintendent of the Deer Park district on Long Island, said she still doesn't grasp how the state determined that two-thirds of students were not proficient in English and math. "How they got the 33 percent (passing) was beyond us," she wrote. "It just seemed very strange to me ... and I'm a mathematician!"
Please see the entire article at the link.
Whenever these discussions take place about what makes the difference between each of the numbers – 1-2, 2-3, 3-4, I am transported back to the test marking experience I endured in 2009. Briefly, each school had to send representatives to the gymnasium of the old Bayard Rustin High School on West 18th Street. After 2 days of intensive training, and a set of tests designed to see if we fully understood the rubrics, we were variously retrained, sent home, or ideally, set to work on sealed boxes of test booklets around tables in groups of 6. While the conditions and safeguards were unimpeachable, the challenge of rating these earnest, heartfelt efforts by thousands of precious individuals subjected to a scoring system of a mere 4 points which would determine so much of their future and also that of their teachers and of their schools, was almost unbearable. Without even reading the names, we could hear their voices, their accents, their struggles with English, their individuality (some even addressed us directly! “Perhaps you are asking me to…. But I find it ….” That was unexpected.) As we also had to circulate the booklets to at least one other person, we could see right away who was inexorable, who gave the benefit of the doubt. One young man at my table simply marked everyone 1 unless they were amazing, when they got a 2, because he was sick of low expectations! We asked the supervisor about this, but she said it would get flagged if he seemed to be unfair. How was not explained.
The book to read on this subject is without a doubt: “Making the Grades: My Misadventures in the Standardized Testing Industry”, by Todd Farley. (2009).http://amzn.to/1nP8Uyr
According to the description on amazon.com, ‘The No Child Left Behind Act uses the phrase 'scientifically-based research' more than 100 times when discussing standardized testing, but Making the Grades raises serious questions about the validity of many large-scale assessments simply by describing one man's career in the industry. This first-hand account of life in the testing business is alternately edifying and hilarious.’
If any reasonable person already had doubts about the accuracy of grading the written portions of tests for hundreds of thousands of students in a timely manner, and arriving at a truthful, accurate assessment of their abilities based on this single measure from a single day in the life of a single human being, this book will confirm your worst fears. For anyone who never considered the mechanics of this mammoth undertaking, and perhaps thought only of the Scantron sheets being fed through an infallible machine, this is an absolute must read. There is nothing like the mind-numbing accuracy of a real-life account to bring home the ridiculous reality.
Are we to lean more and more on these shaky, fallible methods, knowing that not only is the implementation flawed, but the entire evaluation process was questioned by the very people who created it? Would we take plans from an architect and go ahead with our building if he told us that all of his calculations were guesses and projections, with no scientific validity, and that he seriously doubted our choice of the site, and the materials for the purpose of the building?
Well yes, that is apparently exactly what we’re doing. And when the whole thing falls of its own weight a year or two from now, please don’t say we didn't see it coming!