Can Educators Even Answer These Lame Questions?
Related Links
“Ignorance of U.S. History Shown” was the New York Times headline. Less than a quarter of American students were able to name two contributions to U.S. history by Abraham Lincoln and Thomas Jefferson.
A report about last week’s NAEP findings? Try again.
What about this? “Test Shows Knowledge of American History Limited.”
Neither headline was written about the latest NAEP. For the first, we’d have to dust off a test given to college students by the New York Times in 1943. For the second, we’d have to reach back to 1976, our bicentennial year, when the Times ran another survey with Bernard Bailyn's help. Bailyn threw up his hands in disbelief that more students thought the Puritans “guaranteed religious freedom” than not. “I don’t know how to explain it,” he said.
With all due respect to the acclaimed Harvard historian, I do.
Indeed, my powers exceed explanation. I can predict the future. In March 2004, seven years before the release of the last week’s NAEP findings, I wrote the following in the pages Journal of American History:“When the next national assessment rolls around in 2010 do not hold your breath for the headline announcing ‘U.S. Children Score Well on the 100 Most Basic Facts of American History.’ The architecture of modern psychometrics ensures that that will never happen—no matter how good a job we do in the classroom.”
Let me come clean. Here's two tips for predicting the future about students’ historical knowledge.
First, know a bit about modern test construction. No matter what the NAEP spokespeople tell us about the NAEP being “criterion referenced”—or tied to an absolute standard rather than fitted to a bell curve—multiple choice items work by tricking test-takers with “distracters.” An item is “bad” if almost everyone gets it right. So, if during the piloting testing of NAEP, it is determined that most twelfth-graders can identify “The Star-Spangled Banner,” Rosa Parks, the dropping of the bomb on Hiroshima, slavery as a main cause of the Civil War, the purpose of Auschwitz, and Harriet Tubman, these items are all thrown out because they fail to “discriminate” among students.
Second, know how to look beyond the items festooned in McNewspaper. The National Center for Educational Statistics, the group responsible for analyzing the NAEP findings, is a rare gem in government bureaucracy. On their first-class website you will find lists of released items and accompanying sample responses. When you scroll through the lists, you’ll see that kids are tested on information that would make not only Sarah Palin and Michelle Bachman look squeamish, but also members of the NAEP governing board. Can the non-historians among the NAEP governors offer cogent explanations of the Proclamation of 1763, the significance of Shays' Rebellion, or explain the significance of Missouri statehood in the context of secessionism? Give twelfth-graders a break, please.
The release of the NAEP results tells us more about political jockeying than it does about what kids (or adults, for that matter) really know. Listen to some of the responses by the spin doctors. Lee J. White, of the National Coalition for History, said “they have narrowed the curriculum to teach to the test. You can’t expect kids to have great scores in history if they are not being taught history.” But Mr. White, the only ray of light in the latest findings is at the fourth grade, the place where the curriculum was most narrowed by No Child Left Behind. The part of the curriculum largely untouched by NCLB is the eleventh grade. How do you explain that drop?
I will leave this puzzle to the cultural pundits and instead ask us to look beyond the hoopla. Let’s soberly examine the questions we use to test our children. Consider the following item (listed in the NAEP framework for the fourth grade), in which students are asked to read this passage:
Follow the drinkin’ gourd/For the old man is awaitin’ for to carry you to freedom/If you follow the drinkin’ gourd.
Why did African Americans originally sing this song?
a. It helped teach children to read and write.
b. It celebrated American independence.
c. It gave directions about how to escape from slavery.
d. It was written by slaves in Africa.
Forty-two percent of students marked C, which according to NAEP is the correct response. But I have to admit: I would have chosen A from these four flawed alternatives. No slave could have sung the line “For the old man is awaitin’," as it was written in 1947 by Lee Hays, a member of the Weavers. How is it that our national history test uses an interpolated section of a dubious “coded” song—a text that historians have shown to be a fiction of the twentieth century—to quiz kids?
Yes, we'd be better off if kids knew more history. And we'd be better off if adults knew more history, too.