History News Network puts current events into historical perspective. Subscribe to our newsletter for new perspectives on the ways history continues to resonate in the present. Explore our archive of thousands of original op-eds and curated stories from around the web. Join us to learn more about the past, now.
Word from The Volokh Conspiracy and the Chronicle ...
by Cliopatria
There are two matters of interest from The Volokh Conspiracy for historians:
1) Spell-check is revising history, notes the senior Volokh. The dashing ways of Confederate General Robert E. Lee's father, the commander of the Legion of Virginia in the American Revolution, left him with the nickname of"Lighthorse" Harry Lee for two centuries. Apparently the spell-checks of neither Corel WordPerfect nor Microsoft Word recognize"Lighthorse" as a word and correct it to"Lighthouse." Slowly the dashing soldier is becoming known as"Lighthouse" Harry Lee, on a Fairfax County, Virginia, site, two Texasarchives, a legal journal, and elsewhere.
2) Since September 2003, we've been quietly waiting for the National Research Council's report on the reliability of John Lott's research in a 1997 article on the effect of the prevalence of guns on crime. There were other issues that undermined Lott's credibility. As underdog might have noted in the discussion of blogging and gender at Crooked Timber, Lott's use of a sockpuppet, Mary Rosh, puts a different spin on the whole issue of gender and anonymity/pseudonymity. As he did in the case of Michael Bellesiles, The Volokh Conspiracy's Jim Lindgren played a crucial role in outlining the reasons to doubt the integrity of Lott's research claims. Now, says Lindgren's fellow Conspirator, Stewart Benjamin, the NRC's report has been released and it is unfavorable to Lott. It remains to be seen whether the conservative American Enterprise Institute and the Federalist Society will withdraw their sponsorship of his work. Lott's liberal critics have quietly allowed due processes to work in his case. There's been little of the hue and cry that attended the firing of Michael Bellesiles from the Emory University faculty. Tim Lambert, who has been Lott's most persistent critic, responds to the NRC finding at Deltoid.
One would think that there would, if you accumulated all the studies and data sets, be enough data to produce something rather definitive even if it is a null result. Perhaps the NRC study was not a meta-study in that sense, though. But even something which has no effect can be proven to have no effect with enough data.
Richard Henry Morgan -
12/22/2004
In reading my previous posts I see where I did indeed write "nor does it refute it". That was just plain stupid on my part, and at variance with all my other characterizations of the NRC's conclusions. I would add that not only do they not find ANY of the studies under review dispositive, they state they are not sanguine that any study will ever be dispositive.
Richard Henry Morgan -
12/19/2004
Thankyou, Tim, for agreeing with half of what I posted -- that the NRC committee wasn't investigating or pronouncing on the subject of charges of scholarly misconduct concerning Lott. That is not, and never has been within their purview.
I think if you read my posts carefully (including the quotes I provide), you'll see that they report the NRC rejects Lott's findings as non-dispositive, just as they reject all research (they reviewed) in the area of conceal/carry and crime effects as non-dispositive. That hardly seems, therefore, a sufficient basis for institutional discipline visited on Lott. Your charges, if upheld in a due process hearing do seem a sufficient basis. I can't for the life of me understand how that assertion of mine can be miscontrued.
Your second point is certainly valid. You have already won a major battle, of sorts. Lott has travelled from the University of Chicago to Yale to the AEI. Academic institutions like Chicago and Yale have formal procedures for adjudicating charges of scholarly misconduct. I suspect that Lott is at AEI precisely because, as a non-academic institution, they have no such formal procedures (that I know of). I further suspect that Lott won't see ivy-covered walls again, since no academic institution wants to be caught without a chair when the music stops.
Tim Lambert -
12/19/2004
The NAS panel rejected Lott's findings. I quoted their clear statement to that effect on Deltoid. I find it bizarre that you pretend that they took no stand on the matter.
They were not investigating the question of Lott's academic misconduct -- that should properly be done by Lott's employer, but the AEI seems to have no interest in the integrity of its "scholars".
Ralph E. Luker -
12/19/2004
As you know, the NRC report could form the basis of some action by AEI and the Federalist Society, as the outside reviewers report at Emory formed the basis of some action there. Your interpretation of things would, of course, allow AEI and the Federalist Society to take no action whatsoever because they simply will not act on the basis of what Tim Lambert says any more than Emory would have acted on the basis of what Clayton Cramer said.
Richard Henry Morgan -
12/18/2004
The JAH thing is fair enough.
It'd be helpful if you could quote from the NRC where it is at variance from what I said. In fact, it would be most helpful if you could quote where Deltoid is at variance. In the comments section of Deltoid, Lambert says precisely what I've said (the NRC takes no stand on the effect, other than saying that none of the research, which make claims of both a positive and negative effect, is dispositve). He has published many methodological and ethical criticisms of Lott, but the NRC report seems to me quite silent on the ethical issues.
In fact, I'm a hell of a lot tougher on Lott than the NRC -- they draw no ethical implications from the fact that Lott can't back up his original research with its associated data, but instead offers a reconstructed data set. I simply think that if Lott deserves to be horsewhipped for ethical lapses, then it will be on the basis of Lambert's work (and others'), not on the basis of the NRC report. I also am rather sure that your statistical estimate of my mentionings of the JAH is rather off (that's a joke, by the way).
Ralph E. Luker -
12/18/2004
Richard, You take up this matter here at Cliopatria with just about every other comment. My point _was_ that you might _do_ something about it by addressing the _JAH_ directly rather than re-iterating yourself here rather at length and repeatedly. Similarly, your reading of the NRC report is at variance with what you'll find at The Volokh Conspiracy, Deltoid, and elsewhere. How everyone but you could have missed these crucial points, I'm hard pressed to understand. I'd suggest that you go correct the Volokhs, who now have comments activated, and Deltoid, which always has.
Richard Henry Morgan -
12/18/2004
You ask what I would have the JAH do? I would have them conform to established research norms. One such norm is to publish a formal notice of withdrawal of the offending article in the same forum in which it appeared. I don't think that has happened, or is going to happen. If you can point to such notice, I'll gladly shut up on the issue, and even write an apology to the editor of the JAH. Whether the JAH prefers to maintain the awarding of a prize to an article they withdraw, that is up to them (though I think it explains in part why they haven't, to my understanding, withdrawn the article).
Moreover, your question suggests you are unfamiliar with this norm. That is understandable, as I don't think your research has much involved statistical conclusions from data. Stranger still is the fact that a table was published without an "n=..." statement. When I was in grad school, I was was taught that statistical conclusions without such a revelation were useless -- I can't believe my education was so much superior to that of the editors of the JAH, but apparently that is so.
As you said, there are other issues of credibility with Lott -- that is, outside the NRC report. I've quoted from the NRC report, and the quote hardly implicates Lott's integrity. That will have to be addressed in another forum, and might yet cost him. My quibble is that your post allows for the impression that you think the report sufficient to deny Lott a job. That may just be a result of juxtaposition of elements in your post, but I don't think it an equitable recommendation given the quote I offered above.
I have trouble with your characterization of the report. You have it as "the National Research Council's report on the reliability of John Lott's research in a 1997 article on the effect of the prevalence of guns on crime". I think that an unfortunate characterization, open to much misunderstanding, since the report was not restricted to Lott's research, but addressed the whole area of research on guns and crime.
You're right, I'm not familiar with all the issues pertaining to Lott. Nor do I address them, or what should be his fate. I restrict myself to the NRC report, and what you make of it. Interestingly, the NRC report seems generous in its conclusions in some areas (is the NRC committee to be believed?). On p. 121 of the report, the committee says "Lott provides his data and computer programs to all who request them, so it is possible to evaluate his methods and results directly." It then admits that he doesn't have the data set for his original research. That is a no-no by my lights and by research norms -- you, me, Bellesiles, and Lott are equally compelled to withraw publications for which we don't have supporting data.
Interestingly, the NRC doesn't take a stand on the dispute over which coding is correct. And unfortunately, it uses the word 'credibility' in a loose manner -- it applies it to all results that aren't dispositive. By the NRC lexicon then (since it finds none of them dispositive), even the critics of Lott who have offered opposing research, are also not credible. BTW, I have read Deltoid, and I don't find anything there in opposition to what I've written here.
Ralph E. Luker -
12/18/2004
Richard, Why don't you address a letter to the editor of the JAH in re the prize and the article? The editor has, in fact, acknowledged that the major table in the article is mathematically impossible and irresponsible. What more do you want. You want someone to go through every known copy of that issue of the JAH and tear the article out? Hand clip it out of the microfilm? Censor e-copies? This really gets very old, especially when you haven't bothered to familiarize yourself with issues in the Lott affair. Had you done so, you'd know that his data are similarly impossible. Go catch up on your Deltoid.
Richard Henry Morgan -
12/18/2004
I guess my criticism is really two-fold. The NRC doesn't endorse Lott's finding, nor does it refute it, as it relates to the effect of right-to-carry laws. To quote from the report's Chapter 6:
"The committee does not endorse particular findings or consider them to provide better estimates of the effects of right-to-carry laws than do Lott's results. Moreover, the committee recognizes that several independent investigators have used alternative models or data to obtain results that are consistent with Lott's." (p. 127)
The committee also says that other researchers, using other models or data sets have found results that weaken or change the direction of influence. In summary, none of the models and data sets seem to enjoy a privileged position, and changing models or data sets can produce non-robust results. In the end, from the studies reviewed, the NRC committee concludes that no strong causal claim can be sustained, from the literature, in either direction. Translated: none of the studies, either claiming a positive or negative effect, has a dispositive claim to truth. Lott shares that fate with every other researcher on the subject whose work is reviewed by the committee.
Which brings up another topic. The wording of the original post might mislead a few readers. The NRC committee was not convened to investigate the integrity of Lott's research, and was not a due process hearing. Lott still stands accused by others of fabricating a statistical estimate in support of his thesis. He can't deliver the data in support of that estimate, so he should withdraw it (if he hasn't -- I'm not up on the Lott affair). Barring that, his publisher should withdraw the estimate (and any other statistical claim for which he doesn't have the data -- frankly, I'm shocked that publishers don't demand the data upfront, if only to cover themselves). This norm of research, unfortunately, has been weakened by the fact that the JAH continues to thumb its nose at research norms, refuses to withdraw Bellesiles' article, and in fact refuses to even revoke the prize it gave him. I hope this sort of behaviour by the JAH doesn't become accepted, as it will only give refuge to scoundrels.
Ralph E. Luker -
12/18/2004
That is correct. Lott had reported that lawful carry had a decidedly, indeed, overwhelmingly positive effect in reducing crime.
Richard Henry Morgan -
12/18/2004
I should have said no strong conclusion possible from the studies under review (some of which claimed a positive effect, others a negative effect).
Richard Henry Morgan -
12/18/2004
I followed the link to Volokh, and from there to the NRC report. It is heavy reading. It concludes that there is no strong conclusion possible on the causal effects (positive or negative) of right-to-carry laws on a whole range of crimes -- murder, rape, etc. In other words, Lott's conclusions share the same fate in the NRC report as every other researcher who reported a statistically significant positive or negative effect. That's a lot of people who are going to have to be fired, if that is the standard.
Jonathan Dresner -
12/18/2004
You should try it with Japanese, sometime.
My favorite spell-check incident, though was a student who must have done a really sloppy find and replace in addition to the spell-check, stripping out a great many instances of the letter "l", and who therefore refered to Irish peat (the turf used for fuel) as the product of "party decayed pants."
Ralph E. Luker -
12/18/2004
Good point. When I was directing the transcription of Martin Luther King's papers for publication, we had repeatedly to warn student assistants that they were to transcribe the documents, misspellings and all, as he produced them, not as spell check would have them.
Sharon Howard -
12/18/2004
... But this is why historians - whatever your software - should turn off those autocorrect things and spellcheck manually.
Mind you, I like leaving the wavy red lines on just to watch the screen all lit up over the vagaries of seventeenth-century spelling. Like Christmas decorations.
