History News Network puts current events into historical perspective. Subscribe to our newsletter for new perspectives on the ways history continues to resonate in the present. Explore our archive of thousands of original op-eds and curated stories from around the web. Join us to learn more about the past, now.
Market Prices and Peer review: Isomorphic Processes
by Liberty and Power
The discussions we have had on peer review are particularly interesting to a libertarian blog because as a coordinating mechanism, peer review is akin to the price system in the market. That is, both science and markets are emergent orders where information vastly more complex and uncertain than any human being can hope to master is coordinated in ways making it useful to unknown others pursuing unknown purposes.
Prices enable coordination among amounts of information that would cause a barter system to break down. They make the market possible beyond the scale of a local fair. Peer review makes it possible for scientists to link together scientific knowledge across disciplines even though no scientist knows very much about the whole. One measures instrumental value – that is, what other values I can gain or must give up in order to acquire something else on a common standard of prices. Science relies on reputation rather than prices, and peer review is a means for measuring reputation among scientists.
This list has discussed at length the weaknesses in the peer review system. I agree with these criticisms. Prices also have weaknesses. There are at least two. They do not give us an adequate representation for non-instrumental values. Most obviously, there were market prices for slaves back when slaves could be bought and sold. They obviously did not adequately reflect the intrinsic value of being a human being. Something crucial was lost.
Second, prices do not adequately reflect positive and negative externalities, both of which were they adequately represented might substantially change the kinds of things produced and ways of producing them. Even when externalities can be internalized via property rights, the property rights may themselves be defective (as were property rights in human beings).
But these shortcomings are not arguments against prices or peer review as such – just cautions not to worship them. In both cases they reflect confidence in certain values that can change, often unexpectedly. They are the best we have for the jobs they do, but they are not perfect. This post was inspired by this science blog I discovered today. http://scienceblogs.com/insolence/2007/05/cranks_against_peer_review_1.php
The "Sense About Science" site (see PDF here reports...
"There are around 21,000 scholarly
and scientific journals that use the
peer-review system. A high proportion
of these are scientific, technical or
medical journals, publishing over
1 million research papers each year."
So this is not just a system that has "survived" the market test, it has been, "widely endorsed" by it's particular marketplace.
I wouldn't go so far as to say peer or review of publications was essential or obligatory for the scientific process in every instance. Non-peer reviewed material isn't inherently disqualified from the scientific process, but it is generally rare.
Classical liberals all told are net beneficiaries of the peer review process, despite criticisms of the process from some AGW skeptic classical liberals.
The modern media and public policy arena is awash with scientific claims and reports, often used to justify the latest health, nutritional, medical or environmental fad, fashion or movement. Sometimes these stories are pushed by scientists themselves attempting to pre-empt the review process. These 'scares' are often used to sell or provide publicity for lobby groups seeking piecemeal state intervention.
The Peer review provides an mechanism to deflate a large proportion of these 'scare' stories. Although I have no survey evidence, my guess is that the vast majority of scare reports in the media have not been through either peer review or independent verification. Peer review thus provides a brake on the apparently insatiable appetite of activists for government intervention.
Still this test cuts both ways. and as both firefighters and scientists know, not all alarms are false. This brings us back to AGW.
Tim Sydney -
5/20/2007
The UK based site Sense About Science has as it's objective improving the quality of public and media debate on scientific issues by increasing awareness of the role of peer review etc.
Of course, there are two kinds of peer review and in debates such as this it is easy to confuse them. Peer review governing the publication of scientific papers and peer review of research grant applications. It would seem to me that classical liberals per se should have no problems with the first kind of peer review.
The second kind is more problematic and I can imagine some legitimate libertarian concerns about it. Presumably free marketeers have doubts about the existence of government research grant programs as such, regardless of whether they are governed by a peer reviewed evaluation and allocation system.
Still I think a peer reviewed grant governance is a superior mechanism (more classical liberal if you'd prefer) than government agency executives and political appointees carving up the pie. That being said peer reviewed processes governing grant allocation presumably are open to some of "public choice" criticisms that have been raised.
This reminds me of an old story about the 1950s libertarian Frank Chodorov who when confronted with McCarthyite claims of communist infiltration of government, responded that the right way to get rid of communists from government jobs was to abolish the jobs.
I think the same should be emphasised in public choice in science arguments. There is a risk that the critiques of peer review may merely provide ammunition for greater executive control over the allocation of science funds, the result would be non-scientists political hacks determining where public science funds are spent.
Gus diZerega -
5/18/2007
While I think Tim’s basic point is very well made, I have one quibble. Peer review is not a substitute for the scientific process. Nor is it really under present conditions “simply” optional. , It is an integral part of the scientific process. It is one response by scientists to the increasing scale and specialization of their discipline as a whole. I think my earlier analogy of peer review as performing for science many of the functions money performs in a market is important. We can have a realm of voluntary trade based on barter but issues of scale and complexity keep it from becoming too developed because of information problems. Money simplifies the information people need to know to make transactions that, base don what they do know, are advantageous.
Peer review is optional in the sense that money is optional. I can get stuff through exchanges where I barter goods, trade labor, and so on. Sometimes I prefer this, but as a general rule I do not. Money is formally optional, but in practice you can hardly do without it. For a scientist the same holds for peer review. This is particularly the case if you want to participate in the community of scientific research rather than be off by yourself.
Referees, to continue the analogy, are akin to investment bankers. A scientist (entrepreneur) seeks resources he does not have to promote a new finding/theory/discovery (product). For the scientist the resources are publicity among a select audience. Journals are specialized means for reaching those audiences. The referee judges whether or not the submitted work (proposal) is likely to be a valid contribution (profitable).
By specializing in providing special resources important to scientists (entrepreneurs) journals with their referee system (investment bankers) dramatically lower the cost of finding a wider audience (manufacturing the product). But in both cases the power of the judgment of a single individual (referee or investment banker) is enhanced for good or bad.
This point also means that it is not the referee system as such but the role it performs in the process that is truly vital. If some alternative could be developed that lowered the costs of reaching a targeted audience while still keeping it from being flooded with too much information, most scientists would probably immediately go for it, as would most entrepreneurs.
Understanding science and markets as both emergent orders helps illuminate them both.
Tim Sydney -
5/18/2007
I don't disagree with William or Guy. Peer review is certainly not a substitute for the scientific process. It is a prominent although, as William points out, optional part of the process.
It seems to have survived "the market test". Scientists are not obliged to use it but the vast majority do, despite it's flaws.
I think we can focus too much on the producers of scientific papers versus their consumers. They face different incentives. The consumers are mostly other scientists who use peer review to increase their confidence level in any particular paper.
It's like a warranty or well recognised brand in the product marketplace. It's not foolproof but for the consumer it's better to have these on your side than not.
There are lots of institutions that develop in the marketplace (or the 'kosmos') that are imperfect but are well supported as they deliver. This analogy is a little rough but I will try to flesh it out anyway.
For example if I want to run around with friends throwing a football in the backyard I can. There is no law that stops me. Same goes if I want to watch peope throwing a football. I don't need the NFL. I don't need any of the established sporting clubs or competitions. So why do they exist?
Well it's true that major sporting organisations often get a better deal in access to public parks and government subsidies for building stadia and sporting facilities. But it would probably be an exaggeration to say these sporting organisations are just creatures of government intervention. They have a marketplace logic behind them.
If I want to pursue a football career or even play amateur football at a more serious level, it is probably best if I join some kind of organised competitive club or team structure. That benchmarks my performance and provides the "network effects" of being part of that larger structure. And it's easier to work through an existing structure of this type than start a rival one. There is a kind of economy of scale at work.
There is an administrative cost to these sports bodies and the administrative hierarchy is not always sound. There is often a lot of politics between and within these bodies. Still if I don't like the way the competition is organised I am free to establish my own competition. Not many dissidents do.
Producers and consumers of scientific papers who feel the established journals do not serve them well, are free to establish their own journals or engage in self publishing if they want. For example, the Journal of Libertarian Studies was established because many libertarian scholars believed their viewpoints were not being well served in mainstream economics journals.
The fact that most scientists don't start their own journals etc. would seem to indicate that most journals etc do seem to serve their 'marketplace' well. Not all people who buck this system are crazy cranks nor are they all brilliant thinkers, like Tom Gold. And not all the brilliant thinkers are willing to put the managerial / administrative hard yards in to develop their own journals etc.
As the web and other technologies evolve we will probably see new forms of peer review or alternative mechanisms develop, but it's still early days yet. The existing system will probably be here for decades yet mainly because it serves it consumers better than the alternatives.
Gus diZerega -
5/17/2007
Tim Sydney is right and I am not sure that William Stepp’s comments really address his points. In fact, I think I can show they say almost nothing at all. He writes
“Peer review is just a pre-publication editorial process. It's of short duration and involves only a few peers.”
BUYING A COMMODITY IS OF SHORT DURATION AND INVOLVES ONLY TWO PARTIES.
“Scientific criticism is an on going process that can last forever and can involve anyone.”
THE MARKET PROCESS IS AN ONGOING PROCESS THAT CAN LAST FOREVER AND CAN INVOLVE ANYONE.
“Which one is more important for the growth of the spontaneous order known as science?”
I THINK I HAVE SHOWN THE LACK OF COHERENCE IN THIS QUESTION. His second “option” simply rephrases the observation that science is a spontaneous order.
We can now return to the first sentence. Peer review is not the scientific process, it is now a crucial part of the scientific process. Markets can also exist without money, by barter. Money is now a crucial part of the market process.
Gus diZerega -
5/17/2007
Slaves were bought and sold under the market and the fact that that system of property rights did not fit either your or my moral beliefs is utterly irrelevant. The market arises when willing sellers interact with willing buyers. Period.
The market order (exchanges on a vast and anonymous scale is impersonal, as in the stock market. That is one way it differs from a craft faire. When we use an intermediary such as money to facilitate exchanges we simplify the information people need to engage in mutually satisfactory transactions. But all simplification leaves something out, by definition.
There is no guarantee that the information left out will be unimportant to one or the other parties – if they had known it. Mises argues this is an advantage - it minimizes the depth of agreement we need to cooperate. He is right. But the same quality can also mean that by cooperating with someone on one level I am undermining other values I support
Your weasel word is "justly." You thereby equate markets with some preferred variant of libertarian ideology. Presumably we have yet to “really” experience markets since almost all property arises from land and almost all land was at one time or other stolen.
I may believe certain products are immoral and you may believe others are immoral, and that is irrelevant to their price as established between willing suppliers and buyers.
No system of property rights as used in any real world market has 100% agreement as to what is just and what is unjust. You may think the passage of time and generations ends the issue. That is self-servingly convenient - talk to a Mohawk or a Nez Perce.
If you think I am blaming the price system for slavery, read again. I am flabbergasted that I could be read that way by a serious reader.
As to externalities, there is NO way of 100% internalizing all positive and negative externalities. Photons, temperatures, molds, germs, and gasses (among other things) cross borders. Some are desirable, some not. The problem is inherent in applying a model based on discrete objects to a world that at certain levels is not practically comprehensible atomistically.
Gus diZerega -
5/17/2007
Sure helps to not do comments while grading finals and keeping an eye on different links. This comment was intended for Sheldon richmen's post, where it has now been located....
'The 'duh' in duhZerega...
William J. Stepp -
5/17/2007
Peer review is just a pre-publication editorial process. It's of short duration and involves only a few peers. Scientific criticism is an on going process that can last forever and can involve anyone.
Which one is more important for the growth of the spontaneous order known as science?
If Mr. X's un peer-reviewed scientific theory is false, then his peers have every right to criticize it (just as they can criticize it if it is peer reviewed). They can even denigrate him by pointing out that his writings haven't been peer reviewed. But that is not the same thing as scientifically demonstrating that his ideas are false.
Tim Sydney -
5/17/2007
I think it was Hayek who argued that there were two forms of order in society. "Taxis", no not cabs, but top down models of organisation. For example, the centrally planned economy, the creationist view of nature etc. Then there is "Cosmos", essentially a self organising system. For example, the market, language (minus the Institute Francais) etc.
So the real question for Hayekians is "is Peer Review a Taxic or a Cosmic order?". Top Down or Bottom Up, that is the question.
I don't think Hayek saw the categories as mutually exclusive. Governments have from time to time tried to foster market economies. China for instance. Even if they didn't really have to. Then there is the Institute Francais. (Is that it's name?) The French government's attempt to steer the course of the French language.
My guess is that peer review is all told a bottom up social institution developed and supported by the scientific community. It's not perfect and indeed flaws are a plenty. Still it has the broad support of the scientific community and they like it because it's a safety net that reduces the risk of big failure, even if dynamic progress is perhaps somewhat slowed. Governments may have played a role in fostering the development but it does seem to have "gone native" .
Gus diZerega -
5/17/2007
Time is scarce, but even so I began reading the paper. But I did not read far. I stopped when I got to the following bull____:
"This state of affairs accounts for many of the decidedly ideological aspects of the debate. Advocacy of the CAGW-thesis translates easily into ‘More power and more money for the political, bureaucratic, and academic elites’. Scepticism of that thesis is more congenial to those who trust in the proven human ability to react adaptively to changes in the environment, if and when they happen. In contrast, the advocates demonstrate a degree of hubris that is unheard of in the whole of human history. They claim that a small political elite has the power to control the mass of humanity for many generations to come, and should use its power to adapt human society proactively to climate changes that they declare to be catastrophic and imminent. Such hubris is enough to make any sane person sceptical about the CAGW-thesis."
I then googled him to see whether he had any scientific expertise at all on the subject in terms of experience doing real science. He has not. He wrote on his website http://users.ugent.be/~frvandun/:
"My central interest is the philosophy of freedom which I approach from the perspective of a philosopher of law. Hence my preoccupation with the theory of natural law (mainly natural law libertarianism), classical liberalism and logic."
His good-guy bad-guy approach and silliness about how government power and scientific budgets were the real motive behind those taking global warming seriously should have been enough to tip me off that he is unqualified to judge the specifics of the debate.
For what it matters, I have written how dealing with the issue need not lead to any increase in taxes or regulatory power. All one need do to meet these criteria is adopt a revenue neutral carbon tax. revenue generated by the tax is deducted from other taxes. There need be no net gain in governmental power.
Apparently anti-global warming ideologues and dead enders seem to think that repeating their mantras over and over again will somehow win arguments. They are wrong.
William J. Stepp -
5/17/2007
A price system is just a network of bilateral exchanges. Under a free market, both parties to a trade have to swap justly owned property titles.
This obviously is not true under slavery, where slaves are not freely selling their labor services.
Why criticize the price system for not reflecting the "intrinsic value" of a human being? The crucial thing that was lost under slavery was a legal system that protected the rights of slaves. Slavery was not consistent with a market society and a spontaneous order.
Slavery was the law of the land under the U.S. Constitution, so that document and the laws of slavery under the thirteen states should get the blame, not the price system.
(The same idea applies to stolen property that is fenced by thieves. They use the price system, but the problem is the failure to protect the property of crime victims.)
The same principle applies to externalities, although they are a lot more complicated. The price system per se is not the problem--after all, negative externalities like pollution are a lot worse under communism. Reason magazine cited a recent study showing that wealthier countries have lower levels of particulate emissions for a given population than poorer countries.
