EclectEcon

It looked like a routine decision that Steven Levitt had to make. The co-editor of the "Journal of Political Economy" (JPE), who is most renowned for his bestseller "Freakonomics", had to accept or reject a comment written by the Dallas-economist Stan Liebowitz on an article which had been published in the JPE.

Perhaps Levitt should have simply used his far-reaching powers as editor and reject the comment without much ado. He did not do this, He did something instead, which could potentially taint his own good reputation and the reputation of the JPE, and which exemplifies the relatively lax procedural standards at top-flight economics journals.

On closer look, everything about the case was unusual. The comment was phrased very strongly. It was a thinly veiled assertion of data-manipulation. This is quite remarkable, given that the JPE, edited at the University of Chicago where Professor Levitt teaches, is one of the top five economics journals globally.

The subject of the criticised article was a hot topic. The authors, Felix Oberholzer-Gee (Harvard) and Koleman Strumpf (University of Kansas), claimed to prove that file-sharing websites on the internet have not been responsible for the sharp decline in music sales since the turn of the millennium. Music labels might disagree. They have been busy suing file-sharing sites and their users for copyright infringement.

Professor Liebowitz disagreed strongly He had told Stephen much in a letter even before the article was published in the JPE. Liebowitz had pointed to what he considered numerous mistakes and inconsistencies in the paper and complained that the authors would not share their data for replication purposes. Levitt had forwarded the letter to the authors of the study, but had published the study more or less as submitted.

Ignoring Liebowitz was not possible any more, however, after he submitted his counter-study officially as a comment to the JPE in September 2007. Levitt started by asking one of the authors, Koleman Strumpf, for his opinion. Strumpf handed in his reply in November. He defends the study and retaliates by pointing to alleged mistakes in Liebowitz' comment.

In addition, Levitt asked for a report from an impartial referee. The referee recommends publishing the comment in order to "save subsequent researchers from building on a flawed research foundation." While he advises Liebowitz to rephrase his comment such that it would not contain any overt assertions of data manipulation he sides with him on almost all the critical points and comes to a damming conclusion regarding the file-sharing article: "I would suggest that the authors? conclusions are not warranted given the analysis and evidence that they provide."

However, Levitt is not inclined to publish the comment. He anonymizes the reply by Strumpf and uses it as a second referee-report on which he bases his rejection of Liebowitz' comment. "There is no doubt you raise some reasonable points. Nonetheless, I think the negative referee (negative toward the comment, N.H.) is correct in most of what he says", Levitt writes to Liebowitz. The only point he takes up from the impartial referee is the advice to moderate the tone, should Liebowitz wish to submit the comment to some other, lesser journal.