Print
E-MailCampbell R. Harvey is Professor of Finance at the Fuqua School of Business at Duke University, and David Hirshleifer is the Paul Merage Chair in Business Growth at the University of California Irvine Paul Merage School of Business. This post is based on their recent paper.
Papers in economics are three times the length that they were in the 1970s. This does not include internet appendices which push the total length of some papers to more than 100 pages. It takes years from submission to publication. We argue that editorial process in many of the social sciences has become dysfunctional. It is time to reset the norms.
Consider the current state of the peer review process. Authors have to deal with multiple rounds of reviews and sometimes the response document to the referees’ comments is longer than the manuscript itself. Some editors impose what we call the Union Heuristic. Under this “hands off” approach, the author has to satisfy all of the referee comments. Hence, papers become bloated with robustness checks, and the review process is unnecessarily extended.
We argue that this procedure has substantial social costs. Referees are often trying to impress editors by writing long reports with many comments not critical to the paper’s message. Authors need to invest considerable time dealing with long lists of comments. The time spent on frivolous robustness checks is time not spent on new research and developing innovative ideas.
The Union Heuristic is a lazy way to edit. The benefit is that the editor needs to devote very little effort. Essentially, the editor delegates decisions to the referees, with a strong tilt toward complete author deference. While this might work some of the time, inevitably there will be cases with disagreements among the referees that eventually must be dealt with. So while such delegation might seem to save time, often it just creates more work in later revisions when the editor is forced to adjudicate among conflicting requests.
At the opposite extreme is what we call the Intersection Heuristic. Under this model, the editor tells the authors to deal with the common points that the referees raise. This heuristic can also be dysfunctional. While it is true that the editor should carefully look at common comments, it would be too mechanical for the editor to always direct the authors to address each common comment. Indeed, these common comments often reflect research fads. Even more importantly, independent comments often contain key information, such as spotting an error. The editor might choose referees with different types of expertise. Even more seriously, the Editor might draw a negligent referee and the lack of overlap simply reflects the deficiency of the review.
So neither heuristic is optimal. Nevertheless, they act as sign-posts that editors can use to measure how papers are typically being treated. Currently the balance is tilted much too heavily toward the Union Heuristic. Attention to the opposite possibility, the Intersection Heuristic, is a useful reminder that often referee suggestions—even, on occasion, good ones—should be discarded.
Going beyond such mechanical sign-posts, it is essential that the editor put enough time into the initial round to decide and make clear to the author what is required to make the paper publishable. This will routinely involve telling the author to ignore certain referee comments and/or comments of the editorial team. Referees and associate editors are providing advice to the editor. Norms and expectations for both referees, editors, and authors need to change accordingly.
Referees need to expect that editors will not adopt all of their suggestions. Even reasonable suggestions should sometimes not be required, because adopting all good suggestions can make the editorial process too unwieldy. Some useful robustness checks and extensions are best left for future research. The editor should not be worried about offending referees/associate editors.
Another key norm that should be established is what we call up or out. This norm rules out the painful multiplication of rounds of review that have become commonplace in economics and related fields. By up or out, we mean the following: if the paper is issued a revise and resubmit, the norm should be that 90% of these papers should be accepted on the first revision. There are few if any speculative revise and resubmits. However, on somewhat rare occasion, the editor might choose to invite what is essentially a new paper.
Of course, asking the editor to carefully sort out various referee suggestions takes time and effort. We also offer some ideas on how editors can prioritize and economize on time. For example, if the paper is clearly below the bar, the editor should not spend much time trying to justify the rejection. If the paper is above the bar, there can be a greater value to improving it, since it will be exposed to readers. But there should be a greater readiness to simply publish acceptable papers rather than investing time trying to turn them into perfect papers. This is especially the case if the paper is above the bar but not path-breaking. The key is that the editor devote the largest amount of time to shepherding the most innovative papers through the publication process.
While there are many industrious and well-intentioned editors, the current peer review process is dysfunctional. In many cases, papers have become bloated with results that do not add to the message of the research. Authors wait for years before their papers are accepted, and delayed sorting and certification also harms readers and the advance of reliable knowledge. Even worse, the current system shifts time and resources to endless robustness checks and variations as opposed to innovation.
To sum up, norms need to change. Editors must take the lead by abandoning the union heuristic, and by establishing up or out as the norm for peer review publishing.
The complete paper is available for download here.
