Speeding up the journal review process: How economists are doing it

Leave it to our colleagues in economics to find an efficient way to speed up the journal review and publication process. Inside Higher Ed reports that authors whose papers are rejected by the American Economic Review, one of that discipline's most prominent journals, can opt to have the referee reports forwarded to one of four other journals published by the American Economics Association. The second journal can then use those reports (augmented in some cases by referee reports it solicits) to make a relatively quick publication decision.

The advantages of this are obvious: The publication process takes less time, since the second journal starts its editorial process with reports in hand. It might also help to address the problem of journals finding willing and competent referees. If referee R1 already reviewed a paper for the first-choice journal, then the second-choice journal can rely on R1's report rather than spending time identifying new referees.  This might be especially useful for papers on highly specialized topics where the pool of appropriate referees is likely to be small. The process is also entirely voluntary and transparent.

I have to say that I'm attracted to this option. For one thing, I've had several experiences where I had a paper rejected by a highly regarded journal either as a 'close call' or for reasons such as that the journal had recently published a paper on the same topic or because the paper didn''t exactly fit the journal's mission or scope. In those cases, the positive reports I received from the highly regarded journal wouldn't just disappear. They would serve to increase my chances at a less well regarded journal. And it would feel less as if each submission amounts to starting over with a whole new set of referees who may or may not perceive the paper as the previous referee(s) did.

There are some clear challenges in adopting this model in philosophy. Economics journals are more centralized, with the AEA journals playing a more central role in the discipline. Perhaps one of the large publishers, Springer or Brill, say, could try this out. But many of the most important philosophy journals are at university presses that, in many cases, publish few if any other philosophy journals. For example, in order for the Philosophical Review to take part in such an arrangement, it would have to reach an understanding with some other journal at another press.

The other challenge is, well, ego. Implementing this approach would require acknowledging hierarchies of prestige or quality amongst journals, and that may be something journal editors may be loathe to do. I'm not knocking Utilitas when I say that I'd rather publish a paper in Ethics than in Utilitas. But that sort of fact may be too hard for many to publicly countenance.

What do others think of the review process the economists are trying out? Are there other problems or advantages I didn't mention?

Comments

[Jamie Dreier]

I agree with Chris Hitchcock, esp. his point #2.

[Doug Portmore]

Jamie,

I don't see how Chris's point #2 is a real worry if it's up to the author to decide whether the reports get sent on to other journals.

Michael,

Don't you think that editors are going to want to choose their own referees? If I was an editor and I'm forwarded reports from referees R1 and R2 but think that R3 and R4 are better suited to referee the paper, I'm going to ask R3 and R4 to review the paper and then weigh all four referee reports, giving greater weight to the reports from the referees that I trust the most. If editors are inclined to want to choose who referees the papers submitted to their journals, then this is unlikely to speed things up in all but a few cases.

[Jamie]

Doug, I would have thought that lots of editors would be happy to use an already-constructed report if it meant they didn't have to convince new referees, even if the report was not written by their top choice. It's extremely time-consuming to find referees who are willing to write reports. Now, if that's wrong, and the large majority will just pick new refs anyway, then of course the new policy wouldn't have much effect.

I guess allowing authors to veto some reports does help to vitiate one problem, the problem of allowing one very negative report to have too much influence. But then I'd worry that overall the negative reports will have much less influence than the positive ones, which also seems like a bad idea.

[David Sobel]

I would assume that some journals would decide to opt into such a program and others would opt out for the kind of reasons Doug points to. Suppose that is so and suppose it is up to the author whether to send the paper to another of the journals that have opted in using the same ref reports. Assume that the author has to have all the ref reports sent to the new journal (that is, that they can't pick and choose among them).

[I realize now that I am assuming, contrary to what I take to be Jamie's assumption above, that the way it would work is that the author would have a promise that if they choose to send the paper on to a cooperating journal with the existing ref report, that the cooperating journal promises to not seek additional reports. But I suppose each journal could choose its own plan--some might offer the above promise, some might not and only promise to take into account the existing ref reports.]

Given all this leeway for author's and journals to opt in or not and how to opt in, what complaints remain?

One benefit of the plan, insofar as the plan had an effect, would seem to be that less ref reports are needed and so one could work harder on each ref report without adding to the burden on refs generally.

[Mark van Roojen]

I'm with Jamie in worrying about the negatives and in agreeing with Chris on the other thread.

I have a somewhat negative response towards homogeneity that it seems to me this pushes in the direction of. Philosophy is a field in which there is much disagreement, not just about solutions to problems, but also about the importance of issues, what sorts of approaches are fruitful, and so on. I know the proposed system is voluntary, but it seems to me that having two referees determine what happens, not just at one journal but at several, means that the opinions of these two referees on these matters becomes more weighty. I kind of think it is a good thing that different journals, even of the same quality, often disagree about what is worth publishing.

I also think there would be a counterproductive institutionalization of the pecking order.

My main worry is that this might lead to stasis in the quality of journals. If a certain bureaucratic system just stipulated that a particular journal was of the first rank there would be less pressure on it when it slipped. And, similarly, journals which are officially lower down on the pecking order will have less reason and less ability to improve their reputations through good management and editorial guidance.

The economists seemed to rationalize their proposal by suggesting that the most prominent journals were also the most general, so that rejection at a generalist journal would then result in it going somewhere more specialized. It isn't obvious to me that our field is organized in that way. Some top journals publish no history, others do. Ethics is generally thought by people who work in ethics to be as good as the top general journals. So how is this really going to work?

So I'm not that keen on this. But I'm not that keen on lots of proposed solutions to the problems of the publishing racket beyond making sure that refereeing is fully blind, that people meet their deadlines, and that journals actually stay on top of the deadlines for refereeing. If we did those things we'd be in pretty decent shape.

[Ben Leech]

Doesn't a simple variant of the proposal avoid the problem about requiring an ordering on journals? Namely, journal A and journal B have an agreement whereby authors can ask that A reports get sent to B or vice versa.

[Allen Coates]

I think at least some of the problems with the proposal can be solved by tinkering with it. As Ben notes, there's no need to rank the journals, for example.

If you are worried that two referees should not decide the fate of an article, you should welcome this proposal. As things stand, two referees (and an editorial team) can decide the fate of an article (if they recommend acceptance). But on this proposal, you could have three or four referees, for example, look at a paper when it is first submitted. Every paper is then guaranteed to have more than two referees look at it. Yet the total number of referee reports need not increase, since other journals would no longer need to find new referees if the first journal rejects the paper. And the process would still be faster, since the referees would all look at the paper simultaneously. Finally, if there in no hierarchy among the journals, no one journal would be overly burdened with finding referees.

I'm not as worried as Mark is about the number of referees that look at papers. My main point is rather that it might be worth tweaking the proposal a bit to solve problems before deciding against it.