Q: How do you respond to reviewer comments when re-submitting a revised journal article (particularly if you disagree with a reviewer on something)?
Author: Doug Downey, The Ohio State University
Ten suggestions for handling reviewer comments:
1. Write an angry, profanity-laced response to the editor about the lousy reviews you’ve just received. Then tear this up.
2. Distinguish between good and bad reviewer comments. They can’t all be good—sometimes they contradict each other. Look to the editor’s letter for clues to help identify “good” reviewer comments that represent the central issue.
3. For reviewer comments that are simply off-base (e.g., they misread something), consider clarifying your text in a way that makes it very difficult for another reader to make the same mistake. Even though the reviewer was wrong, the fact that they misunderstood something is noteworthy.
4. Directly engage “good” reviewer comments. This is where decent papers become great. Articulate the issues they raise clearly and address their concerns head on in your manuscript.
5. However, avoid “over revising.” Papers need a central idea or selling point. Make sure yours is clear by maintaining a central narrative.
6. Appease reviewers by adjusting tone. If I write that “the Reproductionist position regarding schools and inequality goes unchallenged,” I risk frustrating a reviewer who knows of some work that does challenge it. I may be able to avoid this problem just by modifying the sentence slightly, “the Reproductionist position regarding schools and inequality goes virtually unchallenged.” Or, I could consider using “largely” and be even safer.
7. For quantitative manuscripts, reviewers sometimes ask for changes in the models. We all know that there are lots of decisions to be made when modeling (how to code variables, how to handle missing data, what covariates to include, how to weight the data, what dependent variables to predict, etc.) and that little adjustments here and there sometimes change the results. Develop manuscripts around results that are robust to these different decisions. If your argument depends heavily on a minor finding that only reaches statistical significance under a subset of modeling decisions, then you are setting yourself up for a difficult R&R. And even if the paper does get published its likelihood of replication is low.
8. If you can’t address a reviewers’ concern because of data limitations then admit it up front and make a case for why your manuscript still has value.
9. A surprisingly large number of the papers I review have a basic problem—they lack a credible opposing view. In other words, the research question lacks any controversy. You can articulate the value of your manuscript most clearly to reviewers by highlighting the groups of scholars who view the issue at hand differently, and how your paper helps adjudicate among them.
10. Consider contacting the editor for further advice. Editors are sometimes willing to provide more insight via e-mail or phone conversations.
Q: Could you give any advice about how to write an effective R&R memo to an editor and reviewers? I want to address all the points made by the Editors and the Reviewers, but the memo is getting very long (and it is very time-consuming to write!) — is that a problem? What would you recommend?
Author: Bill Carbonaro, University of Notre Dame
Doug does a great job of outlining the main features of an effective “R&R Memo” in his post. At a minimum, an effective memo demonstrates that the author has significantly improved his/her paper in response to the reviewers’ critiques and suggestions. After all — that is a major goal of the peer review process!
The question also specifically focuses on the length of the “R&R Memo,” which Doug did not comment on in his post. There is no “standard” length for an “R&R memo”- much depends on the substance of the reviews. I have seen some reviews that are less than two single-spaced pages, and others that are nearly 10 single-spaced pages (with tables!). Some “Memos” are too long, and while others are too short. Here are a few guidelines to help keep your memo to “Goldilocks length” (“Just right!”):
- Economize. It is unnecessary to describe EVERY single change that is made to the manuscript. It is preferable to simply say “I decided to adopt all of the minor changes suggested by Reviewer 2,” rather than describing every minor “bullet point” raised by that reviewer (e.g., “The author should change ‘STATA’ to ‘Stata.’).
- But, Be Thorough. It is unwise to ignore specific comments and suggestions that reviewers make, particularly major ones. There is no better way to tick off a reviewer than to ignore his/her feedback! Good reviewers devote substantial time and thought to their reviews, and they want know that the author took their suggestions/critiques seriously. So – be sure that each reviewer sees that you have read, considered, and responded to the major points they made in their review of your work.
- Prioritize. Most of the text in your “Memo” should be allocated to two main items: (1) justifications for suggested changes by the reviewers/editor that you decided AGAINST making, and (2) descriptions of supplementary analyses that you ultimately decided against including the paper. These are the make-or-break items that will likely determine whether the reviewers recommend to “reject” or “accept” your paper. So – be sure to devote adequate space to these crucial points in your letter.
- Avoid redundancy. Always explain HOW and WHY major sections of your paper have changed, but avoid including the same revised sections of the paper in your letter. For example, it is sufficient to say: “As reviewer 1 requested, I incorporated ‘habitus’ into my theoretical framework to better motivate my research questions (see pp.4-5).” The reviewers will (hopefully!) read both the paper and the letter, so it is unnecessary to make the same argument twice.