Q: “How do I write a good review for a journal? I don’t want to use the reviews I have received as an author because many are completely unstructured. ”
Author: Rob Warren, University of Minnesota
Current Editor of Sociology of Education
Everybody hates bad article reviews. Editors, authors, even the other reviewers. In this post I describe what I think makes a good article review, and I offer a few practical pointers for beginners.
Remember the manifest purpose. There are lots of good reasons to agree to review an article—to find out about state of the art research; to contribute to the collective good; to help shape the area, etc.—but don’t lose sight of the fact that what you are mainly doing is making a recommendation to the Editor. Your review is, first and foremost, a letter to the Editor in which you make a recommendation about what decision your think she or he should make.
Whatever else you put in your review, be sure that you are very clear in justifying your recommendation. And, make sure that the specific recommendation you make to the Editor (i.e., accept, reject, invite a revision and re-submission) matches the tone and content of the review. Imagine a movie review that gives a new film 2 out of 5 stars—but does not explain what’s wrong with the film. Or, imagine a movie review that gives a new film 5 out of 5 stars but then goes on to trash the plot, the script, the acting, and the special effects. Neither movie review would be very helpful. The Editor has to make a decision based on your recommendation. Telling the Editor to “accept” the paper and then writing two pages about flaws in the research design puts the Editor in a tough spot. So does recommending that the Editor “reject” the paper when your review contains little by way of critique.
Remember the “Golden Rule.” The manifest purpose of an article review is to provide a recommendation to the Editor about what to decide about a manuscript. But we all know that the peer review process can be a major avenue for improving our research. Reviewers have read things we haven’t read, they have theoretical perspectives that may be useful to us, they know about data or measures we don’t already know about, they are familiar with new methods we haven’t heard of, and they may interpret our results in surprising and constructive new ways. As an article reviewer, you can make a huge difference in helping someone to improve their research—and thus the state of knowledge in the field.
As an author, you want reviewers to provide lots of thoughtful, constructive feedback based on a close and engaged reading of your manuscript. Don’t be a free rider! Be the reviewer you wish would read your next submission. Take the time to be helpful. Mention the new book or article that the author might not know about (and provide the citation). Carefully explain why the author’s methodology is in need of improvement. Engage in the theoretical discussion. Challenge the author to think things through more carefully. Etc. Show the author the respect you would like to receive.
Timeliness is better than godliness. A Pulitzer Prize-worthy review is useless if you send it in weeks (or months) after the journal’s deadline. When you are invited to review an article, the invitation will usually say specifically how quickly the Editor hopes to receive your evaluation. If you know you cannot stick to that time frame, then decline the invitation to review—and provide several possible alternate reviewers. If you agree to review the manuscript, do it by the deadline. Again, be the reviewer you wish would read your next manuscript. Return reviews as quickly as you wish everyone else would.
Structure your review strategically. A long, meandering, unstructured review is not very helpful. Be clear, concise, and to the point. Emphasize points by putting them first. Here is how I wish every review were structured:
- Start by summarize the author’s research question, theoretical perspective, data, research design, and major findings. Do it in just a few sentences, and don’t make any evaluative statements. This tells the author you read the paper and understood what they were trying to say. (Length: 1 short paragraph)
- Provide an overview of your evaluation of the manuscript. Many Editors will say not to include your specific recommendation, but others disagree. Say, overall, what you think of the research and the manuscript. (Length: 1 very short paragraph)
- Say what you like about the manuscript. This is helpful to the Editor, but it also cushions the blow for the author since everything after this section will be more negative. (Length: 1 short paragraph)
- Describe a few major weakness or limitations—and in declining order of importance. If there is fatal flaw, lead with that. Don’t list problems in the order they occurred to you. List them in the order of how serious they are. (Length: Depends. Maybe a page?)
- Describe a few less serious problems or issues. These are things that are not “make or break” issues, but that you’d like to mention by way of being helpful and constructive. (Length: Depends. Maybe a long paragraph?)
If you add it all up, that comes to about two pages. However, the Editor and author will know after less than a page whether (a) you read/understood the article; (b) what you basically think of it; and (c) what you see as its major strengths and weaknesses.
Don’t pretend to be expert on everything. If you know a lot about the substantive area or empirical topic, but nothing about the methodology, that’s OK. Likewise, if you know a lot about the methodology but nothing about the theoretical framework or empirical subject, that’s OK. Comment on those areas about which you are expert. Trust that the Editor will enlist other reviewers to comment on other aspects of the manuscript.
It’s not about you. Evaluate the paper in front of you, not the one you wish was in front of you—or the one you would like to write.
Don’t overdo it. Unless there is some exceptional circumstance, I do not think people should spend more than about 4 hours on an article review. I recommend taking two hours to read the manuscript and take notes, and then another two hours to formulate and draft the review. Or less. If you let it take longer, you will resent it—and thus be more likely to decline to review the next manuscript sent to you. On average, you should expect to review three times as many manuscripts as you submit. Make sure you review manuscripts well, but in a sustainable way.
They have people for that. You do not need to comment on grammar, spelling, or smaller matters of style (unless the manuscript is so badly written that you feel compelled to make it a core issue). Even in this low budget era, journals usually have copy editors. You also do not need to tell authors how to format their references, tables, or figures. Most journals contract with someone to do those things. Spend your precious time and energy on the central task at hand: Making a recommendation to the Editor and providing constructive feedback that will help the author improve their research.