Instructions to Reviewers
A Satirical Commentary
Point out that if the experiment were done differently, different results could have occurred; and as it is therefore, the research is incomplete.
Point out that the author should have answered your question, but avoid specifying a procedure. Give the author the opportunity to resubmit a paper answering your question. After reading it, point out how a re-resubmission could be made even better. It is especially important to let the author know at that point, how your thinking has changed, and what would now make an even better paper because it helps them understand the dynamic nature of science.
Point out that the paper left some interesting questions unresolved and that it is inappropriate for you to wait for the next paper to know the answers. Require that the research be packaged with enough research to make the paper too long. If the paper covers every ramification you can think of, point out that much of the research was unnecessary, derivative, or did not lead to new questions. Require that most of the research be deleted.
Point out that the parts you understood were therefore, by definition, obvious and irrelevant; and that the parts you didn't understand must be made obvious.
Point out that any results that make sense and are expected, are therefore, not contributions to the knowledge base; and that any results that are surprising, are probably therefore artifacts, or the result of confounds. As a minimum, the surprising results should be made to make sense, so that they can be evaluated in the next review process.
Point out that whatever was discussed in paragraph n+1 should have been covered in paragraph n. If that puts paragraph n out of place, it can always be fixed in a re-revision.
There never was a generalization which did not fail somewhere and never a specific which could not be generalized to something. In cases where an author makes a general statement, point out the exception. In cases where the author sticks with the specifics, point out that you are lost in a mire of details. If the author discusses both, then point out that the finding wasn't worth the three pages it took to elaborate the generalities with all the disclaimers. Require that the paper be more focused.
If the author is review hardened enough to know the kinds of comments you will make and heads them off in the original manuscript, point out that the paper has tortured logic, is defensive, condescending, argumentative, and lost in the details.
Your job is to point out how someone could conceivably misunderstand something, not how a reasonable person paying attention to the details provided in the paper would interpret a sentence. To this end, try to read sentences with emphases in unusual places or use alternate definition of a word. When you find one that makes a sentence seem strange, point out that the sentence doesn't make any sense; because after all, it is possible that some reader could actually read it that way. If your objection sounds bombastically silly, emphasize that you yourself did understand what was said; it was only that you are looking out for the less sophisticated reader.
It's usually a good idea to put the author in the proper frame of mind by pointing out sentences which fail to simultaneously inform the best in the field and to communicate to the meanest in the field.
Consider the cost of the two types of errors you could make. If you like the paper and someone else shows the paper to be stupid or in error, you will be revealed to be a fool. If you say the paper is stupid, and in error, and everyone else likes it, you will be assumed to be superior because of your high standards.
If you think the paper may actually be good, and that you are probably in over your head and anything you say may prove you a fool, then simply write "fix" in big letters across the top of the paper.
You don't have to worry about being reversed by a responsible action editor who misunderstands your insightful critical analysis and sees it as simply sophistic, denigrating, unhelpful posturing. That editor failed to be reelected to the board and is no longer with us.
Keep in mind that models of behavior proposed by people you do not admire are coming from people who must be slow of wit, because otherwise, you would admire them. It is important, therefore, for you to save the field from their sort of drivel. If their idea does not account for everything, then reject it out of hand. If their model accounts for many phenomena, then it must be a principle we have known about for a long time, because after all, you are familiar with the phenomenon. If the model comes from an admired researcher, then the idea must be very clever. Don't make a fool of yourself by pointing out when it doesn't fit the data or that it is obvious to a child.
Absolutely never permit an author who is not known to be a good researcher to exercise any of the reserved prerogatives. It will give them a sense of confidence which will make it too easy for them to perform well. That will deny them the opportunity to better themselves as the result of review by their peers. Established researchers add humor to their work, others debase the journal. Established researchers juxtapose terms in order to demand that the reader "grow." Others make weird associations. Established researchers hint at the potential importance of issues and allow the reader to think through ramifications without drudging through brutal details. Others make wild unsupported claims, and make hand waving arguments.
You need not try to understand, nor even tolerate any new ways of doing things if they are offered by anyone who is not an established figure. An established person has good judgment; others could be wrong. It is for this very important reason that you should not attempt to suggest that established researchers function under the same set of rules that are the obvious necessity for others. It is the reader's burden to understand established senior researchers. It is the burden of others to make themselves understood.
Remember, you show your superiority by pointing out mistakes. Keep in mind that any geek can like something and be helpful. It takes a really great researcher to be a reviewer. We are the guardians of quality because we are quality. It is the difference between us and them that proves our worth. Don't worry about the effect on the field, it cannot possibly collapse before we retire.
