I’ve written about Reviewer 3 before (here, here, here, and here). Somehow the third reviewer has come to embody the capriciousness (and sometimes meanness) of the anonymous peer review process. Note that I believe in the peer review process, but am a realist about what it means and what it accomplishes. It doesn’t mean that every paper passing peer review is perfect and it doesn’t mean that every peer reviewer is doing a great job of reviewing.
When I’m a reviewer I see the peer review process through the lens of the line from Spiderman (Stan Lee), “with great power comes great responsibility”. I strive to put as much effort in to each paper I review as I would expect and want from the reviewers who review my papers. Sometimes that means that I don’t get my reviews back exactly on time- but better that than a crappy, half-thought-through review. I’m not sure that I always succeed. Sometimes I think that I may have missed points made by the authors, or I may have the wrong idea about an approach or result. However, if I’ve done a good job of trying to get it right the peer review process is working.
It’s that time of year. You’ve been invited to a costume party and now must decide: what will you dress up as? Well, I’ve assembled some of the BEST Halloween costume ideas for overworked, stressed out, over thinking academics right here. If you have further suggestions please Tweet them with the hashtag #academiccostumes.
Ugck-ptha, et al. report the development of “fire”, a hot, dangerous, yellow effect that is caused by repeatedly knocking two stones together. They claim that the collision of the stones causes a small sky-anger that is used to seed grass and small sticks with the fire. This then grows quickly and requires larger sticks to maintain. The fire can be maintained in this state indefinitely, provided that there are fresh sticks. They state that this will revolutionize the consumption of food, defenses against dangerous animals, and even provide light to our caves.
Reviewer 1: Urgh! Fire good. Make good meat.
Reviewer 2: Fire ouch. Pretty. Nice fire. Good fire.
Reviewer 3: An interesting finding to be sure. However, I am highly skeptical of the novelty of this “discovery” as Grok, et al. reported the finding that two stones knocked together could produce sky-anger five summers ago (I note that this seminal work was not mentioned by Ugck-ptha, et al. in their presentation). This seems, at best, to be a modest advancement on his previous work. Also, sky-anger occurs naturally during great storm times- why would we need to create it ourselves? I feel that fire would not be of significant interest to our tribe. Possibly this finding would be more suitable if presented to the smaller Krogth clan across the long river?
Additional concerns are listed here.
The results should be repeated using alternate methods of creating sky-anger besides stones. Possibly animal skulls, goat wool, or sweet berries would work better?
The dangers with the unregulated expansion of fire are particularly disturbing and do not seem to be considered by Ugck-ptha, et al. in the slightest. It appears that this study has had no ethical review by tribe elders.
The color of this fire is jarring. Perhaps trying something that is more soothing, such as blue or green, would improve the utility of this fire?
The significance of this finding seems marginal. Though it does indeed yield blackened meat that is hot to the touch, no one eats this kind of meat.
There were also numerous errors in the presentation. Ugck-ptha, et al. repeatedly referred to sky-anger as “fiery sky light”, the color of the stones used was not described at all, “ugg-umph” was used more than twenty times during the presentation, and “clovey grass” was never clearly defined.
Sometimes, getting reviews back on a paper feels a bit like this. I’ve actually had this happen, reading through reviewer 1 and 2’s comments and feeling pretty good. Then scrolling down to find the last reviewer has totally chewed it up. Surprise!
Of course, reviewer 3 is (most of the time) not an actual person/reviewer position- but rather represents the bad, unfair, or just plain wrong-headed reviews that we frequently get on papers and grants. Sometimes the part of reviewer 3 is played by the editor too. And sometimes reviewer 3 is actually right.