This week I am working on revising a small paper that was driven by collaborative research and mainly the work of an undergrad. It's a decent paper but nothing too exciting, and as such, we sent it to a small specialty journal. There are stories that belong in these journals and getting it out and published in some form is always better than letting it languish on a desk.
When we got the reviews back it was a classic case of Reviewer 3 syndrome (the video for which, unfortunately, appears to have been taken down). Reviewers 1 & 2 had a few minor and helpful comments, but thought the paper was fine. Reviewer 3, OTOH, went apoplectic on the thing. R3 left dozens of comments throughout the manuscript, most of which with garden variety Devil's advocate shit that can not be answered and isn't the focus of the paper. On two occassion R3 pointed out one sentence and stated "This point would make the basis of a better, more scientific paper." More scientific? WTF?
I started thinking about the times I have had this happen to me, when one reviewer 1) clearly has too much time on their hands, 2) gets themselves worked up over minor issues to the point that they think the world will end if your minor paper in a tiny journal were to be published, and 3) it's clear by the end that they went through the thing a couple of extra times just looking for anything to shit on. I came to realization that this has only happened in the small journals I have sent papers to*. Interesting.
My experience in terms of publishing in a variety of journals is pretty broad, from the little mags to big ones, and the more I thought about it the more this pattern emerged to me. I have certainly gotten bad reviews from big journals, but in those cases the message is usually "I don't like this for these big reasons", end of story. With the small journals I have on several occasions had to write lengthy responses to a mountain of minutiae, including a vicious attack on writing style. Not whether it is written in a "science" way, but just the writing style I use that the reviewer somehow found offensive to their very core ideals. Where does this mindset come from?
Is it the reviewers that will review for these journals? Maybe, but I review for some of them and have never had the urge (or time) to analyze a manuscript in this manner. Is it the editors of smaller journals having a full time job on top of editing? Possibly. Is it the level of work that gets sent to these journals? I don't know what the reasoning, but getting things published in smaller journals seems to take far more of my effort than the stories that are better suited for larger journals, making it far less appealing to get undergraduate-driven research published.
*Fully acknowledging that this can and does happen in some proportion in all journals.
2 days ago
I wonder if this occurs because it's easier to reject things in the big journals. Saying "I don't like this" or worse "this belongs in a more specialized journal" is enough to get something thrown out of a GlamourZine. But in order to get something rejected from a smaller journal might require attacks that are hard to defend against.
ReplyDeleteAn interesting prediction of my hypothesis is that the bigger the name on the paper, the more likely one will see such undefendable-against attacks in the big name journals as well.
(Not that I would ever do anything like that. If I don't think something should be published, I say so. As in "this is so wrong it would damage the literature and should not be published".)
Writing style does matter. A well written paper serves the science in a palatable manner. Even a grammatically correct piece of writing can be hard to read. A poorly written manuscript requires studying before the science behind it can be reached. If a person has to re-read something several times, or struggle to comprehend what the author is trying to get across, they are not going to be sympathetic of the science once they find it.
ReplyDeleteIt might well be the reviewers too. The over enthusiasm and inexperience of a newbie can make a bad reviewer. The bitterness and exhaustion of a failing PI can make for a bad reviewer. All you can do is make your manuscript as pleasant to read as possible.
The less time a person has to spend trying to understand the science, the more likely they are to view the science favorably. A pissed of person is more likely to take the extra time to return the favor. Get a student to read the manuscript, and then have them tell you what they understood. If you can get an undergraduate to appreciate the science then it is well written.
It is not all about the quality of the science, it is about how accessible that science is to a busy reader.
Can you change a few sentences, resubmit, and hope douchenozzle #3 doesn't get assigned to it again?
ReplyDeleteI lean towards the comments of qaz. when you are at the society journal level, IF of 1-3 in biomedical journals, it can be very difficult to actually keep something from being published. low threshold. so some reviewers may feel it necessary to go way over the top. (whether justified is a different issue)
ReplyDeletebut I'll also submit that the tendency for GlamourMagz to ignore *meaningful* so called "minutia" is a problem too.
http://en.wikipedia.org/wiki/Constantin_Film
ReplyDeleteVarious scenes from Constantin Film's movie, Downfall, about the final days of Hitler, had been used for over two years as the basis for hundreds of internet video parodies of Hitler's rants, featuring subjects as different as Michael Jackson's death and Scott Brown's election. Citing copyright infringment, Constantin Film requested that YouTube, the main venue for these parodies, remove them from its site. YouTube acquiesced to this request, and on 19 April 2010, began to remove the Hitler 'Downfall' Parodies.
Tragic....don't they realize how many people went back and watched the actual movie because of these parodies? short sighted...
We're chatting with the editor and I think she's going to give us a pass on most of the "out there" comments. Fixing what the other two reviewers had issue with will only take an hour or two.
ReplyDeleteCertainly the type of reviewing required for the different levels of journals can be considerably different. And yes, the bigger the project the more consequences some of the "smaller" aspects can have.
It is tragic that they took those clips down. What could possibly better publicity for your movie than viral circulation of certain clips, even (esp?) if they are hilariously edited?
I have to confess to having done this the first couple of times I reviewed independently, but only with papers that I thought really needed the help; only realizing later that they probably didn't want my help and just wanted it to be in press.
ReplyDeleteOf course I realized this after getting my own papers reviewed and thinking, "why can't they just accept it???!!!"
I've since learned to make it clear which comments are meant to be constructive but optional, and which are meant to be "requirements for publication". In my naivete, I thought that distinction was the editor's job (!).
I still sometimes find it difficult to draw the line when papers are desperately unfinished or otherwise awful. Then I'm just insulted that it even got sent out - it's a waste of my time.
While my papers, published in Journal of Advisor's Desk, are 1000x better than that, and I can't even submit them. Seems unfair to me. And I'm less inclined to be nice when I feel insulted.
Still, I somehow doubt anything you'd write would be remotely that bad, however. Based on your blogging, my guess is that your scientific writing is orders of magnitude above the kinds of papers I've had the honor to review.
Still, I somehow doubt anything you'd write would be remotely that bad, however. Based on your blogging, my guess is that your scientific writing is orders of magnitude above the kinds of papers I've had the honor to review.
ReplyDeleteCan someone please contact Ms. PhD and tell her that her identity has been stolen and the perpetrator is leaving comments on people's blogs that might, in some societies, be seen as compliments.
Oh, wait, never mind. Phew.
I can see a few possible causes.
ReplyDeleteFirst, authors usually send lesser quality papers to minor journals and save their great work for higher profile journals.
Second, minor journals can't recruit as experienced reviewers. Experience makes you spend less time on reviews, and goes along with being more busy with other things.
I don't know about your field, but we often get grad students reviewing things. If you really want something torn apart, hand it to a third-year grad student. We used to have second-year grad studnets review each other's papers in seminars, and it was brutal.
Third, the more specialized a reviewer is and the closer the submission is to their work, the more picky they are. Whenever I've been the odd negative reviewer out, it was because it was a paper closer to my specialty that I picked apart more thoroughly.