Is professional commenting worth the risk?

Journal articles are the basis for the advancement of science and the discussion of novel data and analyses as they are published is one feature that unites all fields. Between journal clubs, informal discussions, conferences, emails, phone calls, Morse code, etc., we all engage in debate over our results and those of others. But other than in the memories of those involved, these discussions are largely "off the record" and invisible to anyone new entering a particular field.

To foster this type of discussion in a more open and accessible format, some journals have added the option to comment on published papers online. Notably, the BMC and PLoS journals were at the forefront of this movement and Nature has recently enabled this as well. There are others, but you get the point - some high profile papers are available for individuals to comment on and for everyone else to see what those people said.

In an ideal world, this is a great option. It allows for the compilation of people's reaction to certain studies in a publicly accessible way. It brings the water cooler to the source and can provide insight to those who are new to the field. Which papers have which potential flaws? Who are the major players in the debate on this work? It could all be right there.

In practice, how does this work? Well, if you've looked through these journals you will probably note that there are not many comments. An occasional paper will spark some debate, but the mechanism is widely underutilized and the community has not really embraced it. Why?

Well, one reason might be the concern about the repercussions of publicly attacking (even nicely) someone's work. Maybe one of the authors doesn't appreciate the "debate" and consciously or unconsciously holds that against you as a reviewer of your paper/proposal/tenure review package down the road. Maybe it's not even an author on the paper, but someone who reads your comment but will never be known to you. Is the benefit of public debate to the community more important than possible repercussions to your personal work?

I bring this up because I came across a paper just the other day that has what I believe to be fundamental flaws in its approach. The paper's conclusions are interesting, but inherently problematic based on the author's assumptions stemming from their apparent lack of familiarity of the pertinent literature. Knowing that the results might get picked up upon, but the assumptions underlying those results might not be obvious to those interested in the final outcome, I considered commenting on the paper. Thus, I asked some colleagues whether it was important to point this out and the reaction ranged from "This is important for science" to "You would be stupid to risk it".

I know there are plenty of studies on people's interest in public literature commenting (I'm just too lazy to look them up right now), but doesn't it just come down to risk? If there is any perception that one's comments could have negative consequences for one's research, what is the motivation to participate? Do science martyrs get tenure?

This is basically the same debate that goes on in the blogosphere between those who prefer to blog under a pseudonym and those who think that any comment or blog without a "real" name associated with it isn't worth a damn. PLoS, for instance, makes it very clear that all comments must be obviously attributable to an identifiable individual (although they with keep your email address secret, because no one can find that out).

I wonder how commenting would be affected by allowing pseudonyms to comment on professional journal articles. Would early career scientists feel emboldened to engage in public "on the record" debate? Would it empower those who refuse to open themselves to any additional risk to their livelihood? Would the debate improve or devolve into a hotbed of uncivil behavior?