No, we are not having trouble in the lab with data gone missing. What we are having issues with are data reported in glamour pubs that none of a gazillion authors on said paper can seem to locate. If you spent an entire section in your paper (where real estate is expensive) to talk about how you found a whole bunch of data points that you did not expect to find, isn't it a bit fishy that when someone writes you for those data that all of a sudden a game of email hot potato beaks out and the list of cc'd "people who probably have those data" starts to stretch longer and longer? This is not a paper from the last year and if the authors were following up on the outliers, they would have done so already. No, this just seems to be a case of vanishing data.
Damn Good Technician has previously discussed the lack of detail in methods section of glamour pubs and whether or not the caginess around some critical steps is a way to publish the bare minimum while maintaining a competitive advantage over anyone interested in the work, but this is different. In the paper, a big deal is made of the outliers, but less than 5% of those data are discussed specifically. That means that 95% of these interesting data points are never mentioned specifically, only alluded to as a group. Even in the supplementary data. How are people supposed to verify the interesting outliers if they are completely obscured? If it turns out that none of the authors can even produce the data in question, then what does that mean about the results? These data are critical and heavily-cited evidence for several theories. Shouldn't they be easily located? Surely we are not the first lab to inquire about them.
I am hoping that we see some resolution shortly, because if the authors can't provide the data in question I don't know what to do about it, only that it would be a major problem.
1 day ago
We see a lot of this in my field. Can't find the data, can't reproduce the data, etc. Completely disturbing.
ReplyDeleteI'm pretty sure that part of the author agreement for publishing in most Glamour journals (Nature, Science certainly) includes an agreement to furnish raw data on request. You should contact the editors.
ReplyDeleteHow old are these data? Has the effect been replicated?
huh, that sounds a bit strange imho. I mean, I could show you all my raw data. You might not make sense of it - without key and the experimental set up but really, it shouldn't be too hard to interpret. Once you write the publication data has to be available, if not - something smells like smoke...
ReplyDeleteIt sounds a bit like the "I would anticipate that this would happen" turing into "I know that this would happen (if we did the experiment)" into "we saw this happening" - like the whisper game when people are in a circle and the first comment is never like after it's been repeated after 12 people....
Damn. I wonder what the journal would do if you contacted the editors & said "The authors of this paper cannot produce the raw data for these figures." I have a sneaking suspicion that shoulders would be shrugged and you'd be in pretty much the same spot you are now. Perhaps my pessimism is unwarranted, but I kinda doubt it.
ReplyDeleteMy guess is that this is a project with so many fingers in the pie that it's not easy to track down the raw data for certain things. The problem is that the results can't easily be verified because of the way the specific methods are glossed over in the paper. I expected that the details would be in the supp. mat. but no luck. Still waiting to hear back. The conclusions of the paper don't hinge on these data, but they are important to a large sub-field, nevertheless.
ReplyDelete