Don’t judge too fast!

Disclaimer: This is an anonymous blog post submission by Unregistered Submission. We decided to publish it as a valuable contribution to the debate on ‘open science’. However the represented opinion does not necessarily reflect our both/both opinions. To keep with spirit of this post we gave Sophien Kamoun a 24-hours heads up before publishing. Some of his feedback was incorporated into this post by the anonymous author. Slighlty modified paragraphs are highlighted in italic.

There you go, find a duplicated figure panel in an article, make a figure, write one sentence and post it on pubpeer (https://pubpeer.com/), the online journal club where scientific peers can anonymously place comments on scientific publications. Little effort for one person, something that may have a huge effect on people far away from where I am living…

What followed was a tremendously fast response from the authors involved in this manuscript (http://tinyurl.com/lshhv4g) and I think a new world-record in correcting a scientific paper (http://tinyurl.com/mx4dope). I absolutely respect and admire the professionalism by which the authors (Mireille van Damme, Cahid Cakir, Sophien Kamoun et al.) handled the probably very unpleasant situation. I posted my concerns regarding one specific figure of the respective article on Saturday evening, already by Sunday the original data was provided on figshare by the authors to convince any skeptical colleague (http://dx.doi.org/10.6084/m9.figshare.1290790). Let me be clear, I never believed that the authors purposely published data to mislead the reader. Obviously, this was just a simple mistake, something that could happen to anyone actively involved in science.

Why did the authors rush so much to get the original data online so fast? My thought is that the authors wanted to avoid entering a harmful treadmill, in which other anonymous commenters start to dig further trying to add additional “evidence” that the authors purposely misled the reader. In fact, this process almost immediately started after I added my concern. With one person adding some more fuel to the starting fire by talking about the authors purposely rotating another panel in the same figure. What if the authors were unable to provide such a fast response? For example, when data could not be found immediately or someone was on a holiday for two weeks or more? Would the authors have had a fair chance to defend themselves against a growing group of anonymous commenters?

The last couple of weeks I have been following evolving stories around papers of Olivier Voinnet, David Baulcombe et al.,. Doubt about figures in a number of papers were posted on pubpeer in September 2014 (http://retractionwatch.com/?s=voinnet, https://pubpeer.com/search?q=O+Voinnet ). This was followed by an explosion of post refereeing to over 25 papers in January 2015. Worrisome? Yes, but I was a bit shocked by the way colleagues around me spoke with disgust about multiple scientists involved in any of these papers, and how on social media such as pubpeer and retractionwatch people carelessly provide their opinion and accuse scientists potentially involved in figure manipulations and duplications. Probably knowing little to nothing about the factual situation.

To my feeling the pubpeer website in its current form is too much a “hunt the scientist” website, a place where scientists can be suspected of publishing falsified data. Not really the “online community that uses the publication of scientific results as an opening for fruitful discussion among scientists” that it claims or wants to be (https://pubpeer.com/about). Why does a comment that I add myself on a Saturday evening have to appear online in public and to the authors on a Sunday? Why can’t the authors be informed far in advance before making a comment public? Giving the authors ample amount of time to sort things out and reply. The way pubpeer currently works, or better, the way it allows some people to use it, resembles a modern day witch-hunt.

I would like to stress that intentional figure manipulations are indeed extremely bad, in fact it is fraud and that is a very serious crime. This is especially why we should be extremely careful commenting on other scientists work. We should not allow ourselves to create a platform of which the primarily use currently seems to be a public “scientific execution site”. Potentially damaging innocent scientists reputations and that of co-authors who may have nothing or very little to do with the whole situation. To accuse someone from committing a crime is a big thing and I wonder whether this should be discussed so directly and openly in public, with maybe little chances for the authors to reply or defend themselves initially. Do we do the same with other types of crime? We don’t put anonymous notes in supermarkets with names of customers who are suspected of theft, at least not in the country where I live. No, we go to a respectable authority and led them investigate what is actually going on. Shouldn’t pubpeer have a more stringent editorial filter? An online open journal club is something different from an online open crime-report site.

Don’t get me wrong, I am very much pro “open science” and the more discussion the better. Pubpeer is a good initiative, but currently not working optimally. Authors should have a fair chance to defend themselves and one should not judge before all evidence is provided. Also scientists have the right of being “innocent until proven guilty”. In fact, how many of the suspected papers are actually truly worrisome? Yes, for a couple of papers it looks bad, but there also seem to be quite a few with marginal evidence for intentional figure manipulations (http://tinyurl.com/q34yomm, http://tinyurl.com/p3yh9mp, http://tinyurl.com/kyg8ba4, http://tinyurl.com/ojjktz5). Are all these authors and co-authors suspected of fraud or, alternatively, can we accurately point all these cases to one single person?

Pubpeer in its current form is surrounded with a negative and suggestive atmosphere, something you would not like your paper to be associated with. A site where comments seem to be frequently made by over-frustrated scientists. People like sensation: “big names struggling”, always a source of entertainment. Whether these big names are famous movie-stars, politicians or scientists. (Un)fortunately scientists are people too and on pubpeer (and sites like retractionwatch) it is shown that we are often little better then gossip loving yellow press readers in a supermarket.

That brings me to the fact of specifically bringing up the following manuscript/figure on pubpeer. Last week, tweets appeared on social media that were joking about the scientists suspected of fraud (http://tinyurl.com/k8k27yc , sensation!). Did it bring a smile on my face?, honestly yes, it was quite funny. However, would I like to be in the same position as any of the co-authors of the 25+ suspected papers?, and am I 100% sure that all my papers are spotless? My answer would be “No”. Several scientists actively re-tweeted this joke, but isn’t that also a tiny little bit hypocritical? Or are these scientists very sure that their own published work is the gold standard?

I decided to look into a number papers of the re-tweeting scientists present in my literature archive for troublesome data (I was only looking into manuscript main figures, without help of any software). A childish “gotcha game”? Maybe, but I guess that’s how the true wanna-be “science-detectives” on pubpeer work. I discovered one paper of the Kamoun lab with a “serious” issue. Did I suspect fraud?, not for a single moment. But I felt I should bring this up to show how vulnerable we all are as scientist at this moment. The Kamoun lab is without doubt one of the most active labs on social media within the plant sciences field. Something, I actually greatly appreciate and respect! I could thus expect that posting a comment on pubpeer would attract a lot of attention (well it certainly did, http://tinyurl.com/mhbnzlm). Something I hope would aware the community that at this moment we all can too easily be suspected of being a fraud.

Cynical and ironic jokes arise when questions remain unanswered. I understand that the Voinnet lab has had plenty of time to reply to any of the initial concerns posted. This is in stark contrast to the way the Kamoun lab handled the situation, by directly replying to all concerns raised. Nonetheless, with over 25 papers currently in doubt in the case of Voinnet I am aware that ironic jokes currently target a large group of innocent scientist who may not have a fair chance to reply at this very moment.

I would like to open a discussion in which standards of how to criticize a scientific paper are addressed. Should this always be done so open and direct? Shouldn’t websites such as pubpeer have a better editing process for certain type of comments?, especially when issues such as scientific integrity are at stage? At least scientists should be given sufficient time before they are being haunted by a group of “science-detectives”. Lastly, we as scientists should not judge too fast, suspecting or suggesting someone is a fraud is a big leap. It is time we start using pubpeer in a much more positive way by not just posting negative or suggestive comments. To the Kamoun lab: I promise to make a start by now placing my honest, positive and fair opinion on several of your great manuscripts! Pubpeer should be used as online journal club highlighting not only flaws but also the great science out there.

/// See also our second guest post of today by Sophien Kamoun about the same issue. ///

Advertisements

3 thoughts on “Don’t judge too fast!

  1. Pingback: Be positive! From witch hunts to the new reward culture | lushgreengrassatafridayafternoon

  2. “To the Kamoun lab: I promise to make a start by now placing my honest, positive and fair opinion on several of your great manuscripts!” edit: In hope they allow positive comments!

    • Don’t judge too fast! author is partly correct and partly confused. His concern about the impact to the authors is genuine. But he/she would not have to write such a long blog if he/she had written to the author directly rather than blogging.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s