Don’t judge too fast!

Disclaimer: This is an anonymous blog post submission by Unregistered Submission. We decided to publish it as a valuable contribution to the debate on ‘open science’. However the represented opinion does not necessarily reflect our both/both opinions. To keep with spirit of this post we gave Sophien Kamoun a 24-hours heads up before publishing. Some of his feedback was incorporated into this post by the anonymous author. Slighlty modified paragraphs are highlighted in italic.

There you go, find a duplicated figure panel in an article, make a figure, write one sentence and post it on pubpeer (https://pubpeer.com/), the online journal club where scientific peers can anonymously place comments on scientific publications. Little effort for one person, something that may have a huge effect on people far away from where I am living…

What followed was a tremendously fast response from the authors involved in this manuscript (http://tinyurl.com/lshhv4g) and I think a new world-record in correcting a scientific paper (http://tinyurl.com/mx4dope). I absolutely respect and admire the professionalism by which the authors (Mireille van Damme, Cahid Cakir, Sophien Kamoun et al.) handled the probably very unpleasant situation. I posted my concerns regarding one specific figure of the respective article on Saturday evening, already by Sunday the original data was provided on figshare by the authors to convince any skeptical colleague (http://dx.doi.org/10.6084/m9.figshare.1290790). Let me be clear, I never believed that the authors purposely published data to mislead the reader. Obviously, this was just a simple mistake, something that could happen to anyone actively involved in science.

Why did the authors rush so much to get the original data online so fast? My thought is that the authors wanted to avoid entering a harmful treadmill, in which other anonymous commenters start to dig further trying to add additional “evidence” that the authors purposely misled the reader. In fact, this process almost immediately started after I added my concern. With one person adding some more fuel to the starting fire by talking about the authors purposely rotating another panel in the same figure. What if the authors were unable to provide such a fast response? For example, when data could not be found immediately or someone was on a holiday for two weeks or more? Would the authors have had a fair chance to defend themselves against a growing group of anonymous commenters?

The last couple of weeks I have been following evolving stories around papers of Olivier Voinnet, David Baulcombe et al.,. Doubt about figures in a number of papers were posted on pubpeer in September 2014 (http://retractionwatch.com/?s=voinnet, https://pubpeer.com/search?q=O+Voinnet ). This was followed by an explosion of post refereeing to over 25 papers in January 2015. Worrisome? Yes, but I was a bit shocked by the way colleagues around me spoke with disgust about multiple scientists involved in any of these papers, and how on social media such as pubpeer and retractionwatch people carelessly provide their opinion and accuse scientists potentially involved in figure manipulations and duplications. Probably knowing little to nothing about the factual situation.

To my feeling the pubpeer website in its current form is too much a “hunt the scientist” website, a place where scientists can be suspected of publishing falsified data. Not really the “online community that uses the publication of scientific results as an opening for fruitful discussion among scientists” that it claims or wants to be (https://pubpeer.com/about). Why does a comment that I add myself on a Saturday evening have to appear online in public and to the authors on a Sunday? Why can’t the authors be informed far in advance before making a comment public? Giving the authors ample amount of time to sort things out and reply. The way pubpeer currently works, or better, the way it allows some people to use it, resembles a modern day witch-hunt.

I would like to stress that intentional figure manipulations are indeed extremely bad, in fact it is fraud and that is a very serious crime. This is especially why we should be extremely careful commenting on other scientists work. We should not allow ourselves to create a platform of which the primarily use currently seems to be a public “scientific execution site”. Potentially damaging innocent scientists reputations and that of co-authors who may have nothing or very little to do with the whole situation. To accuse someone from committing a crime is a big thing and I wonder whether this should be discussed so directly and openly in public, with maybe little chances for the authors to reply or defend themselves initially. Do we do the same with other types of crime? We don’t put anonymous notes in supermarkets with names of customers who are suspected of theft, at least not in the country where I live. No, we go to a respectable authority and led them investigate what is actually going on. Shouldn’t pubpeer have a more stringent editorial filter? An online open journal club is something different from an online open crime-report site.

Don’t get me wrong, I am very much pro “open science” and the more discussion the better. Pubpeer is a good initiative, but currently not working optimally. Authors should have a fair chance to defend themselves and one should not judge before all evidence is provided. Also scientists have the right of being “innocent until proven guilty”. In fact, how many of the suspected papers are actually truly worrisome? Yes, for a couple of papers it looks bad, but there also seem to be quite a few with marginal evidence for intentional figure manipulations (http://tinyurl.com/q34yomm, http://tinyurl.com/p3yh9mp, http://tinyurl.com/kyg8ba4, http://tinyurl.com/ojjktz5). Are all these authors and co-authors suspected of fraud or, alternatively, can we accurately point all these cases to one single person?

Pubpeer in its current form is surrounded with a negative and suggestive atmosphere, something you would not like your paper to be associated with. A site where comments seem to be frequently made by over-frustrated scientists. People like sensation: “big names struggling”, always a source of entertainment. Whether these big names are famous movie-stars, politicians or scientists. (Un)fortunately scientists are people too and on pubpeer (and sites like retractionwatch) it is shown that we are often little better then gossip loving yellow press readers in a supermarket.

That brings me to the fact of specifically bringing up the following manuscript/figure on pubpeer. Last week, tweets appeared on social media that were joking about the scientists suspected of fraud (http://tinyurl.com/k8k27yc , sensation!). Did it bring a smile on my face?, honestly yes, it was quite funny. However, would I like to be in the same position as any of the co-authors of the 25+ suspected papers?, and am I 100% sure that all my papers are spotless? My answer would be “No”. Several scientists actively re-tweeted this joke, but isn’t that also a tiny little bit hypocritical? Or are these scientists very sure that their own published work is the gold standard?

I decided to look into a number papers of the re-tweeting scientists present in my literature archive for troublesome data (I was only looking into manuscript main figures, without help of any software). A childish “gotcha game”? Maybe, but I guess that’s how the true wanna-be “science-detectives” on pubpeer work. I discovered one paper of the Kamoun lab with a “serious” issue. Did I suspect fraud?, not for a single moment. But I felt I should bring this up to show how vulnerable we all are as scientist at this moment. The Kamoun lab is without doubt one of the most active labs on social media within the plant sciences field. Something, I actually greatly appreciate and respect! I could thus expect that posting a comment on pubpeer would attract a lot of attention (well it certainly did, http://tinyurl.com/mhbnzlm). Something I hope would aware the community that at this moment we all can too easily be suspected of being a fraud.

Cynical and ironic jokes arise when questions remain unanswered. I understand that the Voinnet lab has had plenty of time to reply to any of the initial concerns posted. This is in stark contrast to the way the Kamoun lab handled the situation, by directly replying to all concerns raised. Nonetheless, with over 25 papers currently in doubt in the case of Voinnet I am aware that ironic jokes currently target a large group of innocent scientist who may not have a fair chance to reply at this very moment.

I would like to open a discussion in which standards of how to criticize a scientific paper are addressed. Should this always be done so open and direct? Shouldn’t websites such as pubpeer have a better editing process for certain type of comments?, especially when issues such as scientific integrity are at stage? At least scientists should be given sufficient time before they are being haunted by a group of “science-detectives”. Lastly, we as scientists should not judge too fast, suspecting or suggesting someone is a fraud is a big leap. It is time we start using pubpeer in a much more positive way by not just posting negative or suggestive comments. To the Kamoun lab: I promise to make a start by now placing my honest, positive and fair opinion on several of your great manuscripts! Pubpeer should be used as online journal club highlighting not only flaws but also the great science out there.

/// See also our second guest post of today by Sophien Kamoun about the same issue. ///

ASPB members and Plant Physiology/The Plant Cell authors for pre-prints

///Disclaimer: This is a copy of our request to the American Society of Plant Biology to support pre-prints.///

ASPB members and Plant Physiology/The Plant Cell authors for pre-prints

Based on a recent discussion on twitter we discovered that the American Society for Plant Biology (ASPB) currently discourages preprint deposition on publically available archives such as http://biorxiv.org/, http://arxiv.org/ or https://peerj.com/preprints/. We as engaged members representing a wide variety of plant biologist would like to encourage ASPB to change its stance and allow deposition of a pre-print when submitting to Plant Cell and Plant Physiology. Pre-print compatible policies are quickly becoming the norm at other journals ASPB members publish in, including: Nature Publishing Group, PLOS, PeerJ, Cold Spring Harbor Press, Science, PNAS, eLife, Frontiers journals, EMBO, Development, and Evolution (http://en.wikipedia.org/wiki/List_of_academic_journals_by_preprint_policy). We do not want to be in the position to have to chose between a pre-print and submission to these outstanding community journals.

Potential benefits to the plant science community include:

  • fast dissemination of important new discoveries in the field
  • fast dissemination of work in progress
  • public visibility of successive revised versions of a manuscript
  • feedback during the review process potentially leading to improved manuscripts. (See for example haldanessieve.org)
  • straightforward way to establish precedence

Thanks for considering our request:

If inclined, edit, sign, and distribute widely here.

Is being scooped the flip-side of a pre-print !?! It wasn’t for us with PLoS

Update 01/14, 11 PM, (Late night update while single dad…): The previous title of this blog-post read “Is being scooped the flip-side of a pre-print !?! It was for us.”. However the last night email from PLoS, while I was talking with my co-first author in Israel where it is already morning, made me change the title. To my surprise and with loads of awe and respect PLoS reconsidered their call of rejecting our manuscript based on the novelty issue caused by the publication of a competing manuscript (see details below). Now instead of considering novelty the editors gave us the opportunity to address the remaining reviewers comments within 60 days. AWESOME and OUTSTANDING. We will forge ahead and do so in the weeks to come and resubmit in due time. It is great to see PLoS taking this important issue seriously. Looking forward to read their updated editorial policies on their website soon so in future there won’t be any confusion about pre-prints and being scooped during the review process. Today PLoS improved the publishing landscape once again. That’s what I *love* about PLoS. Way to Go!

Go Pre-Print Go!

Update: A better title might have been -> Posting a preprint before a paper is in press puts you at risk for being scooped.

Update #2, January 07, 2014 ~ 5PM PST: PLoS got in touch and will have a ‘second’ look at the case. That is absolutely something I appreciate about PLoS actually listening to its community. I hope the major outcome of all this is that PLoS openly declares its editorial standards on preprints and (independently) on being ‘scooped’ during the review process. This is especially important for papers that got submitted after the initial PLoS submission date, because no one really knows where papers get passed around.

Let me get this straight right away. I love open science, open access, the idea of pre-prints and doing good reproducible science. Ever since I heard about the concept of pre-prints I started to put my articles on pre-print server, as much as possible, for everyone to enjoy while the manuscript is under review see here, here and here. This means people can actually read the science well before publication, which is often more than 6 months earlier. Also I really like the idea of and PLoS itself having worked with them on open access topics several time. I am also know to be more critical with fiends I like ;).

Here is the story so far in brief.

June 11, 2014 Submission of manuscript to a PLoS journal and deposition on biorxiv.

July 16, 2014 Rejection of manuscript after full review at this particular PLoS journal.

July 25, 2014 Re-submission of manuscript to another PLoS journal. Back to back with another paper of collaborators showing something similar in another plants species. This gets accepted December 4, 2014, also after major revisions.

September 10, 2014 Preliminary acceptance with major revisions within 60 days.

November 16, 2014 Re-submission of majorly revised version. Submission letter indicated that alternative study was published in another journal. (Initial submission August 12, 2014 and acceptance October 28, 2014. Author declare on pubmed that they missed our pre-print study and will add citation in the final print version (here).

December 19, 2014 Rejection of our manuscript based on novelty and one specific experimental request, which we could have addressed within 2-3 weeks if possible (more details below if wanted).

December 20, 2014 Rebuttal of decision. Citing that we had pre-print out and that authors of other competing study omitted our pre-print at time of submission of their paper.

January 07, 2015 Rejection of our rebuttal due to the following editorial practice. I will cite this here straight, because we looked for editorial guidance in regards of pre-prints and competing publications during the review process on the PLoS webpage. Without success!

‘From an editorial perspective, a work would not be considered novel if a similar paper is published before the work is fully accepted for publication. Although we fully understand this is a difficult situation for you and your co-authors, we could not neglect the fact that Lu and associates reported a similar story in a highly regarded, peer-reviewed scientific journal, which reduced both the significance and novelty of your work. We realized that the situation might be a bit tricky with the deposition of your manuscript onto an online pre-print server before the submission of the paper by Lu et al. While PLOS journals are “pre-print friendly”, we do not have a specific policy regarding scooping. Nonetheless, we can assure you that none of the authors in the published paper served as a reviewer for this manuscript and there is no conflict-of-interest issues during the handling of your manuscript by PLOS xxxxxx.’

I am fine with having my paper rejected because reviewers are not satisfied and it is not deemed novel enough at the initial stage of reviewing. That is simply part of the game. Problem here is the missing transparency. PLoS is ‘pre-print friendly’ (here and here) however does not honor submission date*. Of course putting a manuscript on a pre-print server at the point of initial submission exposes you to the risk of being scooped by other folks scrambling to publish their data in time while you in review and revising. Especially in a culture that for now does not ‘honor’ and cite pre-prints.In this case being ‘friendly’ doesn’t really support pre-prints. You can be scooped during the review process anyway.

Will deposit my manuscripts on pre-print servers at time of submission? YES, as much as possible!! I think these are exciting times for biology.

Will I submit to PLoS ‘higher tire’ journals at the same time? NO!! Not without a straight editorial policy on pre-prints and being scooped during the review process.

Update 01/14, 11 PM: Yes with the new and hopefully clear cut editorial policy on pre-prints and being scooped in the review process I will always again submit my paper to PLoS and a pre-print server at the same time. Way to Go!

Update after short twitter discussion. It is not clear if author’s of the competing paper knew about our pre-print. This is only speculative. Main point is #1 they MIGHT have rushed for publication having heard of pre-print or manuscript another way.  #2 journals like PLoS need to get their editorial policies straight about pre-prints and getting scooped during the review process. Honoring submission date will become more important with more pre-prints out.

*Even though only ONE specific additional second round experiment was request. This confirmatory experiment could have been addressed easily. I am willing to post the whole review process and communication without names up here if folks of PLoS don’t mind so you can judge for themselves.

 

 

 

 

FunFunFun: Our Software carpentry bootcamp @UCDavis

Sponsored by the UC Davis Shields Library and UAW5810, the UC post-doc union

We recently hosted a Mozilla Science Foundation Software Carpentry Workshop at UC Davis. For me this bootcamp was somewhat a ‘coming to ages’ event. Organizing an event arising from necessities public institutions are obviously not able to fulfill is a continuation of what I have been doing from my early teens onwards.  Instead of a running a German-punk-rock style ‘food-not-bombs’ in a run-down Bavarian bar we rented for cheap, this time I organized an event with Mozilla Science Lab teaching post-docs at a prestigious sunny Californian university basic bioinformatics skills. Of course the audience is absolutely not comparable and total different needs are addressed. One might be even more important than the other. Yet in the core organizational scheme they are the same. People volunteer their time and talent to create a space in which a commodity is shared and enjoyed by everyone for as close to free as possible.  Sharing ideas, food, company and everything else without being concerned about turning maximum profit is such a liberating feeling. Now and than. Back than in the mid-90s I got to know what the world-wide-web was. One day we hosted an event by a guy from ATTAC, a leading group from the so-called ‘Anti-globalization’ movement. /// A simple fact. Much to be criticized about this movement with loads of great motives./// I was asking him how to best stay in touch and he replied ‘give me your address’. Young, blue-eyed and naïve as I was (and still am) I wrote down my physical address on a bierfilz (beer coaster) and handed it to him. He just stared at me and said ‘no your e-mail address’. I felt so 80’s.  Later that day I went to an Internet coffeshop got my first email address using Firefox the Mozilla browser. Now at UC Davis hosting this software carpentry event I felt more grown up, organizing an event that enabled me to teach myself more easily the skills I and many of my fellow post-docs need for the scientific endeavors in the 21st century.

But what about the ‘hows’ and ‘whys’ of the workshop?

Main Organizers:

Jonathan Catchat, post-doc from the UC Davis library, @jcachat

Benjamin Schwessinger post-doc in plant pathology and UAW5810 UCD head Stewart, @schwessinger

Main Sponsors:

UC Davis Shields Library and the post-doc union at UC, UAW5810

Motives to organize this event:

Both UAW5810 and UC Davis Shields Library are dedicated to provide post-docs and the wider research community at UC Davis with the resource required to perform the best research possible. Bioinformatic literacy is a basic skill beneficial for nearly all research disciplines. Yet adequate affordable training is lacking much behind the demand. Our workshop for 80 people sold out within 10 days nicely illustrating this fact.

How we organized this event:

I put out the idea to organize such an event after hearing about SWC at the “Publish or Perish” workshop at UC Davis. Somehow Jonathan got in touch. We organized the event mostly by emailing back and forth with SWC organizers, talking with folks at UC Davis Shields library and us two meeting in person several times.

Main bottlenecks:

Getting a room -> was easy as UC Davis Shields library offered two excellent rooms

Getting instructors sorted -> took a while but all went perfectly fine and smooth at the end

Providing food and coffee for the event -> all went well as many people helped out and made it happen //Thanks to Randyman,  AmazingJeff, AstroBen, MosswomanJessica…..///

Awesome Instructors ///in random order///:

Naupaka Zimmerman, @naupakaz

Bernhard Konrad, @BernhardKonrad

Andrea Zonca, @andreazonca

Dav Clarak

Amazing Helpers  ///in random order///:

@kseniakrasileva, @CieraReports, @davidjayharris, @noamross

Friendly people helping out:

Jessica, Randy, Ben, Jeff, Aaron, Jeff, Allison, Pat and many many other helping hands I cannot remember right now.

Summary: Awesome super helpful event for everyone. Start of great collaborations and discussions, e.g. PloS Blog about R markdown.

Future: Many more of these SWC bootcamps sponsored by UAW5810 at other UC campuses. Working together makes a better place for everyone.

Thanks SO much to everyone who helped out and made this event happen!! That many more will follow.

Behind the Scenes: The people who debunked Ax21

A few months ago I wrote a blog post highlighting  the two retractions  that took place in our lab ///one Science and one PloS One///. The topic has since attracted much attention, with articles written in The Scientist and Scientific American. Pamela Ronald and the rest of us received much praise for how we handled the situation and strived towards setting the scientific record straight. We even made it into the Top 10 Retractions of 2013 -> In a good way!!! It was certainly a very long and challenging endeavor (18+ months) for all of the actors that were a part of this journey. For sure, at times it was more like a nightmare inflicting both scientific and personal scares, though eventually we exited the once long and dark tunnel stronger and as better scientists.

However, who was the driving force behind all of these scientific corrections? Who are the scientists who figured out that there was something wrong in the proposed role of Ax21 in the bacteria as well as in activating rice immunity? For sure, actual people must have done the work, inspired each another and diligently followed the data.

This answer can be found in our recent congenially communally co-directed PeerJ paper ///An excellent ‘journal’ that we can only most highly recommend to everyone///. The main actors in debunking Ax21 are named as authors and include the following:

Ofir Bahar and Rory Pruitt as experimental, intellectual and conceptual leads.

Dee Dee Luu, Benjamin Schwessinger, Arsalan Daudi, Furong Liu, Randy Ruan, Lisa-Fontaine-Bodin and Ralf Koebnik as support cast, contributing immensely intellectually, conceptually and experimentally.

And, last but not least, Pamela Ronald as scientific producer.

Unfortunately our rice expert support cast Mawsheng Chern did not make it on to the paper due to highly variable scenes in the pre-treatment shots. Nonetheless he contributed intellectually and conceptually.

This last part of the Ax21 story published in PeerJ was also featured on Retraction Watch, where Pamela Ronald accurately answered some tricky questions. In case YOU have any questions left please come and join us at our ‘Ask Me Anything’ session with PeerJ on February 11th at 9:00-9:30 am PST ///link tba///.

In summary we are very excited this piece of data was published. We are happy to set the scientific record straight about the role of the bacterial protein Ax21 in rice biology. We hope other people, who have been working on orthologous proteins, will follow suit. To make life easier for everyone, we also corrected our Annual Review in Plant Biology article. We reasoned unless it was fixed, it might become really confusing for everyone starting to learn about rice immunity by reading a recent review from our lab.

Finally!!! We closed this chapter! Dried out the swampland!

We NOW look forward to being the architects of progressive everlasting castles.

How to get hold of your growing supplies (for rice) in North Cal

After all the seriousness in our recent blog posts we want to share a rather entertaining Northern California story. It is about getting growing supplies in a state where medical marijuana is legal.

The background to the story is… I am working on the rice immune system and growing rice of the same high quality the whole year round is difficult even in California. So because our new assays require constant and fully reproducible growth conditions we recently switched to grow our experimental plants in semi-hydroponic system in the growth chamber. For this purpose we adopted so-called ‘plugs’, a kind of pressed soil mimic that can equally retain moisture, and we use a separate plug for each plant.

How to get these plugs!?! From your local hydroponic (a.k.a. weed grow) shop

One colleague of mine started using the plugs and we aimed to extend our rice plant growing empire. So we went to our local Davis weed grow, mmm… I mean hydroponic, shop in order to get more supplies for our scientific operations. Funny enough, the manager wasn’t always available and our orders were not always processed in time. I guess he was busy doing other business. Because we did not want to wait forever as time is money and we in the science community are mostly short of both I extended our supply search operation to Oakland, my main hometown.

//Oakland is a beautiful city in the East Bay. The second city I lived in, after Glasgow, where advertisement asks the community not to flush their controlled substances down the toilet but to dispose them appropriately.//

After having looked up some distributors on the plugs producers’ webpage I get into my car on a Monday morning with my little four-month-old son in the backseat. The first stop is just a couple of blocks away from my home and a short drive through industrial pretty West Oakland. After a couple of turns Google tells me the shop is just to my right but all I can see is a run-down townhouse with silver foil in the windows and a barking dog in the front yard. Well…the producer might have sold plugs to these guys but for sure not for resale. So I take off to the next distributor on the list. The shop is supposed to be near the harbor and after circling the block twice I even see the name on a hand-painted sign. I park the car, get my little son, still sleeping in his car seat, out of the back and try to find the entrance. Thanks to my street credit earned by my plenty-full tattoos a white guy with gold chains smiles out of the SUV and says: “Yes dude, you are right here. This hole in the wall is the entrance”. So I drag my son and myself through this hole inside a debilitated warehouse with a small very messy room, which is suppose to be the showroom. People are talking over the ‘counter’ about some sort of growing… and become a lot quieter as soon as they see my kid and me. A kind of tweaked out white thin lady asked me what I am looking for. I with my foreign accent and mumbling try to tell her I am looking for these plugs. After several back and forth she says she finally gets what I am after but is out of the supplies and her shipment would come in at 3 p.m. I simply reply I would come back later and call before to confirm. She nods and shows me out of the door in a brisk fashion.

Just around three I look up their phone-number on the web. The website gives the number and states in bold letters DO NOT MENTION MEDICAL MARIJUANA ON THE PHONE OR WE WILL NOT SERVE YOU. Well no problem I think I only grow rice. The voice mail box is full… and I just take a ride there hoping to finally complete my mission… bad luck though the shop is closed and also these guys are also clearly not interested in selling plugs (but maybe other stuff though).

I don’t want to postpone my whole next week of work that I already planned out so I continue my quest. The next shop turns out to be a money laundry hydroponics shop. It looks all fancy from outside, yet the door is locked and I have to ring the bell to enter. The shelves are full to the brim but there are no plugs and no one interested to sell really anything. I slowly get disillusioned with the East Bay grow shops and decided to have one last go. This time I call the shop beforehand again. Wow! Someone picks up the phone, speaks clearly and promptly confirms their actual will and ability to sell these plugs. So off for my final ride of the day. When I arrive I enter a pretty looking small old factory building with loads of plants and greenery inside. Loud hip-hop music fills the air. In the corner stands a nice 80’s looking pinball machine and a pin-pong table occupies the middle of the room. A mid-twenty trendy looking guy with a baseball hat smiles at me from behind the counter and a young woman appears from one of the side rooms. Both greet me warmly and offer their kind assistance. I ask them for my plugs and get 500. Just in case so I don’t have to go treasure plug hunting in Oakland again. Both just smile knowingly. Well if they only knew that I am a rice grower… Luckily, getting a receipt for my purchase is no problem with these guys. So I am finally happy and shove the plugs into the trunk of the car for the ride to UC Davis the following day.

Happy hydroponic plant growing in North Cal everyone.

DO mind the scientific skeletons in the closet

Yet another Science paper retracted today. Nothing much new unfortunately (sting!).           Except that for me and my colleagues this was not just another example of non-reproducible work. We planned our research projects around it. So here are my thoughts about this: just in time for Halloween.

There has been much talk about the problem of reproducibility of data and the rise in retractions. The discourse is mostly centered around the perpetrators and the negative impact this sloppy or knowingly flawed science has on the industry and the perception of scientific endeavor in society. Who did this awful study? What reviewer did not catch this missing control and the missing/sloppy stats? Why did they do it? Why don’t they admit their mistake? What’s wrong with peer-review? Well….much is wrong with the industry and many people have great ideas how to tackle some of these issues.

What is usually missed from the discussion is the impact such dubious science can have on young, early career scientists if they are at the receiving end of non-reproducibility. What do you do if you come to a famous (or not so famous) lab, you get a project, which is based on fantastic data and high impact publications, BUT you cannot reproduce it? How do you approach this issue? What does it mean for you and your career if your project just goes to shreds because it’s simply based on bad data? Do you get another chance? Will you be forever associated with this flawed data and your reputation damaged? Are you going to be so disillusioned with science to the point that you want to leave academia? Do you try to fix the problem or silently move on? Do you try to make it work? Do you tell the boss and explain your situation? Or is it you? Can you just not get it to work? Can it really be that this is all wrong? Why cannot I reproduce this data? Can this BE???

I think everyone can appreciate the complexity of the issue. Many people I talked to had their own experience to share. Here is what’s frightening:  sloppy science and misconduct, I thought, is something you read about in journals and not something I would experience myself. This would never affect me or close friends in other labs, who are all great scientists in my eyes. I was wrong.

We young, early career scientists in our lab went through something very serious and exemplary recently. In mid 2011 to beginning 2012 we just joined the lab of Pamela Ronald during a huge turnover of people. Everyone of the “old crew” of many years, sometimes more than 8, left or were about to leave around the time we came in. We were all super excited to get going and to push the borders and boundaries of science /// Thanks Saul for this awesome expression./// However, no matter how hard we tried, our own data did not add up and we could not make much sense of it in the view of the lab’s recent  immunity-related publications, which appeared in Science and Plos One. /// Here is a three sentence summary: The lab cloned an amazing immune receptor XA21 in rice that confers resistance to the most important pathogenic bacteria on rice (Xanthomonas oryzae pv. oryzae) in 1995. In 2009 they published the identification of the bacterial protein Ax21 as ligand of this immune receptor. In 2011 they reported the function of this bacterial protein Ax21 as a novel cell-cell communication factor (also referred to as quorum sensing factor) in the bacteria./// We, a group of concerned early career scientists in the lab, brought this discrepancy  regarding the 2009 and 2011 studies to the attention of the PI and explained to her what we thought was wrong. A long, intense, careful, multifaceted, repetitive and diligent truth-searching endeavor started. I won’t bore you with the details. In short, we were not able to reproduce the core findings of both Ax21-related papers.  Of course, this was very distressful for everyone involved especially as other people published similar work in relation to the bacterial protein Ax21. /// I am not sure what this says about the current state of science. I find it very concerning./// Kudos to Pamela Ronald, who reached the only scientifically correct decision and retracted both papers. She obtained much praise for this brave step. For sure, not every PI would have followed through in a similar situation. Pamela Ronald’s decision became the topic of the inaugural post in the new category “Doing the right thing” on the famous “retraction watch” blog. You can also read her perspective on how this whole story unfolded in her guest post on the Scientific American blog. I can only agree and applaud Pamela Ronald on making the right decision and retracting these two papers. For sure, it was not easy.

What I would like to highlight is the importance to clear the scientific record once mistakes in publications are discovered. Mistakes will always happen yet they need to be corrected. People should not fear to clear their scientific record but rather be rewarded for doing so. I am sure every reader of this blog knows of multiple scientific skeletons in people’s closets. It takes much courage from the PI’s perspective to bring these mistakes out in the open. They potentially have much to lose….BUT the PI will have the time to correct the situation. On the other side what’s going to happen with early career scientists, who got into one of these situations without their own wrong doing? Who is routing for them? What happens if the PI does not retract the paper(s) and simply disregards the concerns? I know of least of one situation in which a good friend of mine had exactly this happening to her. Finally she simply left academia, because there was no support for her and no way out. Young, early career scientists can be treated like a dispensable commodity if they don’t play along. Of course, this is the worst-case scenario, yet once you are trapped in the non-reproducibility hole you will lose a lot of time no matter what. You might be asked to repeat experiments multiple times, generate the same material anew, and, in general, spend a lot of time on research that is hardly publishable ///Ha!! no impact factor for retractions///. No one can give you this time back in your career and many people might not understand why you are not moving forward. Well, you simply cannot build skyscrapers on swampland. Only once everything is cleared up will you be able to move on with your career without losing too much. This is the only way out for the early career scientists, and PIs should be aware that they are not only dealing with their own careers. ///In this regard again thanks to Pamela Ronald for taking the courage.///

It is not only the PI’s but also the early career scientists’ significant efforts and courage that leads to the clean-up of the scientific record. Be diligent in these situations. Talk critically with your co-workers in the lab. Verify your own stock and everything given to you. Perform the right controls. Ask for advise from other senior colleagues. Follow your understanding of science as a truth-finding endeavor. This journey won’t always be linear and sometimes mistakes happen. Deconstruct them, correct them, learn from them and build something new. In my opinion, this is what truly makes a reputable scientist.

P.S.: So let’s figure out what’s really going on with this amazing rice immune receptor XA21. /// For the “love of wisdom”///.