DO mind the scientific skeletons in the closet

Yet another Science paper retracted today. Nothing much new unfortunately (sting!).           Except that for me and my colleagues this was not just another example of non-reproducible work. We planned our research projects around it. So here are my thoughts about this: just in time for Halloween.

There has been much talk about the problem of reproducibility of data and the rise in retractions. The discourse is mostly centered around the perpetrators and the negative impact this sloppy or knowingly flawed science has on the industry and the perception of scientific endeavor in society. Who did this awful study? What reviewer did not catch this missing control and the missing/sloppy stats? Why did they do it? Why don’t they admit their mistake? What’s wrong with peer-review? Well….much is wrong with the industry and many people have great ideas how to tackle some of these issues.

What is usually missed from the discussion is the impact such dubious science can have on young, early career scientists if they are at the receiving end of non-reproducibility. What do you do if you come to a famous (or not so famous) lab, you get a project, which is based on fantastic data and high impact publications, BUT you cannot reproduce it? How do you approach this issue? What does it mean for you and your career if your project just goes to shreds because it’s simply based on bad data? Do you get another chance? Will you be forever associated with this flawed data and your reputation damaged? Are you going to be so disillusioned with science to the point that you want to leave academia? Do you try to fix the problem or silently move on? Do you try to make it work? Do you tell the boss and explain your situation? Or is it you? Can you just not get it to work? Can it really be that this is all wrong? Why cannot I reproduce this data? Can this BE???

I think everyone can appreciate the complexity of the issue. Many people I talked to had their own experience to share. Here is what’s frightening:  sloppy science and misconduct, I thought, is something you read about in journals and not something I would experience myself. This would never affect me or close friends in other labs, who are all great scientists in my eyes. I was wrong.

We young, early career scientists in our lab went through something very serious and exemplary recently. In mid 2011 to beginning 2012 we just joined the lab of Pamela Ronald during a huge turnover of people. Everyone of the “old crew” of many years, sometimes more than 8, left or were about to leave around the time we came in. We were all super excited to get going and to push the borders and boundaries of science /// Thanks Saul for this awesome expression./// However, no matter how hard we tried, our own data did not add up and we could not make much sense of it in the view of the lab’s recent  immunity-related publications, which appeared in Science and Plos One. /// Here is a three sentence summary: The lab cloned an amazing immune receptor XA21 in rice that confers resistance to the most important pathogenic bacteria on rice (Xanthomonas oryzae pv. oryzae) in 1995. In 2009 they published the identification of the bacterial protein Ax21 as ligand of this immune receptor. In 2011 they reported the function of this bacterial protein Ax21 as a novel cell-cell communication factor (also referred to as quorum sensing factor) in the bacteria./// We, a group of concerned early career scientists in the lab, brought this discrepancy  regarding the 2009 and 2011 studies to the attention of the PI and explained to her what we thought was wrong. A long, intense, careful, multifaceted, repetitive and diligent truth-searching endeavor started. I won’t bore you with the details. In short, we were not able to reproduce the core findings of both Ax21-related papers.  Of course, this was very distressful for everyone involved especially as other people published similar work in relation to the bacterial protein Ax21. /// I am not sure what this says about the current state of science. I find it very concerning./// Kudos to Pamela Ronald, who reached the only scientifically correct decision and retracted both papers. She obtained much praise for this brave step. For sure, not every PI would have followed through in a similar situation. Pamela Ronald’s decision became the topic of the inaugural post in the new category “Doing the right thing” on the famous “retraction watch” blog. You can also read her perspective on how this whole story unfolded in her guest post on the Scientific American blog. I can only agree and applaud Pamela Ronald on making the right decision and retracting these two papers. For sure, it was not easy.

What I would like to highlight is the importance to clear the scientific record once mistakes in publications are discovered. Mistakes will always happen yet they need to be corrected. People should not fear to clear their scientific record but rather be rewarded for doing so. I am sure every reader of this blog knows of multiple scientific skeletons in people’s closets. It takes much courage from the PI’s perspective to bring these mistakes out in the open. They potentially have much to lose….BUT the PI will have the time to correct the situation. On the other side what’s going to happen with early career scientists, who got into one of these situations without their own wrong doing? Who is routing for them? What happens if the PI does not retract the paper(s) and simply disregards the concerns? I know of least of one situation in which a good friend of mine had exactly this happening to her. Finally she simply left academia, because there was no support for her and no way out. Young, early career scientists can be treated like a dispensable commodity if they don’t play along. Of course, this is the worst-case scenario, yet once you are trapped in the non-reproducibility hole you will lose a lot of time no matter what. You might be asked to repeat experiments multiple times, generate the same material anew, and, in general, spend a lot of time on research that is hardly publishable ///Ha!! no impact factor for retractions///. No one can give you this time back in your career and many people might not understand why you are not moving forward. Well, you simply cannot build skyscrapers on swampland. Only once everything is cleared up will you be able to move on with your career without losing too much. This is the only way out for the early career scientists, and PIs should be aware that they are not only dealing with their own careers. ///In this regard again thanks to Pamela Ronald for taking the courage.///

It is not only the PI’s but also the early career scientists’ significant efforts and courage that leads to the clean-up of the scientific record. Be diligent in these situations. Talk critically with your co-workers in the lab. Verify your own stock and everything given to you. Perform the right controls. Ask for advise from other senior colleagues. Follow your understanding of science as a truth-finding endeavor. This journey won’t always be linear and sometimes mistakes happen. Deconstruct them, correct them, learn from them and build something new. In my opinion, this is what truly makes a reputable scientist.

P.S.: So let’s figure out what’s really going on with this amazing rice immune receptor XA21. /// For the “love of wisdom”///.