Deposit full protocols early!

Disclaimer: I am an ambassador of the protocol sharing website protocols.io. Yes I got an $50 amazon voucher from them end of last year with which I bought some books for my son Saul. I think pre-prints are awesome but have some caveats.  And foremost I love free stuff accessible by everyone.

Prelude

I am currently working on long-read genome assemblies of fungi and plants. All successful long read genome projects start with months of optimization for DNA extraction methods. This can often involve phenol chloroform extraction for weeks on end with slight alterations of the same protocols. Mine took about 9+ months to optimize and to finally get some good data back.  I was lucky as  I got three protocols to start with form colleagues around the globe. Looking into the scientific literature was no help at all. The one I build on was from INRA and put online by Jason Stajich here, which I only found because I asked Andrii Gryganskyi for help. So overall I didn’t reinvent the wheel but simply build on what others already created previously. Pretty much was 99%+ of all science really is. Notwithstanding,  I knew several other groups struggled with the same problem so I put my working protocol online on protocols.io and publicized it on the 18th of April 2016.

Now on the 31st of August 2017 we published pretty much the same protocol as a peer reviewed methods chapter in a book a friend of mine edited.

I would like to celebrate this occasion by sharing some thoughts about open sharing of methods early. Overall I obviously think it is advantages for you and everyone else in science to do so. This is in contrast of letting one’s protocols rot on one’s harddisk and to only let them out to see the light of the day once someone offers you a co-authorship.

In semi random order.

Instantaneous availability

The most obvious one first. It took 17 months to publish pretty much the exact same protocol I knew was working when I put it up online. That’s nearly half my son’s life. In the meantime the protocol got 2375 views, I got 10+ emails or messaging requests to explain different parts of the protocol better and some ‘Thank yous’ on conference as other got it to work for their favorite species as well. Overall putting it up online early enabled other to discover more in in less time instead of reinventing the wheel of DNA extraction protocols.

I know some people would not use none peer reviewed protocols that they find online (discussion here and here). I personally think it is an illusion to believe that methods get properly reviewed in papers. I only ONCE got a comment on a small technical detail in my methods section across all my 34 publications. Similarly, I’ve hardly ever (never?) seen another reviewer ask for details or even comment on the methods section.

I know the protocol worked for me and no closest peer review would have changed it.

Discoverability and ease of use

Everyone loves detailed easily accessible protocols. EVERYONE! Everyone despises short methods section in the supplement that provide no detail and cite another paper that cites another paper that cites another paper that cites another paper that supposedly did the experiment the exact way the authors did it.

So putting your full detailed protocol on your lab webpage is good (see this awesome long read DNA extraction protocol) yet putting it up on a publicly accessible and search-able platform like protocols.io is even better. You get a DOI, it has versioning, people can fork it, people can comment on it, people can ask you directly, it is backed up available forever, and you can even add it to collections/usergroups so even more people can discover it. Everyone is happy using detailed protocols they can interact with and cite. Everyone!

Catering to both systems

Of course some people get nervous and fear they cannot publish their methods in esteemed journals anymore. Of course not everyone goes online on novel platforms like protocols.io to hunt down the latest protocol. Many prefer to search the peer reviewed literature. And that is really the great thing about depositing full methods early. You can do both.  You can share your protocol to help others quickly and get the bonus points of a journal publication. That is just what it did! It might well be that more people will cite one’s publication as they already used one’s protocol before it was officially published in a peer reviewed journal. That is the case for preprints and future studies will show if we can observe similar effects for protocols protocols.io. For sure I will cite both protocols.io and my book chapter in my next genome manuscript preprint.

Reproducibility and rigor

There is a lot of talk about the lack of reproducibility in life sciences. Much of this is focused on depositing code online. Not much is focused on the actual data generation. Methods section are often poor and most supervisors don’t care a la ‘just write something no one reads it anyway’. Yet every good project starts with good data, starts with a solid detailed protocol of how things are done and were done. This not only helps others to reproduce one’s work but also oneself and one’s lab. Writing down great protocols often makes one realize the fine grained and important details. Sharing those protocols publicly is even better as it accelerates discoveries, saves money, makes science a better place for everyone. On top we in the western world sitting at elite universities often forget how hard it is to access high quality protocols. One just need to look at researchgate to see what I mean.

Come join us, make everyone happy, and share your detailed protocols online early. Imagine a world were you just have to search for a detailed protocol and you find one pretty much immediately. Just like cooking recipes.  

Both protocols.io and bio-protocols are a great starting point.

 

 

Advertisements

Today I reached my a-index

Today I finally reached my a-index. Yes I just made this up to celebrate my 34 accepted publications at the age of 34.

No a doesn’t stand for a*8hole but for age. The a-index is achieved once you reached the same amount of publications as your age. The convenient fact about this new amazingly made up index is that it only requires one publication per year once you reached it.

So I guess I can take it easy from here on and focus on getting some science done instead of hunting yet another index.

The #FOR community is named Science People of the year

I am ecstatic. For real this is amazing! An awesome achievement. We, the Future of Research community, were named  ScienceCareers People of the Year!

Oh yeah! This feels SOOOOO good. All of use put tonnes of payless volunteered hours in FOR and other important projects aimed at improving the academic enterprise. ScienceCareers put it nicely

“Each December, Science Careers names a “Person of the Year” to recognize those who have made especially significant and sustained contributions to improving the lot of early-career scientists. Up to now, each year’s honoree has been a senior academic figure occupying a prestigious post. This year’s choice, by contrast, is plural; of a different generation; and at the opposite end of the academic status ladder. For their dedicated, creative, and expanding efforts to empower early-career and aspiring scientists with knowledge and awareness so that they can take control of their futures and help bring needed change to the scientific enterprise, we are delighted to name the activists of the Future of Research (FOR) movement as the 2015 Science Careers People of the Year.”

It is great to see an actual community effort by young upcoming early career academics rewarded by an world renowned science stable. Sometimes it simply feels good to get some back patting from the elite. Clearly, FOR got people active and the bigger academic community talking about issues important to the academic enterprise (e.g. diversity). We also create community by bringing together people with the same interested in improving our  and other peoples’ lives. I see this award not only as a recognition of past achievements but as support of our future aspirations. Folks believe that change is possible and there is a opening in history to make things happen. Hopefully FOR will become a stable itself throwout our careers and I meet folks again.

Thanks to each of the many countless people that make this happen.

On a personal note, this really speaks to my heart and soul, because the award does not single out an individual established person but a community. Only by working together science progresses, this is true within the lab and outside. This recognition is soothing to my soul. Like many other I met over the years [15+], I have been volunteering for all sorts of community actions within and outside of academia for quiet a bit. Many of us spend countless hours on planes, on hangouts, in meetings… organizing events that are not really accounted for in the status quo career progression pathways. Yet we do it because we believe in it. We think it makes our communities a better place.  While seeing things we created happening and community forming we often get wet handshakes and some muttered ‘well done’. So having the elite say “Excellent Job!” loud and in the open is nice!

Anyways, I am sure many of us would keep on doing our projects nonetheless. Yet recognition goes a long long long way!

 

P.S.: This early-mid career conference at the Australian National University is our next local project spanning the globe.

Discovery and Redemption emerge from a scientific mistake

///This is a slightly delayed post as it was lost in the labyrinth of popular science glamor mag editor desks. We are finally moving on and decided to self publish instead.///

///Authors:

Benjamin Schwessinger, Ofir Bahar, Anna Joe, Mawsheng Chern, and Rory Pruitt

Internal editor:

Pamela Ronald

///

We were confused and perplexed. Our team in the laboratory of Pamela Ronald at the University of California, Davis could not reproduce previously published results. Instead of building on the reported discovery that a microbial signal triggered the plant innate immune response, we were thrust into trying to figure out what went wrong. It was the most difficult time in our careers. Then, we made a remarkable new discovery.

The story began in the 1970s, when Professor Gurdev Khush and colleagues demonstrated that a wild species of rice was immune to most strains of a serious bacterial disease. This was exciting because such broad-spectrum resistance had not previously been identified. Such traits are agronomically important because farmers can plant resistant varieties rather than spraying pesticides. In 1995, Pam’s lab reported that the broad spectrum resistance was controlled by a single gene, called Xa21, predicted to encode a receptor that senses the microbe and then activates the rice immune response.

In 2009, Pam and her former team members reported the discovery of such a molecule.  However, in 2012, while trying to build on those findings, we discovered major errors in this work. Pam contacted the editors to inform them of the identified problems. She also decided to notify the scientific community of the issues at an international research symposium. Ben still remembers the day when Pam announced the errors in front of so many distinguished colleagues, many whom Pam had known her entire career. “It was remarkable that Pam mustered all this courage to inform the community at this top research symposium. Some in the audience buried their heads in their hands and were clearly uncomfortable but still more people were awed and expressed their support.” Sophien Kamoun, a leader in the field of plant biology, displayed his respect in a tweet:

Finally, in October 2013 we had accumulated sufficient new experimental evidence definitively proving that key aspects of the study were incorrect, Pam and her former colleagues that co-authored the 2009 study retracted the original Science paper.

Pam and the several of us in the lab lost many nights of sleep as we racked our brains to figure out what went wrong. We had many intense discussions on how and why this could have happened. As Ofir remarked, “Pam was on sabbatical in France when we discovered the problem and so we had these electrifying video conference calls to bring everyone on the same page.” It took time and persistence to break down the problems one at a time as we were working backwards identifying what were solid results we could build and what we could not. One day Ofir and Ben were having their morning coffee outside in the Californian sun when Ofir announced that he had found that some of the bacterial strains used in the previous study were mixed up. Rory still remembers this day vividly, “Even though this was not happy news, we were so happy, we finally had a partial explanation of what went wrong.” From that day onwards we were getting back to solid ground. The scientific process had pulled us through and pointed us in the right direction. Our team discussed the mistakes and corrections in lectures, blog posts and in a scientific publication. The process with which we addressed the problems was highlighted in an article in Nature magazine and in Retraction Watch, a blog that reports on retractions of scientific papers, as “Doing the right thing”. In a dubious claim to fame, the 2009 retraction was included in the top 10 retractions of 2013.

While we were notifying our colleagues, discarding old strains and correcting the literature – an essential part of science so that others do not waste (more) time trying to build on fatally flawed work- we were also working hard to discover the microbial partner of XA21. In the mist of all the fog Rory identified a new bacterial mutant strain that was able to infect XA21 rice plants. We were excited but cautious at the same time. Could this really be the correct mutant lacking the microbial partner of XA21? The 15+ year lab veteran Mashweng was especially skeptical as he had lived through the whole story form the beginning. He often suggested critical controls and reminded us to be extra critical with our own results. A superb team of collaborations from around the world allowed us to incorporate additional experiments and independent controls perform in former critics’ laboratories. After many independent tests we were certain of our results. In mid 2015, we reported the identification of this long sought after molecule that activates XA21-mediated immunity in Science Advances.

We named this small microbial protein RaxX. We found that bacteria that lack RaxX are able to evade detection by the rice XA21 immune system. Bacterial strains found in farmer’s fields in India, which express alternate versions of RaxX can cause disease on XA21 rice plants.

Wrestling with trying to reproduce flawed experiments and discovering the new molecule in rapid succession was an enormous challenge. In our view, the key point in straightening out such complicated and delicate situation was the persistence and collaboration of our laboratory team. Pam worked hard to keep the team together. We were a dedicated crew of senior scientists, technicians, postdocs and graduate students (as well as former team members who shared their records with us because they also wanted to know where they had made mistakes) motivated by a supervisor determined to get to the bottom of the situation. Looking back, we find it stunning how this all worked out. “It’s amazing what can happen if you are doing science with a set of respectful and likeminded people. We were focused on setting the record straight rather than blaming others. It clicked. We were on a run”, Ofir still remembers to this day. We felt our careers were on the line. We realized that there was no moving forward without first going backwards. We did not give up even though at times we wished we could. Working together on this daunting challenge buoyed all of our spirits. “Back then, I [Ben] was living in this lovely trailer with a trellis outside. We had ‘poker-nights’, when everyone came along to talk and be jolly. At times it felt like ‘group therapy’ with drinks.”

We were impressed by the supportive and kind response from the scientific community, our editors, and funding agencies including NIH, BSF, and HFSP. We received many letters of encouragement – even from complete strangers. These conversations helped keep us going. We even had new scientists join Pam’s lab in midst of the mess. Anna Joe, a postdoc who joined the lab at the time of the retraction remarked, “Their open mind and transparency attracted me to the lab.”

Pam tells us that there are still hills to climb, “Some scientists may be extra skeptical of results from my lab for a long time to come.” For example, in a critique of our submission, one reviewer asked, “how do we know the strains weren’t mixed up again this time?”

We all know errors are part of the process of scientific discovery (although most of us don’t see ourselves as making these mistakes). The key is to track them down as fast and efficiently as possible, ideally before publication. To this end Pam has improved and instituted new laboratory practices: generating duplicate stocks of key strains (validated and maintained by the lab manager), mandating electronic notebooks for each lab member and requiring that all new assays be independently validated by three independent researchers before publication.

After all this stress and uncertainty, it is still invigorating when a colleague comes by after one of our talks and says, “It is nice to see how that you handled and communicated all the errors, corrections, retractions, recoveries, and discoveries. These stories are so important to tell as they are part of doing science.”

The new finding of RaxX as activator of rice immunity has opened many new research directions. We all are excited to figure out the biological function of this potent novel molecule. We are moving on.

Acknowledgment:

The authors would like to thank Pamela Ronald for critical reading and constructive comments on this manuscript.

Additional reading:

Retraction Watch, What do you do after painful retractions?

Nature News, Rice researchers redress retraction

The Tree of Life, A Phoenix Rises from the Ashes

 

 

Poverty outside the Ivory Tower

Beginning 2014 Tenure, She Wrote published a great post about “Poverty inside the Ivory Tower”. I would recommend everyone to go read it and appreciate how difficult it is being poor and part of academia.

How many of my colleagues are looking down on poor people and ‘bad neighborhoods’ has always bothered me. How can you be so dismissive and disrespectful to your follow brothers and sisters only because they are poor and need help?  In the Bay Area I met many academics that bad mouthed about the Tenderloin in SF or West Oakland. Sometimes I wondered if these academics are actual talking about human beings or some kind of more primitive specimen. It hurts hearing these derogative words about your neighbors, fellows and friends. I bet many never talk with anyone living in these neighborhoods.

So here is a small story to tell you how great folks are living in these neighborhoods and the sense of community they embody.

We recently moved from West Oakland to Australia. The last couple of days of our move we were giving away goods we couldn’t move and didn’t need. This included plenty of food, bags of old spare change, towels, and other small household items. Larry and other (homeless) locals would stop by and ask if we have something else to give away. Everyone was happy to be treated with respect and get a little help to get by. Some of the locals greeted my partner and wife at the local store with ‘….thanks for feeding us all’. Great feeling to actually help. So all went well till the last day. My little family already left and I had to hand over the house. Early morning I wanted to go relax a bit going to the gym before a hectic day. Well this didn’t work out as planned. While getting ready I found that two of my bags got stollen that night one including my laptop, missing half of its keyboard, some paperwork and hiking gear for my last California trip with friends of the lab. Of course you can imagine I was pretty pissed and annoyed. How could someone steal from us when we were sharing so much? Off doing several necessary phone calls. At work using a shared computer I was getting the idea to walk through the (neighbor)hood and talk with the locals to get my stuff back. I made some handwritten leaflets with a short description and a phone number. A one-hour walk through the hood ensued and I talked with Larry, the homeless at our playground and half a dozen other folks. I explained everyone that it was annoying that folks stole from us while we gave away plenty. Everyone assured me to get in touch if they found something. Of course I wasn’t really hopeful. But hey was I in for a surprise. Later the day one of the locals gave me a call that he located one of my bags. After having it picked up. He gave me another call that night that he found some more of my stuff. The next morning the guy from our playground called me that he found a bag and inquired if it was mine. Two days later the first guy called again telling me he located nearly all the remaining stuff. Each and everyone that returned something actually needed the goods (e.g. shoes, cloths, bag) more than I did. They could have kept it all, panned or sold it on. They are really in need. But no, the sense of community and respect had bigger value for them. People actually care about you if you show them respect.

So in these situations when my fellow academics badmouth about poor people I hardly ever find the courage to speak up. In most of these situations I feel people nod and agree. Folks don’t appear to appreciate their own privilege of color, race, class and/or gender rather focusing on their own hardship and achievements. But hey everyone fought his/her own struggle and most people never had the opportunities we had.

Respect your brothers and sisters, cause we are actually all in this together.

FunFunFun: Our Software carpentry bootcamp @UCDavis

Sponsored by the UC Davis Shields Library and UAW5810, the UC post-doc union

We recently hosted a Mozilla Science Foundation Software Carpentry Workshop at UC Davis. For me this bootcamp was somewhat a ‘coming to ages’ event. Organizing an event arising from necessities public institutions are obviously not able to fulfill is a continuation of what I have been doing from my early teens onwards.  Instead of a running a German-punk-rock style ‘food-not-bombs’ in a run-down Bavarian bar we rented for cheap, this time I organized an event with Mozilla Science Lab teaching post-docs at a prestigious sunny Californian university basic bioinformatics skills. Of course the audience is absolutely not comparable and total different needs are addressed. One might be even more important than the other. Yet in the core organizational scheme they are the same. People volunteer their time and talent to create a space in which a commodity is shared and enjoyed by everyone for as close to free as possible.  Sharing ideas, food, company and everything else without being concerned about turning maximum profit is such a liberating feeling. Now and than. Back than in the mid-90s I got to know what the world-wide-web was. One day we hosted an event by a guy from ATTAC, a leading group from the so-called ‘Anti-globalization’ movement. /// A simple fact. Much to be criticized about this movement with loads of great motives./// I was asking him how to best stay in touch and he replied ‘give me your address’. Young, blue-eyed and naïve as I was (and still am) I wrote down my physical address on a bierfilz (beer coaster) and handed it to him. He just stared at me and said ‘no your e-mail address’. I felt so 80’s.  Later that day I went to an Internet coffeshop got my first email address using Firefox the Mozilla browser. Now at UC Davis hosting this software carpentry event I felt more grown up, organizing an event that enabled me to teach myself more easily the skills I and many of my fellow post-docs need for the scientific endeavors in the 21st century.

But what about the ‘hows’ and ‘whys’ of the workshop?

Main Organizers:

Jonathan Catchat, post-doc from the UC Davis library, @jcachat

Benjamin Schwessinger post-doc in plant pathology and UAW5810 UCD head Stewart, @schwessinger

Main Sponsors:

UC Davis Shields Library and the post-doc union at UC, UAW5810

Motives to organize this event:

Both UAW5810 and UC Davis Shields Library are dedicated to provide post-docs and the wider research community at UC Davis with the resource required to perform the best research possible. Bioinformatic literacy is a basic skill beneficial for nearly all research disciplines. Yet adequate affordable training is lacking much behind the demand. Our workshop for 80 people sold out within 10 days nicely illustrating this fact.

How we organized this event:

I put out the idea to organize such an event after hearing about SWC at the “Publish or Perish” workshop at UC Davis. Somehow Jonathan got in touch. We organized the event mostly by emailing back and forth with SWC organizers, talking with folks at UC Davis Shields library and us two meeting in person several times.

Main bottlenecks:

Getting a room -> was easy as UC Davis Shields library offered two excellent rooms

Getting instructors sorted -> took a while but all went perfectly fine and smooth at the end

Providing food and coffee for the event -> all went well as many people helped out and made it happen //Thanks to Randyman,  AmazingJeff, AstroBen, MosswomanJessica…..///

Awesome Instructors ///in random order///:

Naupaka Zimmerman, @naupakaz

Bernhard Konrad, @BernhardKonrad

Andrea Zonca, @andreazonca

Dav Clarak

Amazing Helpers  ///in random order///:

@kseniakrasileva, @CieraReports, @davidjayharris, @noamross

Friendly people helping out:

Jessica, Randy, Ben, Jeff, Aaron, Jeff, Allison, Pat and many many other helping hands I cannot remember right now.

Summary: Awesome super helpful event for everyone. Start of great collaborations and discussions, e.g. PloS Blog about R markdown.

Future: Many more of these SWC bootcamps sponsored by UAW5810 at other UC campuses. Working together makes a better place for everyone.

Thanks SO much to everyone who helped out and made this event happen!! That many more will follow.

Behind the Scenes: The people who debunked Ax21

A few months ago I wrote a blog post highlighting  the two retractions  that took place in our lab ///one Science and one PloS One///. The topic has since attracted much attention, with articles written in The Scientist and Scientific American. Pamela Ronald and the rest of us received much praise for how we handled the situation and strived towards setting the scientific record straight. We even made it into the Top 10 Retractions of 2013 -> In a good way!!! It was certainly a very long and challenging endeavor (18+ months) for all of the actors that were a part of this journey. For sure, at times it was more like a nightmare inflicting both scientific and personal scares, though eventually we exited the once long and dark tunnel stronger and as better scientists.

However, who was the driving force behind all of these scientific corrections? Who are the scientists who figured out that there was something wrong in the proposed role of Ax21 in the bacteria as well as in activating rice immunity? For sure, actual people must have done the work, inspired each another and diligently followed the data.

This answer can be found in our recent congenially communally co-directed PeerJ paper ///An excellent ‘journal’ that we can only most highly recommend to everyone///. The main actors in debunking Ax21 are named as authors and include the following:

Ofir Bahar and Rory Pruitt as experimental, intellectual and conceptual leads.

Dee Dee Luu, Benjamin Schwessinger, Arsalan Daudi, Furong Liu, Randy Ruan, Lisa-Fontaine-Bodin and Ralf Koebnik as support cast, contributing immensely intellectually, conceptually and experimentally.

And, last but not least, Pamela Ronald as scientific producer.

Unfortunately our rice expert support cast Mawsheng Chern did not make it on to the paper due to highly variable scenes in the pre-treatment shots. Nonetheless he contributed intellectually and conceptually.

This last part of the Ax21 story published in PeerJ was also featured on Retraction Watch, where Pamela Ronald accurately answered some tricky questions. In case YOU have any questions left please come and join us at our ‘Ask Me Anything’ session with PeerJ on February 11th at 9:00-9:30 am PST ///link tba///.

In summary we are very excited this piece of data was published. We are happy to set the scientific record straight about the role of the bacterial protein Ax21 in rice biology. We hope other people, who have been working on orthologous proteins, will follow suit. To make life easier for everyone, we also corrected our Annual Review in Plant Biology article. We reasoned unless it was fixed, it might become really confusing for everyone starting to learn about rice immunity by reading a recent review from our lab.

Finally!!! We closed this chapter! Dried out the swampland!

We NOW look forward to being the architects of progressive everlasting castles.