Deposit full protocols early!

Disclaimer: I am an ambassador of the protocol sharing website protocols.io. Yes I got an $50 amazon voucher from them end of last year with which I bought some books for my son Saul. I think pre-prints are awesome but have some caveats.  And foremost I love free stuff accessible by everyone.

Prelude

I am currently working on long-read genome assemblies of fungi and plants. All successful long read genome projects start with months of optimization for DNA extraction methods. This can often involve phenol chloroform extraction for weeks on end with slight alterations of the same protocols. Mine took about 9+ months to optimize and to finally get some good data back.  I was lucky as  I got three protocols to start with form colleagues around the globe. Looking into the scientific literature was no help at all. The one I build on was from INRA and put online by Jason Stajich here, which I only found because I asked Andrii Gryganskyi for help. So overall I didn’t reinvent the wheel but simply build on what others already created previously. Pretty much was 99%+ of all science really is. Notwithstanding,  I knew several other groups struggled with the same problem so I put my working protocol online on protocols.io and publicized it on the 18th of April 2016.

Now on the 31st of August 2017 we published pretty much the same protocol as a peer reviewed methods chapter in a book a friend of mine edited.

I would like to celebrate this occasion by sharing some thoughts about open sharing of methods early. Overall I obviously think it is advantages for you and everyone else in science to do so. This is in contrast of letting one’s protocols rot on one’s harddisk and to only let them out to see the light of the day once someone offers you a co-authorship.

In semi random order.

Instantaneous availability

The most obvious one first. It took 17 months to publish pretty much the exact same protocol I knew was working when I put it up online. That’s nearly half my son’s life. In the meantime the protocol got 2375 views, I got 10+ emails or messaging requests to explain different parts of the protocol better and some ‘Thank yous’ on conference as other got it to work for their favorite species as well. Overall putting it up online early enabled other to discover more in in less time instead of reinventing the wheel of DNA extraction protocols.

I know some people would not use none peer reviewed protocols that they find online (discussion here and here). I personally think it is an illusion to believe that methods get properly reviewed in papers. I only ONCE got a comment on a small technical detail in my methods section across all my 34 publications. Similarly, I’ve hardly ever (never?) seen another reviewer ask for details or even comment on the methods section.

I know the protocol worked for me and no closest peer review would have changed it.

Discoverability and ease of use

Everyone loves detailed easily accessible protocols. EVERYONE! Everyone despises short methods section in the supplement that provide no detail and cite another paper that cites another paper that cites another paper that cites another paper that supposedly did the experiment the exact way the authors did it.

So putting your full detailed protocol on your lab webpage is good (see this awesome long read DNA extraction protocol) yet putting it up on a publicly accessible and search-able platform like protocols.io is even better. You get a DOI, it has versioning, people can fork it, people can comment on it, people can ask you directly, it is backed up available forever, and you can even add it to collections/usergroups so even more people can discover it. Everyone is happy using detailed protocols they can interact with and cite. Everyone!

Catering to both systems

Of course some people get nervous and fear they cannot publish their methods in esteemed journals anymore. Of course not everyone goes online on novel platforms like protocols.io to hunt down the latest protocol. Many prefer to search the peer reviewed literature. And that is really the great thing about depositing full methods early. You can do both.  You can share your protocol to help others quickly and get the bonus points of a journal publication. That is just what it did! It might well be that more people will cite one’s publication as they already used one’s protocol before it was officially published in a peer reviewed journal. That is the case for preprints and future studies will show if we can observe similar effects for protocols protocols.io. For sure I will cite both protocols.io and my book chapter in my next genome manuscript preprint.

Reproducibility and rigor

There is a lot of talk about the lack of reproducibility in life sciences. Much of this is focused on depositing code online. Not much is focused on the actual data generation. Methods section are often poor and most supervisors don’t care a la ‘just write something no one reads it anyway’. Yet every good project starts with good data, starts with a solid detailed protocol of how things are done and were done. This not only helps others to reproduce one’s work but also oneself and one’s lab. Writing down great protocols often makes one realize the fine grained and important details. Sharing those protocols publicly is even better as it accelerates discoveries, saves money, makes science a better place for everyone. On top we in the western world sitting at elite universities often forget how hard it is to access high quality protocols. One just need to look at researchgate to see what I mean.

Come join us, make everyone happy, and share your detailed protocols online early. Imagine a world were you just have to search for a detailed protocol and you find one pretty much immediately. Just like cooking recipes.  

Both protocols.io and bio-protocols are a great starting point.

 

 

Advertisements

Today I reached my a-index

Today I finally reached my a-index. Yes I just made this up to celebrate my 34 accepted publications at the age of 34.

No a doesn’t stand for a*8hole but for age. The a-index is achieved once you reached the same amount of publications as your age. The convenient fact about this new amazingly made up index is that it only requires one publication per year once you reached it.

So I guess I can take it easy from here on and focus on getting some science done instead of hunting yet another index.

The #FOR community is named Science People of the year

I am ecstatic. For real this is amazing! An awesome achievement. We, the Future of Research community, were named  ScienceCareers People of the Year!

Oh yeah! This feels SOOOOO good. All of use put tonnes of payless volunteered hours in FOR and other important projects aimed at improving the academic enterprise. ScienceCareers put it nicely

“Each December, Science Careers names a “Person of the Year” to recognize those who have made especially significant and sustained contributions to improving the lot of early-career scientists. Up to now, each year’s honoree has been a senior academic figure occupying a prestigious post. This year’s choice, by contrast, is plural; of a different generation; and at the opposite end of the academic status ladder. For their dedicated, creative, and expanding efforts to empower early-career and aspiring scientists with knowledge and awareness so that they can take control of their futures and help bring needed change to the scientific enterprise, we are delighted to name the activists of the Future of Research (FOR) movement as the 2015 Science Careers People of the Year.”

It is great to see an actual community effort by young upcoming early career academics rewarded by an world renowned science stable. Sometimes it simply feels good to get some back patting from the elite. Clearly, FOR got people active and the bigger academic community talking about issues important to the academic enterprise (e.g. diversity). We also create community by bringing together people with the same interested in improving our  and other peoples’ lives. I see this award not only as a recognition of past achievements but as support of our future aspirations. Folks believe that change is possible and there is a opening in history to make things happen. Hopefully FOR will become a stable itself throwout our careers and I meet folks again.

Thanks to each of the many countless people that make this happen.

On a personal note, this really speaks to my heart and soul, because the award does not single out an individual established person but a community. Only by working together science progresses, this is true within the lab and outside. This recognition is soothing to my soul. Like many other I met over the years [15+], I have been volunteering for all sorts of community actions within and outside of academia for quiet a bit. Many of us spend countless hours on planes, on hangouts, in meetings… organizing events that are not really accounted for in the status quo career progression pathways. Yet we do it because we believe in it. We think it makes our communities a better place.  While seeing things we created happening and community forming we often get wet handshakes and some muttered ‘well done’. So having the elite say “Excellent Job!” loud and in the open is nice!

Anyways, I am sure many of us would keep on doing our projects nonetheless. Yet recognition goes a long long long way!

 

P.S.: This early-mid career conference at the Australian National University is our next local project spanning the globe.

Discovery and Redemption emerge from a scientific mistake

///This is a slightly delayed post as it was lost in the labyrinth of popular science glamor mag editor desks. We are finally moving on and decided to self publish instead.///

///Authors:

Benjamin Schwessinger, Ofir Bahar, Anna Joe, Mawsheng Chern, and Rory Pruitt

Internal editor:

Pamela Ronald

///

We were confused and perplexed. Our team in the laboratory of Pamela Ronald at the University of California, Davis could not reproduce previously published results. Instead of building on the reported discovery that a microbial signal triggered the plant innate immune response, we were thrust into trying to figure out what went wrong. It was the most difficult time in our careers. Then, we made a remarkable new discovery.

The story began in the 1970s, when Professor Gurdev Khush and colleagues demonstrated that a wild species of rice was immune to most strains of a serious bacterial disease. This was exciting because such broad-spectrum resistance had not previously been identified. Such traits are agronomically important because farmers can plant resistant varieties rather than spraying pesticides. In 1995, Pam’s lab reported that the broad spectrum resistance was controlled by a single gene, called Xa21, predicted to encode a receptor that senses the microbe and then activates the rice immune response.

In 2009, Pam and her former team members reported the discovery of such a molecule.  However, in 2012, while trying to build on those findings, we discovered major errors in this work. Pam contacted the editors to inform them of the identified problems. She also decided to notify the scientific community of the issues at an international research symposium. Ben still remembers the day when Pam announced the errors in front of so many distinguished colleagues, many whom Pam had known her entire career. “It was remarkable that Pam mustered all this courage to inform the community at this top research symposium. Some in the audience buried their heads in their hands and were clearly uncomfortable but still more people were awed and expressed their support.” Sophien Kamoun, a leader in the field of plant biology, displayed his respect in a tweet:

Finally, in October 2013 we had accumulated sufficient new experimental evidence definitively proving that key aspects of the study were incorrect, Pam and her former colleagues that co-authored the 2009 study retracted the original Science paper.

Pam and the several of us in the lab lost many nights of sleep as we racked our brains to figure out what went wrong. We had many intense discussions on how and why this could have happened. As Ofir remarked, “Pam was on sabbatical in France when we discovered the problem and so we had these electrifying video conference calls to bring everyone on the same page.” It took time and persistence to break down the problems one at a time as we were working backwards identifying what were solid results we could build and what we could not. One day Ofir and Ben were having their morning coffee outside in the Californian sun when Ofir announced that he had found that some of the bacterial strains used in the previous study were mixed up. Rory still remembers this day vividly, “Even though this was not happy news, we were so happy, we finally had a partial explanation of what went wrong.” From that day onwards we were getting back to solid ground. The scientific process had pulled us through and pointed us in the right direction. Our team discussed the mistakes and corrections in lectures, blog posts and in a scientific publication. The process with which we addressed the problems was highlighted in an article in Nature magazine and in Retraction Watch, a blog that reports on retractions of scientific papers, as “Doing the right thing”. In a dubious claim to fame, the 2009 retraction was included in the top 10 retractions of 2013.

While we were notifying our colleagues, discarding old strains and correcting the literature – an essential part of science so that others do not waste (more) time trying to build on fatally flawed work- we were also working hard to discover the microbial partner of XA21. In the mist of all the fog Rory identified a new bacterial mutant strain that was able to infect XA21 rice plants. We were excited but cautious at the same time. Could this really be the correct mutant lacking the microbial partner of XA21? The 15+ year lab veteran Mashweng was especially skeptical as he had lived through the whole story form the beginning. He often suggested critical controls and reminded us to be extra critical with our own results. A superb team of collaborations from around the world allowed us to incorporate additional experiments and independent controls perform in former critics’ laboratories. After many independent tests we were certain of our results. In mid 2015, we reported the identification of this long sought after molecule that activates XA21-mediated immunity in Science Advances.

We named this small microbial protein RaxX. We found that bacteria that lack RaxX are able to evade detection by the rice XA21 immune system. Bacterial strains found in farmer’s fields in India, which express alternate versions of RaxX can cause disease on XA21 rice plants.

Wrestling with trying to reproduce flawed experiments and discovering the new molecule in rapid succession was an enormous challenge. In our view, the key point in straightening out such complicated and delicate situation was the persistence and collaboration of our laboratory team. Pam worked hard to keep the team together. We were a dedicated crew of senior scientists, technicians, postdocs and graduate students (as well as former team members who shared their records with us because they also wanted to know where they had made mistakes) motivated by a supervisor determined to get to the bottom of the situation. Looking back, we find it stunning how this all worked out. “It’s amazing what can happen if you are doing science with a set of respectful and likeminded people. We were focused on setting the record straight rather than blaming others. It clicked. We were on a run”, Ofir still remembers to this day. We felt our careers were on the line. We realized that there was no moving forward without first going backwards. We did not give up even though at times we wished we could. Working together on this daunting challenge buoyed all of our spirits. “Back then, I [Ben] was living in this lovely trailer with a trellis outside. We had ‘poker-nights’, when everyone came along to talk and be jolly. At times it felt like ‘group therapy’ with drinks.”

We were impressed by the supportive and kind response from the scientific community, our editors, and funding agencies including NIH, BSF, and HFSP. We received many letters of encouragement – even from complete strangers. These conversations helped keep us going. We even had new scientists join Pam’s lab in midst of the mess. Anna Joe, a postdoc who joined the lab at the time of the retraction remarked, “Their open mind and transparency attracted me to the lab.”

Pam tells us that there are still hills to climb, “Some scientists may be extra skeptical of results from my lab for a long time to come.” For example, in a critique of our submission, one reviewer asked, “how do we know the strains weren’t mixed up again this time?”

We all know errors are part of the process of scientific discovery (although most of us don’t see ourselves as making these mistakes). The key is to track them down as fast and efficiently as possible, ideally before publication. To this end Pam has improved and instituted new laboratory practices: generating duplicate stocks of key strains (validated and maintained by the lab manager), mandating electronic notebooks for each lab member and requiring that all new assays be independently validated by three independent researchers before publication.

After all this stress and uncertainty, it is still invigorating when a colleague comes by after one of our talks and says, “It is nice to see how that you handled and communicated all the errors, corrections, retractions, recoveries, and discoveries. These stories are so important to tell as they are part of doing science.”

The new finding of RaxX as activator of rice immunity has opened many new research directions. We all are excited to figure out the biological function of this potent novel molecule. We are moving on.

Acknowledgment:

The authors would like to thank Pamela Ronald for critical reading and constructive comments on this manuscript.

Additional reading:

Retraction Watch, What do you do after painful retractions?

Nature News, Rice researchers redress retraction

The Tree of Life, A Phoenix Rises from the Ashes

 

 

The true culture of STEM inclusivity

Disclaimer: This is a guest post by a good friend of mine, Shaila Kotadia. I fully endorse this post as it raises many important issues in academia.

As talk of how to restructure the funding of STEM fields to make them sustainable, the word diversity often arises. Almost everyone states this is a part of [insert name here]’s mission but how actively is academia really supporting this notion? In particular, over the past several decades, there has been little change in the makeup of professor positions.

I recently attended a conference organized by a scholars program for undergraduates. This program has existed for over 20 years, is successful at equalizing the playing field for students that are overlooked in and often drop out of STEM majors, and is now evaluating the parts of the program that result in this positive outcome. The morning consisted of a series of excellent speakers that conducted evaluations of programs targeted at underrepresented undergraduates to determine their effectiveness. One study conducted by Mica Estrada at UC San Francisco evaluated a national panel of minority science students for their feelings on self-efficacy, scientific identity, and how their values aligned with scientists to determine scientific integration (Estrada, et al., J Educ Psychol, 2011). This raised important questions in my mind. Are we selecting for a self-perpetuating personality type that think about and approach problems very similarly regardless of their background? Or are we molding these students through formal and informal programming to fit a model of an academician and losing their diverse perspective in the process?

A town hall discussion concluded the event. This is where we were challenged to share our thoughts that called out the academic system and its support for underrepresented students. This is when the conversation got really interesting. Comments on the culture of STEM shifting with the influx of Ph.D.’s and the lack of associate professor positions suggest that we need to adapt. Additionally, it was suggested that little diversity in these positions might be due to individuals from all backgrounds not having a desire for these positions. I felt the need to chime in. We are encouraging these students to pursue Ph.D.’s with the ultimate outcome to retain them in the STEM fields with a heavy emphasis on faculty positions. But why would we do that to them? Faculty positions suck now, in my honest opinion. And as these students progress through the different stages of academia with programmatic help, are we selecting for the same types of personalities or molding these students into what faculty positions require? How is that helping anyone? We may end up increasing diversity “by the numbers” but in the end we are losing the real benefit of diversity of thought. One interesting way to put it is that we need more cognitive diversity (Valantine and Collins, PNAS, 2015).

When I think of diverse populations and those that are underrepresented in STEM that I have worked with or mentored, their values do not align well with what academia values. Those that continue onto faculty positions may feel like they lose parts of their identity and the need to conform in order to be promoted in their career (McGee and Kazembe, Race Ethnicity and Education, 2015). That is going to continuously result in a lack of diversity, whether it be of thought or people leaving the field, despite all of the early programmatic measures.

Additionally, while unconscious bias training will likely decrease microaggressions towards others and might eliminate gender bias in hiring (Carson, TechRepublic, 2015), which is completely necessary and should be required, it appears as though we still select candidates from the same top schools that tend to attract similar personalities (Clauset, et al., Science Advances, 2015). In the end, the change in overall culture will be gradual but hiring more diverse individuals will only become more diverse with a cultural change that people must be willing to enact in the long run.

I do acknowledge that top notch research is conducted at the highly-ranked institutions but I want to also acknowledge that ground-breaking research is often discovered across all institutions. One can even argue that the same schools where postdocs are being hired from cannot employ all of them, which means that these researchers are being attracted to other programs that might offer a more welcoming environment. And there is no doubt that the people being hired are incredibly talented and yet there might well be not a single perfect match for each position but multiple. Hence many brilliant and creative candidates might go home empty handed simply because of the factor of chance.

So, I propose a different approach. Let’s change what we value when hiring faculty. Let’s change the academic environment. Not just the funding structure or implementing training for various STEM career paths or making more permanent research positions. All that is good and well but I predict it won’t increase diversity, cognitive, gender, or racial, at the faculty level. What we need to change is the way we select who continues onto academic positions. Why not use the same “pluck” criteria, or the desire to create change in a spirited or daring manner, and holistic approach, considering more than the candidate’s pedigree and publication record, that many colleges and universities use to admit undergraduates to select faculty? You might be surprised by the results.

More reading: Gibbs and Griffin, CBE LSE, 2013
Warner and Clauset, Slate, 2015

///Edited for three references 28/11/2015///

The Pickpocketing of Postdocs

These are the times when folks all over the USA and beyond are concerned about the current state of academia. Early career researchers and allies are coming together to work towards improving academia and their situation within. The Future of Research conference in Boston kicked it all off for real and now we are seeing it snowballing from New York (NYU), MW-Madison, to the Bay (UCSF) and Chicago…hopefully many more to follow. Similar debates are held in other countries such as India and Australia. Even the glamor mags (here and here) agree that something needs to be done about the current dilemma. The NIH also is thinking about fundamentally changing their granting scheme. And of course there was Tim Hunt; no matter on which side you are on everyone seems to agreed  that there are problems with sexism and lack of diversity within the Ivory Tower.

In the middle of all this falls the current UC postdoc union (UAW5810) bargaining campaign with the University of California. The content of the second contract will define working and living conditions of over 6000 postdocs in California for the next several years. In addition, this new contract will set standards nationwide. So it appears the time is ripe to actually put the money where your mouth is and support early career researchers by creating progressive work environments for everyone. Work environments in which people  from all walks of life can flourish and express their academic creativity.

Well you would have thought so but these are the demands UC came up with. For sure their are not progressive, for sure they will not foster diversity.

  Article 3 Benefits

Article 3

UC wants to restrict the current benefit plan and increase the current postdoctoral scholar’s share of the benefit. In the absence of increased pay postdocs shall take a factual pay cut for worse benefits.

 Article 4 Compensation

Article4

UC wants to change the compensation scheme including introducing salary caps, exclusion of Paid Directs and more… WhatWhatWhat postdocs shall get salary caps? Show me a rich postdoc and I show you a unicorn. Each and every time I read this line I cannot stop laughing. Seriously an institution that pays its leadership and administration mid to high six digits paychecks wants to restrict pay for ‘lowly’ postdocs. The starting salary of postdocs at UC is about $43.000 per year. Seriously? The most frequent case that a postdoc makes more than the minimum salary is when she wins her own fellowship. In this case the funding agency provides the salary. Why restrict this when it isn’t UCs money anyway? Why punish the successful and independent? The only reason I can come up with is that other postdocs might see they are getting paid jack shit and demand more. The actually postdoc salary is less than the living wage for a family in 9 out of 10 UC campuses!

 Article 11 Layoff

Article11

UC wants to be able to lay postdocs off more quickly. Well short 1 year contracts are not enough let’s make the postdoc experience even more precarious and insecure. A postdoc might get too lazy otherwise and sit in the sun the whole day long.

Article 17 Personal Time Off and Article 22 Sick Leave

Article 17 Article22

UC wants to cut postdocs sick days and personal time off. Of course postdocs are too lazy and take too many holidays. They are not productive enough and should work more. Never mind stress and mental health. Also if a postdoc gets sick and shows some sign of ‘lack of work’ UC simply kicks him out under the proposed changes to the layoff rules (Article 11).

New Article

New Article

UC wants to practically do away with the minimum contract length of 1 year and introduce a 1 year probationary period instead. Of course if a postdocs boss doesn’t like her, bad luck. He lays her off and she and her family can go back to their home country after three months and her just having invested thousands of dollars to live the Californian dream. Visas are mostly linked to a postdoc’s employment status in the US so he is practically at his supervisors good will to stay in the country. From personal experience I’ve already seen too many postdocs that didn’t dare to speak up against discriminatory actions this would make their lives even worse. Also only perceiving the threat of contract termination and mandatory removal from a foreign country is very unpleasant as everyone knows that experienced it before.

Instead of being progressive, implementing measures to increase diversity of the postdoctoral workforce and reducing the ‘leaky pipeline’ UCs demands are simply regressive. UC demands factual pay cuts for postdocs, offers worse benefits, less rights and less job security.  This all in times when early career researchers are stretched to the limit and positive messages are scarce. I hope UC gets round. UC is a great place to work, has a vibrant postdoctoral culture and loads to offer to foster a postdocs career. Making this experience of a lifetime more accessible for everyone by improving childcare support, parental leave conditions and fair treatment of all postdocs is what UC really should be aspiring to.

Poverty outside the Ivory Tower

Beginning 2014 Tenure, She Wrote published a great post about “Poverty inside the Ivory Tower”. I would recommend everyone to go read it and appreciate how difficult it is being poor and part of academia.

How many of my colleagues are looking down on poor people and ‘bad neighborhoods’ has always bothered me. How can you be so dismissive and disrespectful to your follow brothers and sisters only because they are poor and need help?  In the Bay Area I met many academics that bad mouthed about the Tenderloin in SF or West Oakland. Sometimes I wondered if these academics are actual talking about human beings or some kind of more primitive specimen. It hurts hearing these derogative words about your neighbors, fellows and friends. I bet many never talk with anyone living in these neighborhoods.

So here is a small story to tell you how great folks are living in these neighborhoods and the sense of community they embody.

We recently moved from West Oakland to Australia. The last couple of days of our move we were giving away goods we couldn’t move and didn’t need. This included plenty of food, bags of old spare change, towels, and other small household items. Larry and other (homeless) locals would stop by and ask if we have something else to give away. Everyone was happy to be treated with respect and get a little help to get by. Some of the locals greeted my partner and wife at the local store with ‘….thanks for feeding us all’. Great feeling to actually help. So all went well till the last day. My little family already left and I had to hand over the house. Early morning I wanted to go relax a bit going to the gym before a hectic day. Well this didn’t work out as planned. While getting ready I found that two of my bags got stollen that night one including my laptop, missing half of its keyboard, some paperwork and hiking gear for my last California trip with friends of the lab. Of course you can imagine I was pretty pissed and annoyed. How could someone steal from us when we were sharing so much? Off doing several necessary phone calls. At work using a shared computer I was getting the idea to walk through the (neighbor)hood and talk with the locals to get my stuff back. I made some handwritten leaflets with a short description and a phone number. A one-hour walk through the hood ensued and I talked with Larry, the homeless at our playground and half a dozen other folks. I explained everyone that it was annoying that folks stole from us while we gave away plenty. Everyone assured me to get in touch if they found something. Of course I wasn’t really hopeful. But hey was I in for a surprise. Later the day one of the locals gave me a call that he located one of my bags. After having it picked up. He gave me another call that night that he found some more of my stuff. The next morning the guy from our playground called me that he found a bag and inquired if it was mine. Two days later the first guy called again telling me he located nearly all the remaining stuff. Each and everyone that returned something actually needed the goods (e.g. shoes, cloths, bag) more than I did. They could have kept it all, panned or sold it on. They are really in need. But no, the sense of community and respect had bigger value for them. People actually care about you if you show them respect.

So in these situations when my fellow academics badmouth about poor people I hardly ever find the courage to speak up. In most of these situations I feel people nod and agree. Folks don’t appear to appreciate their own privilege of color, race, class and/or gender rather focusing on their own hardship and achievements. But hey everyone fought his/her own struggle and most people never had the opportunities we had.

Respect your brothers and sisters, cause we are actually all in this together.