In the following post, I chat with researcher and F16 Fellow, Girija Goyal about the reproducibility question in open science, the challenges of working in resource-limited labs in India and the power of social incentives to foster collaboration between junior researchers. She also tells us about the exciting project she is pitching at this year's F16's Innovation Challenge and how it will help level the field for researchers around the world. Follow her work on twitter: @scimpact_org

As our conversation winded down, I asked Girija to recommend us a nonfiction work that recently inspired her. She took a few seconds to browse her catalogue until she finally went for: ‘Originals: How Nonconformists Move the World’; a book that explores how leaders build alliances, manage risks and nurture cultures of resistance. Her choice, besides giving me a quick glimpse into what a Ph.D. in Biomedical Sciences does to unwind in the evenings, very accurately captured the essence of her work and her refusal to be bound by the norms of the scholarly community.

Her nonconformity manifests in the many battles she is waging. Born in India and currently working in the US, Girija understands the barriers experienced by early career researchers and how they limit their ability to compete in the global scientific community. Her solution – leveraging social networks to bring researchers at level with each other and create a system where scientists around the world can get recognition, credit, and rewards for the rigorous work they put into all stages of the research cycle, regardless of their seniority level. 

Along with her colleague James Akin, she will present their project the MiniReproducibility Project at FORCE16: a highly interactive, social platform built to crowdsource data from reproducible experiments, foster collaboration and give visibility to early career researchers. Through a gamification based ladder of incentives, users will be able to compare results, insights, and methodologies, breaching the communication gap between what happens in the lab and what makes it to the final paper.

In the eyes of Girija, impact needs to go hand in hand with a sustained, self-reflective conversation about what works – but also about what does not work. One part of that discussion revolves around understanding how informal knowledge sharing can help us produce high-quality, reproducible and impactful results – but the other is about taking a step back, looking at the bigger picture and recognizing opportunities to make structural change. Here’s to starting the conversation at FORCE16.

On Reproducibility and Open Science…

What is bringing you to F16?

The work bringing me to FORCE16 is work I initiated on the side with a fellow graduate student, James Akin (Joe), called the MiniReproducibility project. We wanted to capture the same themes as Brian Nosek/COS’s Reproducibility Project, but as you will see, our project is about capturing reproducibility at the single experiment level (hence the “mini”) rather than checking reproducibility after studies are published.

What is the problem the MiniReproducibility project is trying to solve?

Joe and I were in the final years of our Ph.Ds, when we realized that after all the work we had done, our data would only be publishable if we tested the therapies on mice and if they were effective. (That is the model in translational biology -you come up with a hypothesis, reproduce published data to make sure it is correct, do a series of preclinical experiments, to then finally test them on mice). [In the end], you only hear about what works and not about not what did not work.

Joe and I got together and started breaking it down into discrete problems. We realized we had spent years reproducing data that we had never communicated to other people. Even in our university’s community, we would bump into other people, and they’d be: “Oh, but we already tried that”. We had negative data that we wouldn’t and couldn’t communicate. Some of this would be included in that final paper, but that would be years after the experiment. There was a 1 to 5 years lag in the communication of the results.

That’s when we came up with the project, as an initiative to build a platform where we could crowdsource reproducible experiments across multiple platforms (*like Figshare). We started with reproduced data, but it can be expanded to negative data.

How did you connect your project with open access and open science movements?

We got involved in the open access community because they emphasize the parallel theme [of access] – We not only wanted to address these problems, we wanted our solution to be open access.  We started talking to people, looking it up on the internet and participating in conferences.  We no longer accept that if and when we communicate our data, it is accessible only to people that can afford to pay for it.

On How the MiniReproducibility Project will Support Junior Researchers…

What is different about the MiniReproducibility Project compared with what is already out there?

A lot of people like me have started submitting their data and their experiments to open access, single figure, publishing platforms (like Figshare, dryad) but what happens is that this publication does not equal impact, it doesn’t equal conversation. All of these reproductions, by themselves, are not considered worthy of publication because they are reproductions of a single experiment. What we are doing through the MiniReproducibility Project is providing a home for them so people can look them up and see how many scientists reproduced an experiment, if everyone got the same results and what the differences in the methodologies were.

Take me through the MiniReproducibility Project platform, step by step:

We encourage people to submit their reproduced experiments to single figure publishing platforms, and miniReproducibility Project comes into play after that. Either you or anyone else who comes across your data can tag it for our platform to [indicate] if it is a reproduction of a DOI or figure from a DOI. Our platform will display all the reproductions of that figure on a single page.

The MiniReproducibility Project relies on people coming in and submitting their data, reviewing it, making sure there is no missing information and that it is a well-conducted experiment, so we are envisioning a user ladder to incentivize the process. The more you engage with the platform, more gratifying it is, more your voice can be heard and the more you get credit for peer review, which can strengthen the CV and improve career prospects of a junior researcher.

Tell me more about this ladder of incentives. I understand the platform is largely based on gamification principles – how will this lead to the consistent participation of users?

We heard about the struggles of traditional publishers to change the business model. Users need gratification for an act, to come back and do it repeatedly. As a social platform, we are unable to offer people money for it, so we asked: what can we give to our contributors, to make them want to contribute more and make them want to sustain a conversation?

First, we can give them recognition for their skills. The reproduced experiment may not be traditionally publishable, but perhaps the researcher spent months on it and put a lot of effort into it. With miniReproducibility Project, the researcher can be recognized for the fact that they know a technique and that they reproduced someone’s data.

Second, we wanted to give them recognition for interacting with the platform -this is where the idea of having “superusers”, by having users climb a social ladder of recognition and prestige came from. Whether a user has interacted with the platform 5 times or reviewed 100 experiments, we want to reward that.

Lastly, we realized that it is gratifying to have a group listening to you, so we will be adding a social media component. The platform is targeted for people who understand social media platforms and that when you make a comment, people will [respond] -so you need a thicker skin. A certain age group will be more comfortable with this than older scientists.

Who will benefit the most from this platform?

Our project helps junior researchers. If you are in your early years as a new faculty, postdoc, or as a new graduate student, you want to start a conversation about your findings and can publish them even on a daily basis. Senior researchers might not be able to spend the time and are not on the bench anymore, so maybe a single experiment doesn’t matter that much to them – but it really matters for junior researchers. If they are writing a grant, they can say that their data has been published on this platform and [show] that people are talking about it.

On Leveling the Field for Researchers Around the World…

How has doing research in India influenced your views on access to knowledge?

Coming from the developing world, I felt bad about not communicating my data. To put it in perspective, very few international students get into Ivy League programs. For my community, my school, my M.Sc colleagues, it was a big deal I was coming to Harvard -a big dream come true. Then I got here and did years of work, and realized that only 10-20% of it is ever going to get out there, and if it doesn’t go out there, it is not going to make an impact.

Secondly, as a researcher in a well-funded lab in the US, if I order something from Sigma-Aldrich (as a hypothetical example) today, I get it tomorrow. In India when we ordered things, it would take 2-3 months for them to arrive. Also due to the difference in currency, we were very limited. An antibody of 100-300 USD was a big thing for us. We would think a lot before ordering it. In resource-limited settings, sometimes you can’t tackle the big questions and thus, cannot compete to get results that are as fast, as impressive, or as sensational as in the developed world.

In what ways can the MiniReproducibility Project help address these barriers?

It will help researchers in developing nations in two significant ways. Researchers in countries like India are doing very good, rigorous research, but since we only reward the scientific endeavor by that final paper, these researchers don't get credit for it. That is one aspect. If you give credit to researchers for what is at the core of the scientific endeavor: rigorously designing and communicating an experiment or reproducing data, then you bring researchers from all over the world at level with each other.  

Second, in our final vision for our platform, we envision nodes of communities -pioneers, people with an activist bent of mind, committed to open access, who will make mirror sites of our platform for their region in their own language. So that in a few years, when the platform is up and running, basically everyone can replicate it around the world.

Have you experienced resistance when you bring these points up in global forums?

I don’t think anybody denies that things need to change. However, very often I find that established scientists, once their career is “made”, no longer feel the need to participate in the movement.

On Force16…

What are you looking forward to seeing at F16?

In my normal day to day, I am a lab rat *laughs*, I don’t get to talk to people about these topics beyond Twitter or email. These conferences are an outlet to dig really into the themes. It’s time for me to get my fix of open science! It will be interesting to be in the same room with the publisher who wants to charge for publication (the more I read about it, I understand they have a reason to), at the same time as open access advocates. I’m looking forward to that.

What about 2016 makes it an exciting time to have this conversation?

This year has been a time of convergence, where a lot of people from different fields are coming up with tools and ideas. There is a sudden ‘coming out of the closet’ effect where people are talking about open science, reproducibility, all at the same time. There is a lot of conversation going on. It is an exciting time to be in the middle of it all.