Starting again with scientific publishing: Q&A with Alex Freeman

Following our recent interviews on the state of scientific publishing and Open Access in light of Plan S, we take a look at a new initiative that aims to revolutionise the publishing model for primary research.

‘Octopus’ is the brainchild of Alex Freeman, Executive Director of the Winton Centre for Risk and Evidence Communication at the University of Cambridge, who came up with the concept in 2017 in response to a question from Charles Ebikeme, then Science Officer at the International Council for Science (ICSU),who asked “What would you do if you could start again with the way science is done?”

How did you get involved with scientific publishing? Where did the idea for Octopus come from?

After studying biology I worked in the media for 17 years, making documentaries about science and natural history for the BBC. When I came back to academia at the end of 2016, I found myself working with postdoctoral psychologists and I realised how things had changed: their careers were critically tied up with what and where they published, and their storytelling abilities. They collected lots of good data, but what mattered was the story they could tell with it. At the same time, through the UK’s science media centre, I saw how its expert panel was questioning how some papers in high impact journals had even got through peer review.

All of this was mulling in my mind when Charles came along. I realised that scientific publication was at the root of many of the problems I was seeing. We were treating scientists like journalists: they were being judged almost entirely on their publications and where they were getting published. That’s not how science should be: it doesn’t have to be a good story.

Octopus was my way of trying to design a scientific communication system built entirely around the best way to do science. That meant emphasizing and rewarding the things that are important for good scientific practice – reproducibility, lack of biases, fast sharing of ideas, strong collaboration, constructive reviewing.

What exactly does Octopus do?

To start with, it’s free, it’s accessible to everybody, and it has built-in language translation. One of the fundamental aims is to try to make science accessible for everybody. The internet has given us a whole load of new tools to do that, and they’re accessible pretty much wherever you are in the world.

Octopus, though, also tries to tackle what I see as the fundamental problems with the way scientific publishing currently works. It breaks down the unit of publication into the sort of units that science actually comes in. At the moment, in order to write a paper you have to do all the work – from having an idea, doing the experimental design, collecting the results, analysing them and coming up with some conclusions – before you share anything. That makes it very slow, secretive, and it also means you don’t get feedback early on, so you might have a design flaw which nobody points out until right at the end. It also forces people to put a linear narrative onto their scientific process when it’s simply not a linear process. Many of the questionable research practices that go on do so because people are trying to fit what they’ve done to a story. If you’re not forcing people to tell a story it doesn’t matter what the data looks like, well-collected data is good data. Hypothesises, methods and data are all good for different reasons, and so I think we should publish those individual units of science as we do them. Octopus is designed to encourage that and to recognize good work at each of those stages.

What reactions have you had from the research community?

It’s been really positive. So many early-career researchers have told me that they have many small data sets or hypotheses that they’re just desperate to publish. People are also saying it takes months to write a paper when all they want to do is publish some new results. The only negative feedback is from researchers who are under so much pressure to get their next grant, or their next position, that they can’t think about anything else.

The challenge will be flipping from the current way we work to the Octopus way, and I will do everything I can to try and help people flip to this new way of working. Everybody agrees that it’s a far better place to be – we’ve just got to get there together. The Octopus system is designed to help people get there. You’ll be rewarded for getting things out more quickly and you don’t have to write pages every time you want to publish new data, so it should make everybody’s lives easier. It should also be easier for institutions to judge the genuine quality of work. Octopus has an inbuilt rating system, so when you’re ready to publish, you can publish instantly and then your work is reviewed and rated by anybody who wants to. And good reviewing is incentivised – a review counts as one of the eight types of publication.

How?

Anybody logged in can rate a publication on three pre-set criteria – exactly like when you rate your experience on eBay or Amazon. As a scientific community we’ll be able to set what we think are the criteria that define ‘good science’ across each different type of publication. For instance, for publication of results the criteria might be: size of dataset, how well presented the results are (how clean, how annotated, how reusable), and how well researchers followed protocol for collection. The ratings will give anybody an up-front view of the quality of that piece of work. Individuals will each have a page listing all their publications: the types of publications, and their ratings. As an institution or a funder looking to hire or to promote, you’ll be able to see what type of researchers people are.

How do you plan to scale up? How will it be funded in the long run?

At the moment we’re trying to build the technical base, and then I’ll be doing masses of user testing. It will be open source, and will allow all the other brilliant open source technologies to plug in to it so that it’s a collaborative venture, which allows people to develop new functionality. That minimizes the cost of development. The academic community is full of brilliant coders and people who are enthusiastic about designing new metrics and visualization tools. It will have minimal running and development costs because it’s a collaborative open source project and as automated as possible (I want to get rid of the old hierarchical editorial board structures too). In the longer-term, I’m hoping that it will sit on top of a distributed database so that institutions can host the storage they want. Running costs should be minimal, but I’m considering asking a small ‘donation’-type fee for the larger institutions – far smaller than what libraries currently pay for academic publishing.

Who can register to post comments on Octopus? Is there any barrier to entry?

We’re going to be using ORCID as our login system, as we want it to be open to people whether they’re retired, working in industry or in academia, but not just anybody. I think it’s very important that there are no anonymous logins. Octopus is a professional communication platform and all the ratings you make of other people’s work will be logged so that all your future funders and potential employers will be able to see them. There’s a built-in system for red-flagging suspected misconduct or plagiarism, which will give authors a chance to reply, but if it doesn’t get sorted an email will go to their institution’s research integrity office.

So access is restricted to a scientific audience?

Anybody will be able to read for free; publishing will be restricted to authenticated logins, and so, therefore, will the reviewing system. I’ll be testing whether things that have not yet been reviewed should be marked as such (the equivalent of preprints). As the first ratings come in you’ll be able to see how many people have read and reviewed each publication.

I think it will be a better system because at the moment you have to rely on peer review without knowing who reviewed an article before it was published, and what they said about it, so there may be huge problems in a paper that weren’t picked up and you will never know. I think it’s really important to have open reviews where you can learn from other people’s expertise and experience. Authors will also be able to re-version in the light of reviews, but the old version will be archived. I think it will make the system more up-to-date and more reliable.

What about people who already have very strong publication records and perhaps don’t need to worry so much about promotion – do you think they’ll need extra incentives to join?

I see users at both ends of the spectrum: at one end the new people who want to publish but can’t afford to do so, and at the other, people who are not under the cosh of funders and institutions, who want to publish somewhere where their work will get maximum readership, and to share their expertise by writing reviews of other people’s work. Many eminent researchers went into science to do just that kind of work, and they’re keen to have a platform where they can do so and can help people. And they’re fed up with writing very long papers.

One of the early interviews in this series points out that 100 years ago the national academies and disciplinary associations used to be more involved in peer review. Are you speaking to learned societies?

That is something I must do. There’s a real opportunity for journals to reinvent themselves – especially journals published by learned societies. When I first thought of Octopus I thought it may put journals out of business. Now I think that’s wrong. Octopus is the new home for primary research, but unless they’re actually involved in that area, most people don’t read primary research. What most people want to read is an editorialised version – expert opinion, or a summary – and that’s where journals can come back into their own. This is especially true for the journals of the learned societies, who can commission that kind of content aggregation and editorialization for their readership.

In ten years’ time, what do you expect will have changed with regards to OA and what do you hope to see?

Ten years is a really long time, and I’m very optimistic: I really hope that I can get this off the ground. Culture change can happen really fast. In the online world, when something comes along which allows you to do your everyday tasks more easily it sweeps the world in months.

So many people want this to work, so I’m confident that if we can get the technical build done then it could sweep academia quickly. Then we will all be publishing and reading primary research for free online, immediately, language agnostic. There’s nothing stopping us doing this.

[related_items ids=”7664,7411,7679,8172″]