Our paper, Selective schooling and health: a natural experiment, is the first Registered Report published in BMC Medicine. Registered Reports are a new way of communicating research that is better, in our experience, for both science and for scientists. This format allows the study to be designed, peer-reviewed and published while the study results are not yet known to those involved.
To achieve this, researchers draft a detailed analysis plan before collecting data. This plan is then submitted to the journal, along with an introduction explaining the motivation for the study, which is then peer-reviewed as normal.
Once reviewers and editors agree on the rigour of the design and usefulness of the study, the manuscript, and therefore the project, is given an in-principle acceptance, at which stage, data collection begins.
When data collection and analysis are complete, the authors submit a second version of the manuscript, which is then peer-reviewed again by the journal. If there are no significant variations from the initial approved method, and the findings are adequately supported by the data, the second version of the manuscript is then published as a Registered Report.
This was our first Registered Report, and while it was very different from submitting a research article, we certainly recommend it.
Compared to our usual experience in writing a ‘standard’ manuscript we spent considerably more time at the beginning of the study, designing the methods, and substantially less at the end while re-writing the final version of the text we intended to submit.
Successfully publishing a Registered Report hinges on the quality of the study plan, meaning authors need to provide a very detailed and structured idea of how they intend to perform their research.
For instance, we provided specific participant inclusion and exclusion criteria, what variables we measured and derived, every hypothesis we would test, and numerous other specific details concerning our study.
We recommend using a simulated dataset to create a functional analysis pipeline. Actually performing the analysis is helpful to understand where key decisions should be taken, and where contingency plans should be drawn.
Performing a thorough literature review is essential during the early stage of write-up, as the introduction section of a Registered Report cannot be changed after in-principle acceptance. This is one of the core aspects of the two-stage submission process, where the initial idea and methods must be consistent.
Throughout this experience, our impression was that Registered Reports would be a better format for science, allowing researchers to be rigorous in their experiments and encouraging the publication of null or ‘insignificant’ results.
We were ultimately surprised by how relaxing, even joyful, it was to do research knowing that reviewers had improved the analysis and publication was assured.
We found that working with reviewers felt less adversarial and more helpful than peer-review can sometimes be. Reviewers’ suggestions helped improve the analysis exactly when it should be changed – before it begins.
Knowing the first version of the manuscript was accepted removed all the usual worries about whether the results would be significant, which journals might consider the paper, if reviewers would require substantial additional work before acceptance, and how long it might take before publication.
Interestingly, like many Registered Reports, the results of our study were negative – we did not find a significant health difference in people who attended a selective school, making the assurance of publication even more important.
Currently, 242 journals offer Registered Reports, with BMC Medicine being the first medical journal to do so. Guidance for submitting a Registered Report to BMC Medicine are here and a wealth of additional guidance can be found on the Center for Open Science Registered Reports page here.
Our experience in writing a Registered Report for BMC Medicine was excellent. The research was held to a very high standard of rigor and the publication was far less stressful than usual.