Peer review: do we throw early career researchers in at the deep end?

Should peer reviewers receive training? Should there be accreditation? These topics and more were discussed at a recent BioMed Central roadshow in Sydney, Australia. Sarah Hayes, one of the attendees, gives us her point of view.

1

At the BioMed Central Roadshow I recently attended in Sydney, one of the most vocal discussions was on how peer review needs to change in order to meet the needs of the research community. As scientists and academics, we are supposed to embody the core principles behind peer review – the ability for logical thinking, and for questioning and debate. So, in theory, reviewing a paper or a grant application should be second nature – is there really a need for peer review to evolve?

Well, in my experience as a very early career researcher (ECR), my first few peer reviews were truly intimidating and definitely not easy. Not only did I question the manuscript I was reviewing, I questioned myself. Was I being over-critical in my analysis? Was I missing something crucial? Did I actually have the expertise to critically review this work?

56% of people surveyed from the 2009 Sense About Science peer review survey reported a lack of any guidance on how to conduct a review.

I’m certain I’m not alone in admitting that reviewing someone else’s work is daunting, considering the shortage of formal or informal training available. 56% of people surveyed from the 2009 Sense About Science peer review survey reported a lack of any guidance on how to conduct a review. With the number of publications increasing 3-4% annually, some form of training for peer review is desperately needed.

Mentoring and training for peer review

Anecdotally, what we see from ECRs and students across multiple STEM disciplines is that supervisors will often pass off papers for peer review to their students or postdocs, with the review then submitted under the supervisor’s name.

Results from Sense about Science’s survey also indicated few established reviewers actively train younger colleagues as part of the review process itself (only 3.2% of cases). No wonder peer review is such an intimidating task for a young scientist to undertake.

Perhaps the only change needed is the development of mentorship programs – and not just in peer review – for ECRs in individual laboratories. Many labs (including ours) have a ’Journal Club’ where junior and senior scientists come together to discuss and critique recently published papers. This informal peer review practice is invaluable for students and ECRs to hone their reviewing skills in their field.

If we could build on this and encourage labs to establish career development programs especially for ECRs and students featuring a range of issues (including the process of peer review), this would help to create more well-rounded young scientists in general.

How can publishers support the process?

A strategy publishers could adopt is peer review ’accreditation’, with supervisors reviewing several papers in parallel with their student or ECR.

A strategy publishers could adopt is peer review ’accreditation’, with supervisors reviewing several papers in parallel with their student or ECR. To assist in this task, ECRs could be provided with guidelines/checklists for each area of study with essential information for publishing in that journal (similar to the EQUATOR Network).

After feedback from the supervisor or editor, these responses could be submitted to the journal individually or as a combined report (under both names). After several combined ‘trial’ reviews, the young researcher could be listed on the publisher’s database as an accredited reviewer, who could be called upon to undertake stand-alone reviews.

With the BMC series using open peer review in its medical journals, both ECR and supervisor comments could be published openly, which would help to build the ECR’s track record and reputation in their field.

I am very much in favour of open peer review, but I realise that this might be an unpopular view to have. In fact, 58% of people in the Sense about Science survey reported that they would be less likely to review if their signed report was published. However, it has been shown that open peer-review reports are generally more constructive with a greater amount of evidence substantiating the reviewers’ comments provided. More relevant here is that open peer reports can be a very useful resource for inexperienced peer reviewers to refer to when performing their own reviews.

Deserved recognition for peer review

Finally, I think one of the key aspects of peer review that needs to change is the recognition awarded to those who peer review. The vast majority of those surveyed in the Sense about Science survey regarding why they participated in peer review said they did so because they consider it as part of their role in the academic community (90%). However, when such a task is so daunting or time-consuming, it’s natural that people may not want to peer review when there is no recognition for doing so (particularly as a student or ECR helping out a supervisor).

With many grant applications emphasising the need for scientists to be actively engaged in the research community, perhaps a rating system (e.g. a ‘5-star reviewer’ as previously suggested here) or another metric could be provided by journals for use in an individuals’ track record.

Perhaps recognition for peer reviewers should be more tangible – acknowledgment in that issue of the journal, a month’s free journal subscription, or money/journal ‘credit’ (that can be used towards future page/figure charges). Any monetary compensation probably would open a whole new can of worms (who will incur the cost: the author or publisher?), but the other low-cost alternatives should be considered closely.

Health and medical research is always changing – so, yes, I do think we will need to adapt our current style of peer review. I think we are only just scratching the surface on how peer review can evolve, but it is really encouraging that the publishing community are already thinking on how we can support and empower our young scientists.

View the latest posts on the Research in progress blog homepage

One Comment

Kiran

I would like to go with open reviewers’ comments. Names of the reviewers can be omitted. A fair play is expected everywhere.

Comments are closed.