Should I call my study a ‘pilot study’ or a ‘feasibility study’? Can I make changes to the intervention at this point, and if I do, do I have to assess feasibility all over again? Should I use pilot data to generate signals of the effectiveness of the intervention?
These may be questions familiar to public health researchers who have struggled with determining whether, and if so how, to proceed to a full-scale evaluation of a complex intervention. The likelihood of hard work, spanning years or even decades, generating useful evidence of the effectiveness of interventions may hinge on first addressing questions about their feasibility and future evaluation.
In pharmaceutical research over 80% of drug trials fail to progress to ‘Phase III’ effectiveness trials.
In pharmaceutical research over 80% of drug trials fail to progress to ‘Phase III’ effectiveness trials. It is unlikely complex public health interventions would be exempt from the same probabilities of success. Complex interventions are, after all, called ‘complex’ for a reason.
Their complexity often lies in the uncontrolled settings in which they occur. Even interventions which seem simple on the surface are often highly complex in terms of how they interact with their contexts. Hence, the historical tendency for interventions to progress too rapidly to evaluation means that many of these highly expensive studies had little chance of achieving their aims.
Increased prevalence of feasibility and pilot studies
The unique complexity that often accompanies public health interventions therefore requires appropriate guidance that reflects its nature. Earlier MRC guidance has emphasized the need for feasibility and pilot studies in order to avoid wasteful full scale evaluations. This has led to rapid increases in their funding in recent years.
The movement toward greater consideration of feasibility and pilot work is encouraging, but there is a need for clearer guidance on how best to design, conduct and report such studies. At the moment, guidance for the diversity of public health interventions is patchy and often conflicting, although more recently some have made good progress in making sense of this disorderly, but paramount, field of research.
As part of a larger study funded by the Medical Research Council and National Institute of Health Research Methodology Research Programme, which aims to develop GUidance for Exploratory STudies of complex public health interventions (GUEST), we conducted a systematic review to provide an overview of existing guidance on feasibility and pilot studies and identify areas of agreement and disagreement in recommendations.
While we used the term ‘exploratory study’, the findings of the review, and ongoing stakeholder consultations since, have indicated that the term ‘feasibility study’ has most resonance with producers and users of this kind of research.
The results of this review show there is still a lot of work to be done and progress to be made in this field with much existing disagreement. These findings have been used within the wider GUEST study to inform a Delphi survey of researchers, funders and publishers of public health research, which in turn will inform a consensus meeting, and ultimately the production of new guidance.
Comments