About outcome reporting bias in trials (ORBIT)

In the medical research field, it is well known that clinical trials achieving statistical significance are more likely to be submitted and accepted for publication. However outcome reporting bias, which has been defined as the selection for publication of a subset of the original recorded outcome variables based on the results, is a potentially greater threat to evidence-based medicine.

This bias may lead to incorrect decisions concerning effective healthcare but until recently little was known about the prevalence and impact of outcome reporting bias in systematic reviews.

Key messages to date

Our study found that few systematic review authors mentioned the potential problem of outcome reporting bias despite our assessment indicating that it was suspected in over a third of reviews. Our analysis suggested that a fifth of decisions may have been incorrect as a result of this problem, and a quarter of reviews may have seriously overestimated the benefits of treatment. There is a clear need to raise awareness of both the existence and potential impact of bias when trialists choose which outcomes to report on the basis of the results for those outcomes. Proposed solutions, alongside educational initiatives, include the standardisation of outcomes in specific clinical areas, and improvement of trial funders’ guidelines for researchers.  See “Aims” for the full list of aims and objectives to the ORBIT study.

Acknowledgements

The study group are grateful to the many Cochrane review authors who collaborated in some way to make this research possible.  We would also like to thank the Cochrane Steering Group Co-ordinators, Cochrane statisticians for their help and advice when needed.