Reducing the risk of bias in biomedical publications
The technology is a text-mining tool, which identifies the reporting of measures to reduce the risk of bias in biomedical research publications. This would allow stakeholders, such as publishing groups and funding agencies, to assess the quality of their output, increasing confidence in the results and minimising the waste of resources on unreliable research.
Allows research stakeholders to track whether their publications meet required standards.
Can guide future policies for further improvements in quality.
Proof of concept
Biomedical science is suffering a replication crisis, suggesting that a significant proportion of research resources are wasted and that many results are unreliable. However, this can be countered effectively with simple actions, such as properly accounting for the known risks of bias in in vivo research. Although few studies report the measures taken to reduce these risks, doing so is a stated requirement for many research stakeholders (such as journals, funding bodies and universities) who wish to increase the standards of their research.
A text-mining tool has been developed which can identify the reporting of measures such as randomisation, sample size calculations, and blinded assessment of outcomes, in research publications. Aggregating this information for a large number of publications allows research output to be monitored, and comparisons – such as changes in practice over time, or between different journals – to be made. The technology is a new method to drive improvement in the standards of biomedical research by providing a rigorous metric for the quality of research publications.
The text mining method has been validated by several complementary studies (Macleod et al., 2015), some with samples of more than 2500 publications. Comparisons of the reported risks of bias are made between different time periods, journals of different impact factor, and universities. More recent work has shown the method to have sensitivities of 100%, and specificities >80%, for each of three different reported measures.
- High sensitivity and specificity for multiple measures
- Fits within the existing framework of research standards
- Easy to monitor and guide improvements in practice
- Reduces the waste of resources on low quality work
M. R. Macleod et al. (2015) PLoS Biol. 13(10) e1002273