An international collaboration of researchers has introduced a new tool designed to identify unreliable randomized controlled trials (RCTs), including those that may be fraudulent. The tool, named INSPECT-SR, was recently published on the pre-print server medRxiv. It stems from a collective effort involving over 150 experts in research integrity and health, led by Dr. Jack Wilkinson from The University of Manchester, and funded by the National Institute for Health and Care Research (NIHR).
This initiative is a response to mounting concerns over the integrity of research studies, particularly in relation to systematic reviews that rely on RCTs for evidence. In 2023 alone, more than 10,000 research papers were retracted globally, as reported by Nature, many of which were based on flawed RCTs.
Addressing Research Integrity Challenges
The INSPECT-SR tool aims to detect problematic RCTs that may contain manipulated data or critical errors, as well as those that are entirely fabricated, often produced by organizations known as “paper mills.” The tool employs a systematic approach, guiding users through 21 checks across four distinct domains.
Dr. Wilkinson emphasized the importance of this tool, stating that problematic RCTs could undermine medical research, hinder drug development, and jeopardize promising academic studies. He pointed to the controversy surrounding the drug Ivermectin, which was initially touted as a potential treatment for severe COVID-19. Investigations revealed that some trials supporting its efficacy may have been fabricated, with health authorities from the US, UK, and EU expressing concerns over the lack of reliable evidence.
Another notable example involved the National Institute for Health and Care Excellence (NICE), which reversed its recommendation for a device called a fetal pillow after three clinical studies supporting it were retracted. An article published in BJOG: An International Journal of Obstetrics & Gynaecology suggested that a trustworthiness assessment could have prevented the flawed evidence from influencing NICE guidelines.
Future Implications and Developments
The impact of unreliable RCTs extends beyond individual studies, affecting clinical practice guidelines and healthcare decisions. Dr. Wilkinson noted, “When a systematic review is carried out, it includes all randomized controlled trials on a given topic. But historically, there has been no way to identify fraudulent or otherwise problematic RCTs, meaning that these studies are inadvertently included in systematic reviews.”
He expressed concern that the rise of artificial intelligence (AI) could exacerbate the issue of fraudulent research. “Most fraudulent RCTs are produced by individual researchers rather than commercial paper mills, but with the advent of AI, I fear this is likely to become more of a problem in the future,” he said.
The INSPECT-SR tool is not just a measure against fraud; it also identifies critical errors that could affect the validity of clinical trials. Dr. Wilkinson anticipates that the tool will become a standard for assessing RCT trustworthiness, particularly as it has been developed in collaboration with Cochrane, a leading publisher of health systematic reviews.
He added that ongoing work aims to develop automated systems, potentially utilizing AI, to assist in identifying issues across various types of research studies, not solely limited to clinical trials. The introduction of INSPECT-SR may signal a significant advancement towards enhancing the integrity of medical research and ensuring that healthcare decisions are based on reliable evidence.
