Evidence Synthesis, Umbrella Reviews, and the Escalating Risk of "Evidence-Laundering"​

In research, evidence synthesis plays a critical role in gathering and analyzing information from various sources to reach well-informed conclusions. However, there is a growing concern about the potential for evidence synthesis to be misused as a form of "evidence-laundering," akin to money laundering. In this scenario, low-quality evidence is aggregated through systematic reviews and presented as high-quality evidence, obscuring its true nature and leading to misleading results.

Scientific evidence laundering refers to aggregating low-quality or biased primary research through evidence synthesis techniques, such as systematic reviews or umbrella reviews, to present a precise estimation. This synthesized evidence is then portrayed as high-quality evidence, often by citing the review without acknowledging the limitations or flaws of the primary research.

The concept of "garbage in, garbage out" (GIGO) is particularly relevant here. GIGO is an aphorism in computer science and information management that suggests if the input (in this case, low-quality evidence) is flawed or unreliable, the output (the synthesized evidence) will also be flawed or unreliable.

The risk of evidence-laundering is further magnified when umbrella reviews are conducted. Umbrella reviews synthesize the findings of multiple systematic reviews on a particular topic, adding another layer of complexity between the original research and the conclusions drawn. While umbrella reviews can provide a valuable overview of the available evidence, they also increase the potential for low-quality evidence to be masked and presented as high-quality. This is especially true if the primary studies and systematic reviews included in the umbrella review are not thoroughly appraised for their quality and rigour.

To combat evidence laundering and safeguard the integrity of scientific research and evidence-based decision-making, researchers and clinicians must take proactive two measures:

1. For clinicians - Be critical consumers: Approach research findings with a healthy dose of scepticism. Critically appraise the quality of primary studies, systematic reviews, and umbrella reviews before incorporating their conclusions into your practice or policy decisions. The Journal of the American Dental Association (JADA) features an informative series of articles that provide guidance on interpreting and understanding the dental scientific literature, authored by Alonso Carrasco-Labra, Romina Brignardello-Petersen and Michael Glick -> read here.
2. For researchers - Promote research transparency: When conducting or citing research, be transparent about the limitations and potential biases of the primary studies and synthesized evidence, and provide the original raw data to facilitate reproducibility. Encourage open discussions about the quality of the evidence and its implications on the conclusions drawn.