Generally speaking, drug trials involve a comparison of an experimental drug and an already-approved drug or a comparison of an experimental drug and a so-called "placebo," usually an inert substance. The classic placebo is a sugar pill, although some drug trials have drawn criticism for using non-inert substances as placebos, which may mask adverse events resulting from the experimental drug. In a forthcoming article in BMC Medicine, a team of French researchers casts down upon the legitimacy of several antidepressant clinical trials, noting a "major publication bias" in the funnel plots of the trials.
Seeking to investigate "whether the effects of placebo [sucrose] in one situation are different from the effects of placebo in another situation," the researchers, led by Florian Naudet, performed two meta-analyses of published and unpublished studies found in Medline, the Cochrane Library, Embase, clinicaltrials.gov, Current Controlled Trials, and other sources. The two meta-analyses were designed to compare response and remission rates among three placebos: first, placebos being compared in trials with fluoxetine; second, placebos being compared in trials with venlafaxine; third, placebos in trials being compared in trials with both fluoxetine and venlafaxine. (Fluoxetine is marketed by Eli Lilly as Prozac; venlafaxine is marketed by Pfizer as Effexor.)
The research team whittled their initial 11,051 trials to a list of 31 that met their criteria (including non-duplicate publications and availability in English, French, or Spanish). They then used funnel plots to look for the existence of publication bias in the studies used for their meta-analysis. A funnel plot is a scatter plot of treatment effect plotted against a measure of study size. Looking at the plotted data points, an observer sees a symmetric (roughly the same size and shape to the left and to the right of the midline) inverted funnel (a triangle with the base resting on the X axis) when the data is not suggestive of publication bias. An asymmetric funnel (one in which the areas to the left and to the right of the midline are not of equal size and shape) suggests a relationship between treatment effect and study size, which may indicate publication bias.
After drawing six funnel plots -- three for response and three for remission -- for each meta-analysis previously noted. They observed some asymmetry in the funnel plots for the trials comparing fluoxetine to venlafaxine, in favor of venlafaxine.
The researchers found that venlafaxine generally outperformed fluoxetine, and that both experimental drugs generally outperformed placebos. Additionally, they found that placebos among the three groups of trials had similar results to each other. However, the researchers note that the possible publication bias in favor of venlafaxine may have influenced these results. They additionally note that all thirty-one trials included in their meta-analysis were sponsored by pharmaceutical firms, and recall the previous finding that drug trials sponsored by drug manufacturers are significantly more likely to report positive results for the experimental drug. "The hopes and expectations of the physician are just as crucial as those of the patient in the healing process," the researchers caution. "When one uses a treatment that relies on expectation, one must also be careful as to its possible harmful consequences."