Seen the “98% of studies were ignored!” one doing the rounds on social media. The editorial in the BMJ put it in much better terms:
“One emerging criticism of the Cass review is that it set the methodological bar too high for research to be included in its analysis and discarded too many studies on the basis of quality. In fact, the reality is different: studies in gender medicine fall woefully short in terms of methodological rigour; the methodological bar for gender medicine studies was set too low, generating research findings that are therefore hard to interpret.”
Ms Reed is not an objective source. Nor does it appear she has much experience with systematic reviews. Indeed she repeats many of the “myths” such as the one about 98% of studies being thrown out.
"A closer inspection of the reviews released alongside the Cass report reveals that 101 out of 103 studies on gender-affirming care were dismissed for not being of “sufficiently high quality,” "
That is a lie.
This is another lie from Ms Reed. I am beginning to think this “debunking” article you’ve shared is actually the original source of most of the myths being peddled on social media.
It’s not a lie, they were mostly dismissed even according to your own article, they were dismissed and synthesized into one conclusion. That is still dismissal.
101 of 103 studies were not dismissed. All systematic reviews classify their source studies based on the quality of the work. Of the 103, two were classed as high quality, 58 as moderate quality and the remaining 43 as low quality. For synthesis, only high and moderate quality studies were drawn on. That’s more than half, not 2%.
So yes, Erin is lying.
You can’t say she’s lying until we do a systemic review of why the Cass study dismissed everything but 2 studies for the numbers it used to reach its conclusion. You can’t say she’s lying without that review no more than I can support Erin by reading each study that was dismissed. What I can tell you is that dismissing that many studies is not normal scientific analysis. It reeks of bias.
This is the lie. They didn’t dismiss all but two studies, they actually included 60. More than half of the 103 studies identified for the review.
So yes, Erin, and now yourself, are peddling a lie.
It’s key part of synthesising multiple sources into a meta-analysis. Including poor quality studies dilutes the quality of the overall analysis.
By design, it’s biased towards higher quality research.
Synthesis is a paragraph summary inclusion ONLY, it means they didn’t use data from the study, it is dismissal. I’m done arguing that with you.
They have absolutely used the data from those 60 studies. You can read where they say explicitly that in the report if you cared to.
You are utterly mistaken and firm on your conviction, these are not the qualities of skepticism.
“Don’t seek refuge in the false security of concensus”
That’s not what synthesis means. I’ve written synthesis reports before and the data you include from those reports once you have dismissed them as inaccurate, it is an entirely selective process of whatever you want to include from them. We even have a phrase for it in law, Summarily dismissed.
And of the 103 reviewed they included data from 60. It is a lie to say they “dismissed all but two.”
Legalese is irrelevant. A systematic review of scientific literature is a different beast to “writing a few synthesis reports”.