Seen the “98% of studies were ignored!” one doing the rounds on social media. The editorial in the BMJ put it in much better terms:
“One emerging criticism of the Cass review is that it set the methodological bar too high for research to be included in its analysis and discarded too many studies on the basis of quality. In fact, the reality is different: studies in gender medicine fall woefully short in terms of methodological rigour; the methodological bar for gender medicine studies was set too low, generating research findings that are therefore hard to interpret.”
101 of 103 studies were not dismissed. All systematic reviews classify their source studies based on the quality of the work. Of the 103, two were classed as high quality, 58 as moderate quality and the remaining 43 as low quality. For synthesis, only high and moderate quality studies were drawn on. That’s more than half, not 2%.
So yes, Erin is lying.
You can’t say she’s lying until we do a systemic review of why the Cass study dismissed everything but 2 studies for the numbers it used to reach its conclusion. You can’t say she’s lying without that review no more than I can support Erin by reading each study that was dismissed. What I can tell you is that dismissing that many studies is not normal scientific analysis. It reeks of bias.
This is the lie. They didn’t dismiss all but two studies, they actually included 60. More than half of the 103 studies identified for the review.
So yes, Erin, and now yourself, are peddling a lie.
It’s key part of synthesising multiple sources into a meta-analysis. Including poor quality studies dilutes the quality of the overall analysis.
By design, it’s biased towards higher quality research.
Synthesis is a paragraph summary inclusion ONLY, it means they didn’t use data from the study, it is dismissal. I’m done arguing that with you.
They have absolutely used the data from those 60 studies. You can read where they say explicitly that in the report if you cared to.
You are utterly mistaken and firm on your conviction, these are not the qualities of skepticism.
“Don’t seek refuge in the false security of concensus”
That’s not what synthesis means. I’ve written synthesis reports before and the data you include from those reports once you have dismissed them as inaccurate, it is an entirely selective process of whatever you want to include from them. We even have a phrase for it in law, Summarily dismissed.
And of the 103 reviewed they included data from 60. It is a lie to say they “dismissed all but two.”
Legalese is irrelevant. A systematic review of scientific literature is a different beast to “writing a few synthesis reports”.
So you don’t know what you are talking about. Gotcha.
Synthesis reports in a scientific study when presenting data, are the parts of the report where you explain why you are dismissing data, so in this case ~98% of the data or studies. So what you just said is ~98% of the data was included in the synthesis report. that’s not inclusion of the data. That’s selective inclusion to support a conclusion. A normal scientific study can’t dismiss 98% of available data. That reveals bias.
You can read the reviews for yourself
https://t.co/82Rjs2L1pA
https://adc.bmj.com/content/early/2024/04/09/archdischild-2023-326669
Let me know where you find the bit where they dismiss 101 out of 103 papers.
Hint, they didn’t. It is lying to say they dismissed 98% of the studies they looked at.