Seen the “98% of studies were ignored!” one doing the rounds on social media. The editorial in the BMJ put it in much better terms:
“One emerging criticism of the Cass review is that it set the methodological bar too high for research to be included in its analysis and discarded too many studies on the basis of quality. In fact, the reality is different: studies in gender medicine fall woefully short in terms of methodological rigour; the methodological bar for gender medicine studies was set too low, generating research findings that are therefore hard to interpret.”
That’s not what synthesis means. I’ve written synthesis reports before and the data you include from those reports once you have dismissed them as inaccurate, it is an entirely selective process of whatever you want to include from them. We even have a phrase for it in law, Summarily dismissed.
And of the 103 reviewed they included data from 60. It is a lie to say they “dismissed all but two.”
Legalese is irrelevant. A systematic review of scientific literature is a different beast to “writing a few synthesis reports”.
So you don’t know what you are talking about. Gotcha.
Synthesis reports in a scientific study when presenting data, are the parts of the report where you explain why you are dismissing data, so in this case ~98% of the data or studies. So what you just said is ~98% of the data was included in the synthesis report. that’s not inclusion of the data. That’s selective inclusion to support a conclusion. A normal scientific study can’t dismiss 98% of available data. That reveals bias.
You can read the reviews for yourself
https://t.co/82Rjs2L1pA
https://adc.bmj.com/content/early/2024/04/09/archdischild-2023-326669
Let me know where you find the bit where they dismiss 101 out of 103 papers.
Hint, they didn’t. It is lying to say they dismissed 98% of the studies they looked at.
Read it. Their only inclusion in the report is to half explain why the were discluded, exactly what I said. Most of the dismissals are unscientific, not supported by a statistical analysis of why it was discluded. Data doesn’t become unreliable just because it is incomplete.
That report is absolutely rife with white washing and selection bias, I’d expect a scientific review of trans literature and studies to be a book at this point not 32 pages dismissing 98% of the data. It’s frankly insulting to anyone that’s read or written any number of scientific studies.
They did not dismiss 98% of the data.
Putting 98% of the relevant available data in a supplementary table like 4 is not including the data.
Supplementary Table 4 (from the first review) is a list of each of the 53 studies included in the review and how they were scored based on the Newcastle-Ottawa scale.
The “data” is in supplementary tables 3, 5, 6 and 7. Only studies that were scored as low quality were excluded from the synthesis.
“They dismissed 98% of data” is a lie.
No it’s not. None of the dismissals are statistically/ scientifically supported, and the data they present is blurbed and incompletely presented in a way that isn’t inclusive of what those studies actually say.