Feedback to the Ministry of Health regarding the puberty blockers evidence brief
The email below was sent to the Ministry of Health earlier this week and is published her for a broader audience.
Dear Director General of Health,
I am the spokesperson for the group Fully Informed. We advocate for the right of patients and their parents to have accurate and understandable information on puberty blockers. We are pleased that the Ministry set in motion an evidence brief on puberty blockers late last year. The public, the patients, and their parents have been poorly served by the available practice guidelines and consent forms, as well the previous wording on the Ministry’s own website that puberty blockers were “safe and fully reversible” – a claim that senior clinical advisors now believe to be inappropriate.
There are recent systematic evidence reviews of puberty blockers (1,2), the findings of which could be usefully summarised and presented to the public with the Ministry’s imprimatur. We look forward to the Ministry’s support of the rights of consumers to receive accurate and understandable information on their treatment options and the public becoming better informed on this issue.
The evidence brief was originally due to be released around the middle of this year. However, on 22 August, the Ministry published a statement notifying the public that they are expanding the scope of the puberty blockers evidence brief “to include studies regarding mental health and wellbeing outcomes” and delaying the release to later in the year, after the election.
The statement and the timing for the evidence brief release at the end of this year raises some questions.
First, while the original scope of the evidence brief was the “safety, reversibility, and long-term clinical outcomes of puberty blockers for gender-dysphoric adolescents”, the expansion of the scope to include mental health and wellbeing outcomes is confusing as these should have already been included in the “long-term clinical outcomes” focus. However, to our knowledge there are no long-term studies reporting clinical mental health outcomes. There are some low quality, short-term studies of subjective self and parent reports of mental health outcomes with mixed results. The evidence reviews systematically evaluate these studies and conclude that the evidence for mental health improvements following puberty suppression is inconclusive.
The Ministry’s statement suggests that the Ministry may:
Be intending to report the mental health outcomes results of the systematic reviews (i.e. that are inconclusive).
Be intending to report a selection of results from the short-term studies directly.
Need an excuse to delay the release of the evidence brief beyond the election.
(1) would be entirely appropriate, (3) is understandable (though see my point about delays below). However, (2) is questionable and would undermine the Ministry’s reputation and to the integrity of New Zealand’s Science Advisory System for reasons which I outline below. I seek clarification from the Ministry that scientific integrity is paramount.
If the Ministry was planning on reporting on a selection of results from low quality, short-term satisfaction / psychosocial outcome studies of distressed teenagers, then:
How will the Ministry select the studies to report on? For example, will the Ministry report the results of Carmichael et al, (2021) (3) which found no change in psychological function, or the results of Costa et al, (2015) (4) which found no statistically significant differences between the group given psychosocial support together with puberty suppression and another group given psychosocial support only?
Will the evidence brief make the causality of the results clear? i.e. whether the results are attributable to the puberty suppression intervention, concurrent interventions such as psychosocial support, the placebo effect, or regression to the mean? (5)
Will the Ministry evaluate and report on the quality of studies and situate them within an evidence hierarchy?
As the Ministry knows, systematic evidence reviews mitigate the bias of the above issues. Systematic reviews work on the basis of an “hierarchy of evidence” – observational studies are unreliable in the evaluation of treatment effects specifically and replication of findings by controlled trials is necessary before firm conclusions can be drawn about a treatment’s effectiveness.
I make an OIA request for Ministry documents or policy that reference or describe the “hierarchy of evidence” or similar and how it applies to Ministry decision-making and reporting.
Unfortunately, the use of low quality studies to support pre-determined views is already common in this area of medicine, including the so-called “Standards of Care: Version Eight (SOC8) published by the World Professional Association of Transgender Health (WPATH). The SOC8 guidelines are highly unusual and are more appropriately seen as a legitimation exercise than a scientific document. Chapter 9 of SOC8 recommends that eunuch-identified males receive castration, minimum age requirements for surgery were hurriedly removed prior to publication, and WPATH is currently under a subpoena by the state of Florida to reveal communications related to the development of the guidelines – an action that would not be needed if WPATH had a commitment to transparency and the scientific process. It is, of course, unthinkable that the Ministry would endorse these controversial guidelines or their derivatives.
Second, the Ministry may not be aware that the Cass Review also plans to release followup research on approximately 9,000 gender dysphoric adolescents who have attended the Gender Identity Development Service towards the end of this year (including approximately 1,000 treated with puberty blockers). Unlike the subjective measures mentioned above, this research will have access to NHS patient records and be able to report on more objective long-term clinical outcomes such as fractures, antidepressant use and so on encompassed by the original, and current scope of the evidence brief.
It would be highly embarrassing and confusing if the Ministry were to report positive mental health outcomes from a potentially biassed subset of low quality studies, while at the same time the Cass Review was to release research reporting negative outcomes from the largest, highest quality observational study to date. Parents of adolescents treated with puberty blockers would no doubt have questions for the Ministry if this were to happen.
In the interim the Ministry may wish to review the medical claims about puberty blockers made by PATHA executive member Dr Rachel Johnson. The Ministry is in the process of reviewing the evidence and is in a good position to judge if these claims withstand scientific scrutiny and present a full and accurate picture of the evidence. It is odd that Dr Johnson has taken the lead in creating an evidence brief of her own when she presumably knew that the Ministry would soon be publishing an ‘official’ one. Rather than duplicating (or perhaps even contradicting) the Ministry regarding the existing evidence literature, it would be more useful if Dr Johnson could provide the Ministry with follow-up data on the adolescents treated with puberty blockers at her clinic. Despite the usage of puberty blockers for gender dysphoria for over a decade in New Zealand clinicians have not yet provided any follow-up data, and Dr Johnson’s more direct contribution here would be welcome.
If the Ministry judges Dr Johnson’s presentation of the evidence literature to be unbalanced, then the Ministry must take responsibility for delaying the evidence brief and allowing actors with conflicts of interest to fill the information vacuum. The Ministry may also wish to consider the adequacy and ethics of the current line that “[d]ecisions on the use of puberty blockers are best made by patients and their families in consultation with appropriate clinicians” when one of the “appropriate clinicians” is presumably Dr Johnson.
References
1. Ludvigsson JF, Adolfsson J, Höistad M, Rydelius PA, Kriström B, Landén M. A systematic review of hormone treatment for children with gender dysphoria and recommendations for research. Acta Paediatr [Internet]. [cited 2023 Apr 19];n/a(n/a). Available from: https://onlinelibrary.wiley.com/doi/abs/10.1111/apa.16791
2. NICE. Evidence review : Gonadotrophin releasing hormone analogues for children and adolescents with gender dysphoria. 2020;(October).
3. Carmichael P, Butler G, Masic U, Cole TJ, de Stavola BL, Davidson S, et al. Short-term outcomes of pubertal suppression in a selected cohort of 12 to 15 year old young people with persistent gender dysphoria in the UK. Santana GL, editor. PLoS ONE. 2021 Feb 2;16(2 February):e0243894.
4. Costa R, Dunsford M, Skagerberg E, Holt V, Carmichael P, Colizzi M. Psychological Support, Puberty Suppression, and Psychosocial Functioning in Adolescents with Gender Dysphoria. J Sex Med. 2015;12(11):2206–14.
5. Clayton A. Gender-Affirming Treatment of Gender Dysphoria in Youth: A Perfect Storm Environment for the Placebo Effect—The Implications for Research and Clinical Practice. Arch Sex Behav. 2023 Feb 1;52(2):483–94.
Great piece of work Simon. It's now a whole year since the Evidence Brief was suggested. One thing I've come across recently is the GRADE approach to assessing papers - which is an approach the evidence based reviews have adopted I guess. Presumably at some point OIAs will reveal the quality or not of the approach that has been taken.
Brilliant! Nice work Simon.