Why a RCT in aGvHD for “The Greater Good” is a Myth
The Limitations of The “Gold Standard”
To borrow Prof Vinay Prasad’s phrase again, if something isn’t working we should stop doing it.
Re. CHF, IBD and pain, I referred to peer-reviewed papers in posts such as this one51850873where researchers/physicians don’t mince their words expressing frustration that the “gold standard” metric used to determine the efficacy and safety of medical interventions is failing their real-world patients.
John Ioannidis MD has long been in the vanguard for medical reform. He co-authored a recent paper finding more than 9 in 10 healthcare interventions studied within recent Cochrane Reviews are not supported by high-quality evidence and harms under reported “despite Cochrane authors being encouraged to report harms as primary outcomes of reviews”.
(There’s been an interesting development since I last referred to this paper. Link below to a report published in Nature on the funding crisis and closures at Cochrane.)
Jiggering and data falsification (a common trick is excluding participants, which is a problem with large numbers) aside, in the case of acute GvHD, a successful “proper” trial run properly would yield an average result over the cohort that would have to be applied to the mythical average patient in a condition which is the most heterogeneous it’s possible to get. Not only that, but for a therapy that responds to the individual state.
@JB1975 I have great respect for your scientific knowledge greatly but surely a scientific process should be as much about reasoning as about facts and I’ve set out my reasoning why a RCT in aGvHD could never have been for the greater good.
“FDA was clear about what it wanted and Si should have done it for the greater good”
The above criticism has been voiced on this forum.
I recall from around a year ago a webcast where SI said they were willing to do a RCT in high risk adults against something approved and therefore Ruxolitinib if that’s what the FDA wanted.
For those who say MSB should have done a “proper” RCT before, where would they have recruited from? Would it have been from the centers where Ryoncil has been used in an EAP?
Would adults with grades C/D GI aGvHD, who are likely aware that Ryconcil has been available in an EAP and therefore works, have agreed to a 50% chance of being randomized to a tablet in a trial which likely wouldn’t have a crossover?
What about investigators who have to give prospective participants proper informed consent in writing? If I recall correctly, KOL Dr. Susan Prockop said in an interview that after her center’s experience with treating three patients with Rux, they wouldn’t be participating in a clinical trial.
Was the option of the BMT CTN running the RCT for MSB always available to them?
Patient-Specific Confounders
The rationale for a RCT is that it reduces confounding factors. Ideally patients should be reasonably balanced in each arm.
In this post I examined the extreme difficulty of balancing when you have so many patient-specific confounding factors that can affect response68935952The disparity in grading organs for severity among centers is well acknowledged in peer-reviewed literature.
Further to this, I note even the conditioning regimen can affect response; Magenau et al. (2018) found patients achieving CR had received a greater proportion of radiation-containing conditioning.
And if you plan to trial a therapy for steroid-refractory aGvHD, of course one of the biggest confounders is that different centers define steroid refractoriness differently.
Treatment-Specific Confounders
Ruxolitinib provides no mortality benefit over SoC beyond a few months.
Prior to Rux approval, Ryoncil would have to have been trialed against SoC, which would introduce more degrees of confounding from the treatments themselves.
@LeftYahoo has provided a list but in my view it’s best to go with the one provided by Malard et al. (2023) because it’s most recent and comes from a group who have deep familiarity with the challenges of the condition itself. They refer to Holtan al. (2022) which was published by the same prestigiousNaturegroup.
Malard et al. provide a list of therapies (which they term a treatment algorithm) for aGvHD refractory to steroids and Rux. It’s in alphabetical order.
The first thing to notice is how many of these therapies have been used for decades, such as ECP, Sirolumus, ATG.
In the Holtan study in the category of patients who received a third-line treatment, the majority (19%) received Sirolimus, then polyclonal antibodies (18%), which includes ATG which has got some very poor results, followed by MMF (17%).
More recent additions on the Malard algorithm are FMT (fecal microbiota transplant), Vedolizumab, α1-antitrypsin and MSCs.
In a RCT against SoC, bias would be unavoidable:
For ethical reasons, organ involvement and severity would inform treatment (some studies found therapies had best response in skin), which could be at the discretion of the investigator, and they’d likely use what they were familiar with.
For most of the above therapies, dosing as a second-line therapy hasn’t been established in RCTs or duration of dosing.
What Prof Kurtzberg says about children with GI symptoms having difficulty keeping down tablets holds true with adults too. In some cases there’s also hemorrhaging. Common sense calls for an IV/injectable option.
The Malard Treatment Algorithm for Patients Refractory to Steroids and Rux
Alemtuzumab.Khandelwhal et al. (2016) provide good refs to studies showing highly variable survival rates for adults (ranging from 0 - 50%). In their study in 15 children with predominantly grade 3 severity they report a survival rate of 80% at 6 months. GI response was less favorable than skin. Chronic GvHD incidence was significant among survivors.
In a phase 2 trial reported by Martinez et al. (2009) in ten adult patients with grades 3 and 4 SR acute GI GvHD, despite some responses, all patients had died by day 88.
AntitrypsinMalard et al. say this therapy has shown “some success”. There are a few small studies. Magenau et al (2018) report on a study in 40 patients: 7/40 had grade 4 gut and/or liver; 14/40 had grade 3 gut and/or liver involvement. Overall survival at 6 months was 45%; however, the authors say the cohort was older.
Anti IL-2 receptor antibodies
There have been a few on the market for a couple of decades such asInolimomab.Bay et al. (2005) report on 85 patients. Survival at 20 months was 26%.
DaclizumabPerales et al. (2007) report on a study in 57 patients. (This study is unusual in that there was a long-term follow-up.) Of 57 patients (21 of whom under 18), 14 remain alive at 98 months median follow up, 8 of whom are free from recurrent GvHD. Better response for skin than gut and liver.
Herbelin et al. (1994) report on a promising study inBT563in 15 children, 10 of whom were long-term survivors. Unfortunately, only the abstract is available, so I don’t know about organ involvement and severity.
Anti-TNFThat would include bothInfliximab and Etanercept. These are imo the worst choices. I put the refs to studies with poor results in this post 52579284; however, I note from Holtan et al.'s pie chart, Etanercept accounts for a significant percentage (13%). If I take a more charitable view of Reach2 investigators’ choices, they may reflect the dismal options for gut GvHD.
ATGThis is most commonly used in pediatric centers. Arai et al (2002) report on dismal results:52579284
“Of the 69 patients treated with ATG for steroid-refractory GVHD, only 3 (5%) are currently alive”. Authors say other studies got similarly disappointing results.
Macmillan et al. report on a study in ATG in 79 patients. Probability of survival at 1 year was 32%. Best response was in skin and there was a very high rate of chronic GvHD.
A phase 2/3 RCT comparing ATG to ABX-CBL found similar responses and survival at 18 months was 45% and 35% respectively.
MMFI referred to studies in this post52579284which includesa phase 3 study finding no benefit to adding MMF to steroids.
VedolizumabThis drug is most commonly used in IBD for patients refractory to both Infliximab and Humira. Mehta et al. (2020) report on “suboptimal” results in 20 patients with SR lower GI aGvHD: 15 patients died.
Li et al s meta analysis (2022) found similarly dismal survival rates:
“Fifty-seven out of 87 (pooled overall survival, OS = 34.5%) and 46 out of 65 (pooled OS = 29.2%) patients expired at 6 and 12 months after the use of vedolizumab, respectively”.
Most of these studies above report a very high incidence of infections. Many report responses, some quite impressive, but they often don’t translate to survival. Very few report longer term survival, even at 1 year.
I ask anyone who believes a RCT against SoC could have gone ahead if they could give an idea of what proper written informed consent might look like to high risk patients in a proposed RCT against SoC that would make it likely they’d agree to a 50% chance of landing in the SoC arm.
Could a RCT against SoC Inform Clinical Practice?
Taking those BAT options above, I ask if it’s possible to design a trial against SoC which could help a physician decide the best treatment for the following:
A patient has grade B/C/D skin and minor GI involvement. Is the investigational therapy superior to say Sirolimus alone, ECP alone, or any of the other therapies alone or in combination?
A patient has severe GI. Is the investigational therapy superior to each of the SoC therapies, including IV options, either alone or in combination with another, especially if there’s liver involvement (which is less commonly affected, as Malard et al. say).
Is the investigation therapy superior to SoC in combination with any of the above therapies (ie. a synergy) and if so, what should be the timing and dosing of the synergistic BAT therapy?
(Note Malard et al. say one of the most promising therapies is FMT. Not being a consistent product, could this most promising therapy for lower GI even be a BAT in a trial against SoC?)
A “successful” RCT against SoC could only yield an average result you’d have to apply to individuals suffering the most heterogeneous condition. It could never properly inform clinical practice.
@whytee has said Osiris managed to run two RCTs. This assertion is imo disingenuous in three ways:
Prochymal was combined with steroids and Ryoncil is for steroid-refractory cases.
A significant number of milder skin cases were included in NCT00366145, which caused the trial not to meet its PE (steroids do a good job with skin) but liver and gut response did show significant superiority compared with placebo.
The potency has improved since Osiris days. Ryoncil has been available in an EAP and patients and investigators know it works.
A RCT against SoC can be Misleading
A RCT against SoC can be misleading in that it can make a therapy look less effective than it is.
In this post69705273I discussed how Remdesivir in CARDS could have been a major potential confounding factor. Also, SoC changed to using repurposed drugs in the latter half, which could have made our cells less effective if they dampened inflammation, our cells needing peak inflammation to work best.
I take the point about patients in the latter half being older but I also wondered if prior to Remestemcel-L, they’d received drugs that could deplete T cells, as Inolimomab is said to do.
The EAP was run by Mount Sinai who are doing groundbreaking work in predictive and response biomarkers. I believe Remestemcel-L was appropriately delivered here and that this small study was more scientific and a better reflection of the cells’ true power than the RCT.
It took a whole year for the report on the EAP to be published. I noted how muted it was in contrast to the shock (in a good way) of Dr. Keren Osman who was interviewed on the successful study.
The paper contained the caveat that the effect of prior drugs couldn’t be ruled out. I’d have thought the same rule applies in ARDS as in acute GvHD: If the treatment isn’t working and you add something, the effect (in this case liberation from vents) is attributed to whatever you added.
I’m no scientist but I don’t see how a RCT in acute GvHD can be in any way scientific. It looks like more of an art to me.
Depending on the condition, a single-arm trial using predictive and response biomarkers can be more scientific and provide further benefits:
Less reliance on RCTS which often require large numbers, are costly and can be slow to enrol (fear of placebo)
Makes selection of patients easier and more ethical (only given to those likely to benefit) and reduces costs
Makes timing of the therapy easier and therefore more effective
Reduces the potential for fakery and bias
Makes real-world evidence easier to monitor, reduces the potential for different interpretations and controversy, including the gaslighting of patients and physicians
In my next post I’ll present my thoughts on predictive and response biomarkers, again with heavy reference to the Malard paper.
ALL IMO
https://www.nature.com/articles/s41572-023-00438-1
https://www.nature.com/articles/d41586-023-02741-z
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5865235/
https://www.astctjournal.org/article/S1083-8791(16)30356-1/fulltext
https://pubmed.ncbi.nlm.nih.gov/19361757/
https://pubmed.ncbi.nlm.nih.gov/17618322/
https://pubmed.ncbi.nlm.nih.gov/11939605/
https://pubmed.ncbi.nlm.nih.gov/11858189/
https://pubmed.ncbi.nlm.nih.gov/33781539/
https://pubmed.ncbi.nlm.nih.gov/36439135/
(20min delay)