Innovative Pathways: How impactful are synthetic control arms in regulatory and HTA evaluations?
Introduction
This second article on the current and fascinating developments in artificial intelligence (AI) will dive deeper into the evaluation of synthetic control arms (SCAs). If you are new to synthetic data or missed the first article, make sure to catch up here
As a reminder, synthetic data are created by generative AI models trained to learn the patterns and statistical properties of a sample real-world data set. Once a generative model is trained, it can generate artificial information with the same statistical properties as the sample real-world data, making it indistinguishable (Kokosi & Harron, 2022). Synthetic data are used to expand real-world data sets, train deep learning models and improve the accuracy of model outputs. As such, synthetic data can be used in control arms to help strengthen a product's evidence base, which can be particularly beneficial for rare disease treatments where patient data are scarce (Kokosi & Harron, 2022). However, currently, synthetic control arms (SCAs) are rarely used to support regulatory or HTA submissions (Amy McKee in PharmExec, 2021). This article presents Blincyto® as a case study into the use of SCAs, reviewing regulatory and HTA outcomes to better understand the impact of SCAs on these assessments and identify challenges associated with the use of SCAs.
Blincyto®
Blincyto® (blinatumomab) is a recombinant antibody indicated for the treatment of adults with Philadelphia chromosome-negative relapsed or refractory B cell precursor acute lymphoblastic leukaemia (ALL), a rare type of cancer. It has received EU orphan designation, FDA breakthrough therapy and orphan drug status, FDA priority review and orphan status by the Therapeutic Goods Administration (TGA).
Regulatory Outcomes for Blincyto® (EMA, 2015; TGA, 2018)
In 2015, Amgen applied for regulatory approval for Blincyto® in Europe and Australia, submitting evidence from the pivotal phase 2, open-label, single-arm, multicentre study. Supportive studies included: (a) an exploratory phase 3 efficacy and safety study; (b) a historical comparator study on haematological remission rates and survival in adult patients receiving standard of care chemotherapy; (c) propensity score analyses; and (d) a model-based meta-analysis (MBMA) to evaluate the proportion of complete remission, duration of complete remission and overall survival in adult patients receiving salvage therapies. The MBMA provided the basis for the SCA, which was described as a “simulation”. The simulation was used to project the proportion of complete remission, duration of complete remission and overall survival in patients with relapsed or refractory ALL, which is a similar patient population to the one in the pivotal trial.
Both the EMA and the TGA noted that the single-arm design of the pivotal trial provided moderate quality evidence and stated that they would have preferred data from a phase 3 trial. The EMA commented that the historical data studies – referring both to the historical comparator study and the MBMA – did not provide sufficient support to give confidence in Blincyto®’s comparative superiority, noting that historical data analyses are generally associated with limitations such as insufficient randomisation and a lack of accounting for changes in treatment standards over time. Nevertheless, the EMA considered the historical analyses to be of good quality overall. The TGA felt more positive about the MBMA methodology, but, in alignment with the EMA, noted that some analyses included subjects treated with an outdated standard of care.
The EMA concluded that combined with the historical analyses, the evidence using the SCA demonstrated a clinical benefit, but uncertainty remained around the magnitude of the treatment effect. Therefore, Blincyto® was granted conditional approval contingent on providing additional safety data, efficacy data from a phase 3 comparative post-authorisation efficacy study, the implementation of risk minimisation measures, and the creation of educational materials for various clinical stakeholders. The TGA concluded that “while it would have been ideal to have considered the final results of a phase 3 study, there is sufficient strength to the efficacy data presented” (p.31, TGA, 2018) and granted Blincyto® conditional approval. Like the EMA, the TGA requested the report of the phase 3 study once available and the provision of educational materials but did not require additional risk mitigation measures. However, Amgen was required to conduct a post-registration marketing study to assess the effectiveness of the educational materials.
HTA outcomes for Blincyto®
Australia, Germany, France and Norway were among the first HTA submissions initiated in 2015 and 2016; these excluded results from the post-authorisation phase 3 study.
Interestingly, the submission in Australia did not include the MBMA data despite the relatively positive feedback from the TGA. The Pharmaceutical Benefits Advisory Committee (PBAC) expressed much uncertainty around the comparative effectiveness and the ICER and therefore rejected Blincyto® (PBAC, 2015).
In Germany, the G-BA did not comment on the MBMA but noted that the historical comparison study lacked validity due to multiple methodological flaws, thereby making it impossible to assess the clinical benefit of Blincyto®. Therefore, Blincyto® was considered to have an unquantifiable benefit (G-BA, 2016). Notably, when the G-BA re-evaluated Blincyto® in 2017 with the phase 3 data available, a considerable added benefit was accredited to Blincyto® (G-BA, 2017).
The comparative effectiveness was also considered to be uncertain in France, where Blincyto® received an ASMR IV, indicating a minor improvement compared to currently available treatments. Nevertheless, the medical benefit (SMR) was considered important, resulting in a reimbursement rate of 65% (HAS, 2017).
In Norway, the evidence submitted from the historical comparator study and the MBMA was accepted, however, in alignment with regulatory feedback, the regulatory authority noted that the selected data includes subjects receiving types of care that are no longer practised, making it less representative of the current standard of care. Blincyto® did not receive reimbursement (NOMA, 2016).
Discussion
This Blincyto® case study shows that the inclusion of synthetic data, and more generally the use of historical data, can yield mixed results but overall does not eliminate uncertainties associated with a single-arm trial. Currently, there is no formal guidance from regulatory and HTA bodies for SCAs, and the infrequent use of SCAs can at least partially be attributed to uncertainty around the expectations of evaluating agencies. As a proxy, manufacturers may consult the FDA and EMA guidance on the design and use of external control arms (ECAs), as some elements, such as the source selection, can be transferable. However, the EMA and FDA guidance for ECAs has been criticised for providing top-level recommendations that leave much uncertainty around how the quality of evidence is assessed and what criteria must be met for the data to be considered suitable for submissions (Jaksa et al., 2022). Consequently, decisions on regulatory approval, pricing and reimbursement involving ECAs can vary significantly. This is reflected in Blincyto®’s history: while several agencies shared critiques, the outcomes differed. An analysis of regulatory and HTA outcomes for seven oncology drugs that submitted evidence from ECAs indicated that regulatory agencies and HTA bodies interpreted ECA data differently and came to opposing conclusions in several cases (Jaksa et al., 2022). The unpredictability surrounding ECAs is likely to be exacerbated for SCAs, as critical elements of SCA design, including the validity of the AI model used, are not covered by the ECA guidance.
Conclusion
This article reviewed the regulatory and HTA outcomes of Blincyto®, a product that used synthetic data to strengthen its evidence base and reduce uncertainty related to the single-arm pivotal trial. Regulatory bodies commented on the use of synthetic data, highlighting methodological flaws and granting conditional approval contingent on the provision of data from a confirmatory phase 3 study. Notably, the synthetic data were not included in all HTA submissions. HTA outcomes varied with most agencies expressing uncertainty around the comparative effectiveness. Likely, a lack of specific guidance on the design, use and evaluation of synthetic data and control arms presents a significant barrier to their adoption, leaving both manufacturers and decision-making bodies uncertain about how to objectively assess the quality of the evidence. Overall, the synthetic control arm appeared to have limited impact on the regulatory and HTA outcomes with uncertainties remaining that required additional evidence generation.
Bibliography
European Medicines Agency (EMA) (2015). Assessment report for Blincyto. Procedure No. EMEA/H/C/003731/0000.
Gemeinsamer Bundesausschluss (G-BA) (2016). Tragende Gründe zum Beschluss des Gemeinsamen Bundesausschusses über eine Änderung der Arzneimittel-Richtlinie (AM-RL): Anlage XII - Beschlüsse über die Nutzenbewertung von Arzneimitteln mit neuen Wirkstoffen nach § 35a SGB V – Blinatumomab.
Gemeinsamer Bundesausschluss (G-BA) (2017). Beschluss des Gemeinsamen Bundesausschusses über eine Änderung der Arzneimittel-Richtlinie (AM-RL):Anlage XII – Beschlüsse über die Nutzenbewertung von Arzneimitteln mit neuen Wirkstoffen nach § 35a SGB V – Blinatumomab (Neubewertung nach Fristablauf).
Haute Autorité de Santé (HAS) (2017). Avis de la Commission de la transparence pour blinatumomab.
Jaksa A, Louder A, Maksymiuk C, Vondeling G T, Martin L, Gatto N, Richards E, Yver A, Rosenlund M (2022). “A Comparison of 7 Oncology External Control Arm Case Studies: Critiques From Regulatory and Health Technology Assessment Agencies”, Value Health, 25(12): 1967-1976.
Kokosi T and Harron K (2022). “Synthetic data in medical research”, BMJMED, 1:e000167. doi:10.1136/ bmjmed-2022-000167
Norwegian Medical Products Agency (NOMA) (2016). Hurtig metodevurdering. Blincyto (blinatumomab).
PBAC (2015). Public Summary Document – November 2015 PBAC Meeting. Blinatumomab.
PharmExec (2021). Can Synthetic Control Arms Address Regulatory and HTA Evidence Requirements? Link: https://www.pharmexec.com/view/can-synthetic-control-arms-address-regulatory-and-hta-evidence-requirements [Last accessed: 22.07.24]
Therapeutic Goods Administration (TGA) (2018). Australian Public Assessment Report for blinatumomab.
Written by Sophia Naegeli
Decisive Dialogue 7th August 2024
Get in touch with Decisive Consulting Ltd to explore how we can help navigate your journey through Market Access.