Leveraging Structured Expert Elicitation in Highly Specialised Technology Evaluations of Ultra-Orphan Medicines

As we look to commemorate World Rare Disease Day on 29th February, we are reminded of the profound challenges faced by individuals and families affected by ultra-rare diseases. At Decisive Consulting, we recognise the critical importance of equitable access to innovative treatments in these uniquely challenging healthcare landscapes.

In our latest Decisive Dialogue, we explore the role of structured expert elicitation in highly specialised technology evaluations to fill data gaps and enhance evidence packages for the assessment of ultra-rare medicines.

NICE stands out among HTA bodies for its dedicated pathway for evaluating “ultra-orphan” medicines, known as the Highly specialised technologies (HST) process. “Ultra-orphan” medicines are defined as those which target a condition affecting no more than 1 in 50,000 individuals in England and effectively means that no more than 300 individuals will qualify for a medicine eligible for the HST process.

The process was developed to address the challenges associated with assessing medicines for “ultra-orphan” diseases, specifically relating to the typical scarcity of data. Small heterogeneous patient populations complicate clinical trials due to recruitment difficulties or ethical concerns and Phase III trials often lack comparative effectiveness data leading to single-arm Phase II studies with limited statistical analysis. In addition, the absence of validated endpoints as well as appropriate comparators hinders the prediction of long-term effects and economic modelling is complicated by a paucity in natural history data.

Given the challenges, NICE indicates that methods of expert elicitation may be used to fill data gaps by capturing expert opinions and associated uncertainties. In the updated 2022 NICE Process and Methods Manual (1), NICE provided more guidance on expert elicitation. Specifically a preference towards “structured” expert elicitation (SEE) following existing protocols over “unstructured” elicitation such as expert interviews or advisory boards. NICE highlight that SEE protocols should involve the assessment of probability distributions, usually after responders have been trained on common cognitive biases. Commonly used SEE protocols include Cooke’s classical method (2), the Sheffield elicitation framework (SHELF) (3) and the modified Delphi protocol (4). The medical research council (MRC) protocol (5) has also recently been developed specifically for use of SEE in HTA submissions and was highlighted in the updated manual as an example protocol.

Prior to the release of the updated NICE Process and Methods Manual, the Delphi protocol (often referred to as a Delphi panel) was predominantly used for expert elicitation in HSTs. Delphi panels can be a valuable source of evidence as the iterative nature of Delphi panels facilitates consensus building among experts. Indeed, the Delphi protocol has been used to provide expert insight into utility values, modelling assumptions and parameters, health state transition probabilities, healthcare resource utilisation costs and hazard ratios. However, despite valuable insights gathered from a Delphi panel, there often remains a level of uncertainty as simple estimates are often elicited over probability distributions.

Since this update to the NICE manual, SEE utilising the elicitation of uncertainty through probability distributions have been included in four HST evaluations and all four resulted in a positive recommendation. These medicines included velmanase alfa (SHELF) (6), birch bark extract (IDEA) (7,8), lumasiran (SHELF) (9) and atidarsagen autotemcel (undefined protocol) (10).   

Figure 1 demonstrates the updates of NICE processes and methods in 2022

This analysis demonstrates that SEE utilising the elicitation of uncertainty through assessment of probability distributions is gaining traction in HST evaluations, and indeed, a review of the Committee Papers shows a clear assessment and critique of the SEE protocol submitted. This analysis demonstrates that NICE is willing to accept SEE when evaluating HSTs but only when a clear and robust protocol is followed.

With limited clinical data available for rare disease, SEE is a valuable technique to fill evidence gaps and inform decision-making. Pharmaceutical companies should consider integrating SEE into their evidence package to enhance submissions and facilitate access to orphan medicines.  

Decisive Consulting specialises in HTA strategy and developing evidence generation plans with the needs of HTA bodies in mind. Reach out to us if you are interested in understanding how SEE protocols can support your access decisions.

References:

1.      Introduction to health technology evaluation | NICE health technology evaluations: the manual | Guidance | NICE. www.nice.org.uk. Published January 31, 2022. Accessed February 19, 2024. https://www.nice.org.uk/process/pmg36/

2.      Cooke R, Mendel M, Thijs W. Calibration and information in expert resolution; a classical approach. Automatica. 1988;24(1):87-93. doi:https://doi.org/10.1016/0005-1098(88)90011-8

3.      Gosling, J.P. (2018). SHELF: The Sheffield Elicitation Framework.

4.      Lecours A. Scientific, professional and experiential validation of the model of preventive behaviours at work: protocol of a modified Delphi Study. BMJ Open. 2020 Sep 13;10(9):e035606. doi: 10.1136/bmjopen-2019-035606. PMID: 32928848; PMCID: PMC7488793.

5.      Bojke L, Soares M, Claxton K, Colson A, Fox A, Jackson C, Jankovic D, Morton A, Sharples L, Taylor A. Developing a reference protocol for structured expert elicitation in health-care decision-making: a mixed-methods study. Health Technol Assess. 2021 Jun;25(37):1-124. doi: 10.3310/hta25370. PMID: 34105510; PMCID: PMC8215568.

6.      Overview | Velmanase alfa for treating alpha-mannosidosis | Guidance | NICE. www.nice.org.uk. Published December 13, 2023. Accessed February 19, 2024. https://www.nice.org.uk/guidance/hst29

7.      Overview | Birch bark extract for treating epidermolysis bullosa | Guidance | NICE. www.nice.org.uk. Published September 20, 2023. Accessed February 19, 2024. https://www.nice.org.uk/guidance/hst28

8.      Hemming V, Burgman MA, Hanea AM, McBride MF, Wintle BC. A practical guide to structured expert elicitation using the IDEA protocol. Anderson B, ed. Methods in Ecology and Evolution. 2017;9(1):169-180. doi:https://doi.org/10.1111/2041-210x.12857

9.      Overview | Lumasiran for treating primary hyperoxaluria type 1 | Guidance | NICE. www.nice.org.uk. Published April 19, 2023. Accessed February 19, 2024. https://www.nice.org.uk/guidance/hst25

10.  Overview | Atidarsagene autotemcel for treating metachromatic leukodystrophy | Guidance | NICE. Accessed February 19, 2024. https://www.nice.org.uk/guidance/hst18

Abbreviations:

HST, Highly Specialised Technologies; HTA, Health Technology Assessment; MRC, Medical Research Council; NICE, National Institute for Health and Care Excellence; SHELF, Sheffield Elicitation Framework.

Written by Rosa Willock and Sophia McGovern

Decisive Dialogue 28th February 2024

Get in touch with Decisive Consulting Ltd to explore how we can help navigate your journey through Market Access.