The Transformative Potential of Generative AI in Health Technology Assessment
Generative artificial intelligence (AI) is swiftly transforming industries, and health technology assessment (HTA) is no exception. A recent ISPOR Working Group Report, published in Value in Health [1], examines how generative AI, including large language models (LLMs), can improve HTA processes while emphasising the necessity for human oversight. The report, " Generative Artificial Intelligence for Health Technology Assessment: Opportunities, Challenges, and Policy Considerations: An ISPOR Working Group Report," analyses the potential benefits, limitations, and policy considerations surrounding this technology.
Key Areas of Impact
The report highlights three crucial areas in which generative AI could markedly enhance HTA workflows:
1. Systematic Literature Reviews (SLRs)
Generative AI has the potential to streamline systematic literature reviews by:
Automate the development of search terms and strategies, including MeSH terms for biomedical databases.
Assisting in the screening of abstracts and full-text articles, several studies have demonstrated human-comparable accuracy when refined prompts are employed.
Extracting data from studies and producing code for meta-analyses.
Despite these advantages, human verification remains essential, as suggested search terms and data extraction can produce inaccurate results or biases.
2. Real-World Evidence (RWE) Analysis
Generative AI can analyse vast collections of real-world data, including unstructured clinical notes and imaging, to extract valuable insights. Key applications include:
Efficiently processing unstructured electronic health records, including physician notes and radiology reports.
Leveraging "few-shot learning" techniques to extract specific variables from unstructured notes.
Utilising domain-specific LLMs that are trained on clinical texts to improve accuracy.
However, potential risks include inaccuracies and AI-generated fabrications, necessitating rigorous validation before adoption in decision-making.
3. Health Economic Modeling
Generative AI can assist in developing and validating health economic models by:
Assisting in the conceptualisation, parameterisation, implementation, and evaluation of models.
Reproducing published models with great accuracy.
Automating parts of model development, leading to increased efficiency.
Despite these advantages, human expertise remains crucial for ensuring appropriate conceptualisation and parameterisation. AI serves as a supportive tool rather than a substitute for human analysts.
Challenges and Limitations
While generative AI presents immense potential for HTA, the ISPOR report outlines key challenges:
Scientific Validity and Reliability
Generative AI models can generate factual inaccuracies, known as "hallucinations". Prompt engineering, retrieval-augmented generation, and fine-tuning with domain-specific data can help mitigate these errors.
Bias, Equity, and Fairness
Bias can emerge during data collection, training, or deployment, leading to inequities in healthcare assessments. Strategies such as data augmentation, reweighting, and algorithmic fairness measures can help mitigate these risks.
Regulatory and Ethical Considerations
Regulatory frameworks for generative AI in biomedical research are still evolving. Compliance with data privacy laws such as HIPAA and GDPR is crucial, particularly regarding the use of patient-level data. Ethical concerns also arise regarding using AI models to infer sensitive patient characteristics.
Recommendations for HTA Agencies
The ISPOR report presents several actionable recommendations for HTA agencies seeking to integrate generative AI into their workflows:
Develop Clear Guidance: Establish guidelines on the appropriate and inappropriate use of LLMs in HTA.
Standardise and Harmonise Processes: Ensure consistency in AI-driven HTA processes and maintain transparent reporting.
Prioritise Health Equity: Incorporate equity considerations, ensuring representation of historically marginalised populations.
Invest in Training: Enhance AI literacy among HTA professionals to maximise the responsible use of generative AI.
Encourage Multistakeholder Collaboration: Foster partnerships to develop best practices and incorporate patient perspectives.
Assess LLM Qualification: Explore qualification or accreditation mechanisms to enhance AI-driven HTA submissions.
The Future of Generative AI in HTA
The report concludes that human supervision remains essential, while generative AI presents transformative potential for HTA. As AI models and user expertise develop, ongoing evaluation of AI applications in HTA will be crucial to ensure responsible and effective implementation.
Conclusion
The ISPOR Working Group Report underscores the potential of generative AI to enhance HTA efficiency and evidence generation. However, responsible and ethical implementation necessitates addressing AI limitations and potential biases, alongside ensuring continuous human oversight. This report serves as a vital resource for HEOR professionals, HTA agencies, regulators, and manufacturers navigating the integration of generative AI into healthcare decision-making.
Are you exploring how AI can optimise your HTA strategy? Reach out to Decisive Consulting via: enquiries@decisiveconsulting.co.uk for expert insights on harnessing generative AI while ensuring regulatory compliance and ethical best practices.
Key Definitions
Generative AI: AI systems that produce text, images, or other content based on input data.
Foundation Models: Large-scale pre-trained AI models capable of adapting to various tasks.
Few-Shot Learning: A machine learning approach where an AI model learns to make accurate predictions from a very small number of labelled examples.
LLMs: A subset of foundation models trained on extensive text datasets to recognise, summarise, translate, and generate text.
Prompt Engineering: The practice of crafting precise prompts to direct AI models in producing desired outputs.
Retrieval-Augmented Generation: AI technique that enhances responses by first retrieving relevant external information before generating an answer. This makes the AI more accurate, up-to-date, and reliable, especially for specialised or rapidly changing topics.
Bibliography
Fleurence, R., Bian, J., et al (2024). Generative AI for Health Technology Assessment: Opportunities, Challenges, and Policy Considerations. Value in Health. 2025; 28(2):175–183
Written by Lance Richard
Decisive Edge 18th February 2025