Fellowship Bio

I am an ST3 Public Health Registrar with an analytical background working across Local Authority, NHS and the UK Health Security Agency. I have an interest in analytics and AI, and the potential they have to improve population health and reduce health inequalities.

Fellowship Project

Supporting a Global AI Community of Practice through Evaluation Clinics

University Hospitals Birmingham NHS Foundation Trust

The University of Birmingham supported a global community of practice by launching evaluation clinics to provide expertise to research teams who were undertaking evaluations of AI technologies, which included using Large Language Models (LLMs). Each evaluation clinic session was designed to provide tailored guidance and support from academics, to enhance the quality of the evidence base and strengthen the evaluation methodologies being used in AI research. My role involved: Designing a survey to gather appetite from teams for an evaluation clinic; Undertaking initial scoping meetings with each selected team to gather more information on the project and identify where support should be focused; Attending, engaging and supporting in evaluation clinics; Upon completion of the clinics, mapping knowledge and identifying gaps. Progress to date: Five out of eight evaluation clinics have been performed, with the need to understand and establish a robust evaluation method the most common obstacle cited from teams.

Fellowship Testimonial

This Fellowship has increased my understanding of the AI subject area whilst consolidating some of my Public Health skills. I have enjoyed working on clinical AI projects, supported by a network of academic colleagues from a variety of backgrounds. I believe the practical experience I have gained has enhanced my skillset to support the operationalisation of AI in the wider health system, which I intend to utilise as I progress in my career. Working on the Community of Practice has allowed me to gain a greater understanding of processes used to evaluate LLMs in an academic domain and given me my first experience of working with colleagues across a global scale. This was a great opportunity to understand how AI is being proposed in low- and middle-income countries (e.g. Nigeria, India, Kenya) and I enjoyed identifying the common obstacles that teams were facing in their research, such as the regulation of medical devices, lack of published validation frameworks, translational issues and the time commitment required for comprehensive evaluations and how this compares to operational need. Other work I have been involved in include developing a budget impact analysis to examine the costs of introducing an AI-derived software. This has identified the difficulty in determining and quantifying current practice and the evidence gap that exists in understanding the delivery of real-world care. A further piece of work I was involved in examined the exploration of implementing a new AI technology in a hospital department. This allowed me to consider the impact of vender lock in, that decisions should be made at a system-level and the need to understand any unintended consequences of AI commissioning, such as time, cost, training, IG and IT connotations.