Validation and quality assurance of AI-generated discharge letters at Biel’s hospital centre

Validex investigates which automated quality metrics align with the feedback from medical experts on AI-generated discharge letters at Biel’s hospital centre, Spitalzentrum Biel.

Factsheet

Duration (planned) 01.09.2026 - 31.03.2027
Head of project Prof. Daniel Reichenpfader
Project staff Albnora Seljmani, Spitalzentrum Biel
François Panosetti, Spitalzentrum Biel

Baseline

AI systems are being increasingly used to summarise clinical information and draft discharge letters more efficiently. At the same time, there is a lack of validated automated metrics capable of ensuring the quality of these summaries. In clinical practice, it is therefore essential not only that quality metrics can be computed, but also that they accurately reflect expert assessments.

Goals

The primary objective of Validex is to identify which automated quality metrics align with the feedback from medical experts. To this end, AI-generated discharge letters are assessed by experts and compared against a range of metrics. Based on this, the project will determine which methods are suitable for ensuring the quality of clinical AI-generated summaries in routine practice and for their long-term integration into clinical workflows.

Course of Action

As part of the project, AI-generated discharge letters are first reviewed, corrected and assessed by clinical specialists. Concurrently, Validex calculates various automated quality metrics. In a second step, the project will assess which metrics best reflect the expert assessments. The findings will form the foundation for quality assurance in clinical practice and the subsequent integration of the system into routine clinical workflows.

Results

Validex combines a web-based annotation and validation portal with automated quality metrics. The solution shows how well individual metrics align with expert feedback, thereby helping to select appropriate methods for reliable quality assurance. The objective is to develop an application that integrates into clinical workflows and supports the safe use of AI-based summaries.