Patients derive numerous benefits from reading their clinical notes,
including an increased sense of control over their health and improved
understanding of their care plan. However, complex medical concepts and jargon
within clinical notes hinder patient comprehension and may lead to anxiety. We
developed a patient-facing tool to make clinical notes more readable,
leveraging large language models (LLMs) to simplify, extract information from,
and add context to notes. We prompt engineered GPT-4 to perform these
augmentation tasks on real clinical notes donated by breast cancer survivors
and synthetic notes generated by a clinician, a total of 12 notes with 3868
words. In June 2023, 200 female-identifying US-based participants were randomly
assigned three clinical notes with varying levels of augmentations using our
tool. Participants answered questions about each note, evaluating their
understanding of follow-up actions and self-reported confidence. We found that
augmentations were associated with a significant increase in action
understanding score (0.63 $pm$ 0.04 for select augmentations, compared to 0.54
$pm$ 0.02 for the control) with p=0.002. In-depth interviews of
self-identifying breast cancer patients (N=7) were also conducted via video
conferencing. Augmentations, especially definitions, elicited positive
responses among the seven participants, with some concerns about relying on
LLMs. Augmentations were evaluated for errors by clinicians, and we found
misleading errors occur, with errors more common in real donated notes than
synthetic notes, illustrating the importance of carefully written clinical
notes. Augmentations improve some but not all readability metrics. This work
demonstrates the potential of LLMs to improve patients’ experience with
clinical notes at a lower burden to clinicians. However, having a human in the
loop is important to correct potential model errors.
Patient comprehension of clinical notes is crucial for their understanding of their health and care plans. However, complex medical language often hinders this comprehension and may lead to patient anxiety. In order to address this issue, researchers have developed a patient-facing tool that utilizes large language models (LLMs) to simplify and add context to clinical notes.
The Study Methodology
The researchers used a specially engineered version of the GPT-4 language model to perform these augmentation tasks on both real and synthetic clinical notes. The study involved 200 female-identifying breast cancer survivors in the US who were randomly assigned three clinical notes with varying levels of augmentations using the tool.
Participants were asked to answer questions about each note, evaluating their understanding of follow-up actions and self-reported confidence. The results showed that augmentations were associated with a significant increase in action understanding score compared to the control group.
Patient Feedback
Furthermore, in-depth interviews were conducted with seven self-identifying breast cancer patients. The participants responded positively, especially regarding the definitions provided through the augmentations. However, some concerns were raised about relying solely on LLMs for augmentations.
The Role of Clinicians
Clinicians played an important role in evaluating the augmentations for errors. It was found that misleading errors were more common in real donated notes compared to synthetic notes, emphasizing the importance of carefully written clinical notes. While augmentations improved some readability metrics, it is important to have a human in the loop to correct potential errors made by the language model.
The Multidisciplinary Nature
This study highlights the multidisciplinary nature of improving patient experience with clinical notes. It combines expertise from natural language processing, healthcare, and patient advocacy. The use of LLMs offers a promising solution to enhance patient comprehension and control over their health, but it also underscores the need for collaboration between technology and human expertise.
Future Implications
Going forward, further research is needed to address the concerns raised by patients regarding reliance on LLMs. Providing clear definitions and explanations is crucial, but finding the right balance between pre-generated augmentations and real-time, clinician-generated notes will be critical. Additionally, ongoing collaboration between researchers, clinicians, and patients can help refine and improve the tools developed to ensure they meet the diverse needs of patients.
This work demonstrates the potential of LLMs to improve patients’ experience with clinical notes at a lower burden to clinicians. However, having a human in the loop is important to correct potential model errors.