Automating evaluation of LLM-generated responses to patient questions about rare diseases

Authors

• Min Zhao, Washington University School of Medicine in St. Louis
• Inez Y Oh, Washington University School of Medicine in St. Louis
• Aditi Gupta, Washington University School of Medicine in St. Louis
• Sally Cohen-Cutler, Children's Hospital of Philadelphia
• Kathryn M Harmoney, University of New Mexico
• Albert M Lai, Washington University School of Medicine in St. Louis
• Bryan A Sisk, Washington University School of Medicine in St. Louis

Journal

JAMIA Open

Publication Date

4-1-2026

Volume

9

Issue

2

First Page

ooag054

Document Type

Open Access Publication

DOI

10.1093/jamiaopen/ooag054

Rights and Permissions

Min Zhao, Inez Y Oh, Aditi Gupta, Sally Cohen-Cutler, Kathryn M Harmoney, Albert M Lai, Bryan A Sisk, Automating evaluation of LLM-generated responses to patient questions about rare diseases, JAMIA Open, Volume 9, Issue 2, April 2026, ooag054, https://doi.org/10.1093/jamiaopen/ooag054. https://academic.oup.com/jamiaopen/article/9/2/ooag054/8658421. © The Author(s) 2026. Published by Oxford University Press on behalf of the American Medical Informatics Association. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited.

Recommended Citation

Zhao, Min; Oh, Inez Y; Gupta, Aditi; Cohen-Cutler, Sally; Harmoney, Kathryn M; Lai, Albert M; and Sisk, Bryan A, "Automating evaluation of LLM-generated responses to patient questions about rare diseases." JAMIA Open. 9, 2. ooag054 (2026).
https://digitalcommons.wustl.edu/oa_4/6541

Department

ICTS (Institute of Clinical and Translational Sciences)

Additional Links

Supplemental material is available for this article at publisher site.