European Archives of Oto-Rhino-Laryngology, vol.282, no.9, pp.4711-4720, 2025 (SCI-Expanded, Scopus)
Objective: This study aims to evaluate online patient education materials on retrograde cricopharyngeal dysfunction (RCPD) by comparing the readability, understandability, and quality of content generated by large language models (LLM). Method: A web search in December 2024 evaluated 51 online resources and four LLMs (ChatGPT 4.0, Gemini 1.5 Flash, Perplexity GPT-3.5, DeepSeek-V2.5). Readability was analyzed using Readable.io, understandability actionability was assessed using PEMAT, and information quality was assessed using DISCERN. Results: The average readability level of the online material and the LLM responses was at the 11th-12th grade level. The Flesch Reading Ease score was lowest for the LLMs, especially for the DeepSeek-V2.5 model (24.21). While PEMAT understandability scores were adequate for online (82%) and LLMs (79%), actionability was low across all groups (25–37%). DISCERN analyses showed that both sources of information were of limited quality in supporting treatment decisions. Conclusion: This study revealed that both online and LLM-generated materials on RCPD exceeded the recommended readability levels. Although the materials demonstrated acceptable understandability, they exhibited low actionability and inadequate overall quality, emphasizing the need for more patient-centered digital health communication.