Readability, understandability, and quality of online education materials and large language models for retrograde cricopharyngeal muscle dysfunction

Türe N., Tahir E., ENVER N.

European Archives of Oto-Rhino-Laryngology, vol.282, no.9, pp.4711-4720, 2025 (SCI-Expanded, Scopus) identifier identifier identifier

  • Publication Type: Article / Article
  • Volume: 282 Issue: 9
  • Publication Date: 2025
  • Doi Number: 10.1007/s00405-025-09628-x
  • Journal Name: European Archives of Oto-Rhino-Laryngology
  • Journal Indexes: Science Citation Index Expanded (SCI-EXPANDED), Scopus, Academic Search Premier, BIOSIS, CAB Abstracts, EMBASE, MEDLINE, Veterinary Science Database
  • Page Numbers: pp.4711-4720
  • Keywords: Artificial intelligence, Deglutition disorders, Health literacy, Patient education, Readability
  • Marmara University Affiliated: Yes

Abstract

Objective: This study aims to evaluate online patient education materials on retrograde cricopharyngeal dysfunction (RCPD) by comparing the readability, understandability, and quality of content generated by large language models (LLM). Method: A web search in December 2024 evaluated 51 online resources and four LLMs (ChatGPT 4.0, Gemini 1.5 Flash, Perplexity GPT-3.5, DeepSeek-V2.5). Readability was analyzed using Readable.io, understandability actionability was assessed using PEMAT, and information quality was assessed using DISCERN. Results: The average readability level of the online material and the LLM responses was at the 11th-12th grade level. The Flesch Reading Ease score was lowest for the LLMs, especially for the DeepSeek-V2.5 model (24.21). While PEMAT understandability scores were adequate for online (82%) and LLMs (79%), actionability was low across all groups (25–37%). DISCERN analyses showed that both sources of information were of limited quality in supporting treatment decisions. Conclusion: This study revealed that both online and LLM-generated materials on RCPD exceeded the recommended readability levels. Although the materials demonstrated acceptable understandability, they exhibited low actionability and inadequate overall quality, emphasizing the need for more patient-centered digital health communication.