Online patient information on temporomandibular disorders provided by UK NHS hospitals: assessment and improvement of readability standards using AI-chatbots

Abstract

Temporomandibular disorders (TMD) are complex conditions that burden patients and healthcare systems. Disparities in health literacy may hinder patient comprehension of online educational materials, potentially influencing outcomes. Artificial intelligence (AI)-driven chatbots offer a promising solution to improve the readability of patient information materials. We assessed the readability of available online materials on TMD in the United Kingdom (UK) and evaluate the ability of three AI-chatbots to improve readability. A search was done of all UK public hospital websites with Oral and Maxillofacial Surgery (OMFS) or Ear, Nose, and Throat (ENT) units for TMD-related patient information. Readability was assessed using five standard scoring systems. Three AI-chatbots (ChatGPT, Claude, and Google Gemini) were used to revise the content to an 11-year-old (sixth-grade) reading level. A total of 31 out of 122 of UK hospital Trusts provided online TMD materials. Of these, 12/31 and 1/31 met the target readability according to the Flesch-Kincaid Grade Level (FKGL), and the Gunning Fog Index/Coleman-Liau Index (GFI/CLI), respectively, with mean (SD) readability at 64.68 (6.79) for Flesch Reading Ease Score (FRES). After AI modification by Gemini, 96.8% met the target readability per FKGL, 54.8% per GFI, and 29.0% per CLI. Gemini improved the mean (SD) score significantly to 82.59 (5.73) (p < 0.001) for FRES, meeting the target readability level. Online patient information on TMD exceeds the recommended Year Six (sixth grade in the US) reading level. AI chatbots, particularly Gemini, can significantly enhance the readability of these materials, enabling them to meet health literacy standards according to certain readability tools.