Large language models and child mortality: opportunities and challenges in answering public queries on under-5 causes

BackgroundReducing under-5 mortality remains a global health priority. Large language models (LLMs) are increasingly used by the public to access medical information. However, current evidence evaluating LLMs’ performance in public-facing child health communication is scarce.MethodsWe selected the top five search terms related to each of the five leading causes of under-5 mortality (prematurity, pneumonia, birth asphyxia, malaria, and diarrhoea) using Google Trends, generating 25 representative public queries. Responses were collected from four LLMs (ChatGPT-4.0, Claude 3.5 Sonnet, Bing AI, and Gemini) and independently evaluated by four pediatricians. We used the DISCERN instrument for information reliability; 5-point Likert scales for accuracy, completeness, and comprehensibility; Flesch Reading Ease (FRE) and Flesch–Kincaid Grade Level (FKGL) indices for readability; and the Patient Education Materials Assessment Tool for Printable Materials (PEMAT-P) for understandability and actionability. Differences among models were evaluated with Kruskal–Wallis and ANOVA tests, with statistical significance set at p < 0.05.ResultsWe found significant performance variations among the four models across most evaluation metrics. Bing AI achieved the highest total DISCERN score (median 42) and the highest reliability subscore (Section A median 28). Claude consistently underperformed across multiple domains. Notably, readability was poor for all models, with high language complexity (mean FKGL score 12.4). Critically, actionability scores were near zero for all models on the PEMAT-P scale, reflecting a universal lack of clear and practical behavioral guidance.ConclusionWhile LLMs can generally provide accurate health information, limitations in readability and actionability restrict their practical application in public health communication. Future development should prioritize language simplification and clearer behavioral guidance to enhance their value in public-facing child health communication.