Comparative Performance of State-of-the-Art LLMs on the KDLE: A 2025 Benchmark Study

Introduction and aims: To evaluate the diagnostic and reasoning capabilities of 4 state-of-the-art large language models (LLMs) on the Korean Dental Licensing Examination (KDLE) and to assess their potential as educational tools in dentistry. Methods: Four LLMs—ChatGPT-4o, Claude-4 Opus, Gemini 2.5 Pro, and DeepSeek-V3—were evaluated using official KDLE question sets from 2024 and 2025 (n = 642 questions total). The primary endpoint was overall accuracy across all items, with modality-level and subject-wise analyses conducted as secondary and exploratory assessments. Questions covered 13 dental subjects and included both text-only and image-based items. Performance was analyzed using Cochran's Q test for overall comparisons, McNemar's test for pairwise contrasts, and Cohen's kappa for inter-model agreement. Statistical significance was set at p < .05. Results: All LLMs exceeded the passing threshold of 180 points. ChatGPT-4o (mean score: 251.5), Claude-4 Opus (mean score: 256.5), and Gemini 2.5 Pro (mean score: 270.5) achieved performance approached or exceeding student examinees, while DeepSeek-V3 underperformed (mean score: 218.5) despite passing. Significant performance differences existed among models (Q = 116.40, p < .001), except between ChatGPT-4o and Claude-4 Opus (p > 0.05). All models demonstrated superior performance on text-only versus image-based questions. LLMs consistently outperformed students in Oral Biology but underperformed in Oral and Maxillofacial Radiology. Cohen's kappa revealed substantial inter-model agreement (κ = 0.631-0.778). Conclusion: Contemporary LLMs demonstrate competent performance on standardized dental licensing examinations, with 3 models achieving near-human competency. However, persistent limitations in visual interpretation and clinical reasoning suggest their role should remain supplementary to human expertise in dental education and practice. Clinical Relevance: While LLMs show promise as educational tools for exam preparation and knowledge reinforcement, their limitations in visual interpretation and integrative clinical reasoning necessitate continued human oversight in clinical decision-making contexts.