ChatGPT-4.5 versus pharmacists in OTC counseling: A bilingual comparative study

Background: Over-the-counter (OTC) medications enable self-care but require accurate counseling to prevent misuse, particularly in multilingual settings with limited health literacy. Pharmacists provide expert guidance; however, time constraints hinder comprehensive interactions. Generative artificial intelligence (AI) models such as ChatGPT-4.5 offer potential as decision-support tools, but their performance compared with pharmacists in bilingual OTC counseling remains underexplored. This study compared ChatGPT-4.5 and community pharmacists using standardized gastrointestinal case scenarios in English and Arabic to assess response quality, safety, and appropriateness. Methods: This cross-sectional study evaluated 22 standardized OTC cases (e.g., constipation, diarrhea) from authoritative sources, presented in English and Arabic. Fifty-eight Jordanian community pharmacists (n = 29 per language) solved five cases each. All 44 AI responses were generated via standardized prompts without iteration. Responses were blindly rated by four experts using the CLEAR framework (completeness, lack of false information, evidence-based, appropriateness, and relevance; 5-point Likert scale). Cosine similarity quantified textual overlap; Welch’s t-tests, ANOVA, and correlations analyzed differences (SPSS v.29; α = 0.05). Inter-rater reliability was assessed using ICC. Results: ChatGPT-4.5 achieved higher mean CLEAR scores (4.8/5) than pharmacists (3.7–3.8/5; p < 0.001) across languages and domains, with no language differences (p > 0.05). AI excelled in completeness and lack of false information; pharmacist performance declined with case difficulty. Cosine similarity was low (0.04–0.11), indicating distinct yet coherent AI phrasing. Younger pharmacists scored higher (r = -0.33, p = 0.01). Qualitative analysis showed more consistent structure in AI-generated responses and greater variability among pharmacists. Conclusion: ChatGPT-4.5 demonstrates robust, language-agnostic performance for OTC counseling, suggesting its role as an educational benchmark and clinical adjunct to standardize advice and reduce variability. Pharmacy curricula should integrate AI literacy and targeted training in red-flag recognition to enhance human–AI synergy, fostering safer patient outcomes and interprofessional collaboration.