FP4

Ophthalmic Triage in the Age of AI: Assessing Chatbot Accuracy and Utility

Study Design: Cross-sectional study.

Purpose: To evaluate the efficacy of AI chatbots in diagnosis, triage, and referral of ophthalmic conditions.

Methods: Twenty standardized clinical vignettes were assessed by ophthalmology trainees and AI models (DeepSeek R1, GPT-4.1, Claude 4.0, Gemini 2.5 Pro). Outcomes included accuracy of top diagnoses, triage urgency, and referral advice.
Results: Human trainees showed highest diagnostic accuracy (92%), followed by GPT-4.1 (84%) and Claude 4.0 (83%). Triage accuracy was highest for trainees (94%), with GPT-4.1 (88%) and Claude 4.0 (86%) performing best among AI. Referral accuracy was also highest in trainees (93%), while GPT-4.1 (86%) and Claude 4.0 (85%) led among AI. DeepSeek R1 and Gemini 2.5 Pro showed lower consistency and occasional unsafe recommendations.

Conclusion: AI chatbots demonstrate promising utility in ophthalmic triage but remain inferior to human expertise, especially in complex cases.