Objectives<\/p>\n
This study aimed to evaluate the limitations of large language models (LLMs) in orthodontics by comparing their performance on licensing exam questions categorized by knowledge domains and error types.<\/p>\n
Materials and methods<\/p>\n
Deepseek-R1(DS) and ChatGPT-4(GPT) were evaluated using 396 text-based questions from the Chinese National Orthodontic Specialist Licensing Examination. Questions were classified through dual taxonomies: (1) “knowledge domains” including foundational biomechanical principles, cross-disciplinary medical integration, specialized orthodontic theory, and clinical decision-making skills; (2) “error types” including factual inaccuracies, logical deficits, and semantic misinterpretations.<\/p>\n
Results<\/p>\n
DS demonstrated significantly higher overall accuracy than GPT (80.3% vs 52.3%, p\u2009<\u20090.001), with statistically significant differences in foundational knowledge (79.8% vs 43.4%) and cross-disciplinary domains (81.0% vs 53.0%). Factual errors were predominant in both models (DS:57.7%, GPT:69.3%), though DS exhibited higher logical error rates (24.4% vs 16.4%).<\/p>\n
Conclusions<\/p>\n
While DS outperforms GPT in general orthodontic knowledge assessment, both models show limitations in specialized domains requiring clinical reasoning.<\/p>\n
Clinical relevance<\/p>\n
The superior performance of DS in standardized exams suggests potential for AI-assisted decision support in orthodontic training and licensing evaluation. However, the persistent factual errors and domain-specific limitations highlight the necessity for clinician verification in real-world applications. Integrating domain-specific knowledge refinement with logical reasoning modules could enhance LLMs’ clinical utility in orthodontic practice.<\/p>\n","protected":false},"excerpt":{"rendered":"Objectives This study aimed to evaluate the limitations of large language models (LLMs) in orthodontics by comparing their…\n","protected":false},"author":2,"featured_media":301399,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6],"tags":[64,63,5004,28300,106362,96167,18369,166002,172229,105],"class_list":["post-301398","post","type-post","status-publish","format-standard","has-post-thumbnail","category-technology","tag-au","tag-australia","tag-chatgpt","tag-deepseek","tag-dentistry","tag-exam","tag-large-language-model","tag-oral-and-maxillofacial-surgery","tag-orthodontics","tag-technology"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/301398","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/comments?post=301398"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/301398\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media\/301399"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media?parent=301398"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/categories?post=301398"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/tags?post=301398"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}