{"id":174490,"date":"2025-09-22T18:22:07","date_gmt":"2025-09-22T18:22:07","guid":{"rendered":"https:\/\/www.newsbeep.com\/us\/174490\/"},"modified":"2025-09-22T18:22:07","modified_gmt":"2025-09-22T18:22:07","slug":"ai-llm-models-pass-cfa-level-iii-exam","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/us\/174490\/","title":{"rendered":"AI LLM Models Pass CFA Level III Exam"},"content":{"rendered":"

In 2024, a study by J.P. Morgan AI Research and Queen\u2019s University<\/a> found that leading proprietary artificial intelligence models could pass the CFA Level I and II mock exams, but they struggled with the essay portion of the Level III exam. A new research study<\/a> has found that today\u2019s leading large language models can now clear the CFA Level III exam, including the essay portion. The CFA Level III is widely known as one of the most difficult professional exams in the finance industry.<\/p>\n

The new research was conducted by the NYU Stern School of Business and Goodfin, an AI wealth platform for exclusive private market investments. It set out to assess the capabilities of large language models in specialized domains like finance.<\/p>\n

The study<\/a>, Advanced Financial Reasoning at Scale: A Comprehensive Evaluation of Large Language Models on CFA Level III, benchmarked 23 leading AI models, including Open AI\u2019s GPT-4, Google\u2019s Gemini 2.5 and Anthropic\u2019s Claude Opus 4, against the CFA Level III mock exam. LLMs are a subset of generative AI that are applied to perform language-related tasks specifically.<\/p>\n

The study found that Open AI\u2019s o4-mini model had a composite score of 79.1%, while Gemini\u2019s 2.5 Flash model scored 77.3%. While most models performed well on multiple-choice questions, only a few excelled at the essay prompts, requiring analysis, synthesis and strategic thinking.<\/p>\n

Related:The WealthStack Podcast: AI, Advice and the Future of Financial Planning with Ken Lotocki<\/a><\/p>\n

That said, NYU Stern Professor Srikanth Jagabathula said the reasoning-based LLMs of the recent past have shown immense capabilities in performing tasks that require a lot of quantitative and critical thinking, such as the essay portion. The models now have the ability to think through the problem and provide a reasoning for the response.<\/p>\n

To grade the essay portion, Jagabathula had another LLM act as a judge, giving the LLM the essay response, the true response, some context about the question and a grading rubric. He also had that same set of responses graded by a certified human grader. They found that the LLM was actually stricter than the human, assigning fewer overall points to the same question.<\/p>\n

\u201cWe thought they would be more lenient in assigning their grades, but we found in this case at least that it is the other way around,\u201d he said.<\/p>\n

The study also found that prompting the AI models drove performance on the essay portion. Specifically, they used chain-of-thought prompting, which involves asking the LLM to think through the response and provide a reasoning. That process yielded a better answer from the LLM than a more direct response. In fact, this boosted essay accuracy by 15 percentage points.<\/p>\n

Related:Q&A: What\u2019s Behind Mark Casady\u2019s New Role at FMG<\/a><\/p>\n

In response to the study\u2019s results, Chris Wiese, managing director of education at CFA Institute, pointed out that the qualifications for the CFA designation go beyond passing all three exams. They also require 4,000 hours of qualifying work experience, a minimum of two references, an attestation of following the CFA code of ethics and standards, and completion of hands-on practical skills modules.<\/p>\n

\u201cWithout knowing the details of how this study was conducted, we can only note that at CFA Institute, we continue to believe that a combination of trust, human relationships, sound ethical judgment and professionalism are as important as ever in financial markets,\u201d Wiese said. \u201cOur own research shows that AI will continue to grow in utility and efficacy for investment managers\u2014just as it is across a range of disciplines and industries\u2014and we are committed to keeping our members, candidates and the profession abreast of these opportunities.\u201d<\/p>\n

When asked whether an LLM could perform the job of a CFA professional, Jagabathula said it\u2019s difficult to forecast what capabilities the models will develop. But he pointed to some preliminary results of a small-scale study he\u2019s conducting, in which a set of users were asked to interact with both the AI model and a human for financial advice.<\/p>\n

Related:SigFig Rebrands as Tandems; Rolls Out AI-Embedded Tools<\/a><\/p>\n

\u201cWhat we found was, the LLM was often quite good at giving very precise answers to specific questions for which there was a precise answer, but they often struggled in capturing context that was not very explicitly stated by the user. And at least in some cases they could not,\u201d he said. \u201cThe end user found it a little bit difficult to trust the system. So, as of now, it seems clear that LLMs can significantly augment the abilities of existing financial professionals. As to whether they can actually replace them, the jury is still out.\u201d<\/p>\n","protected":false},"excerpt":{"rendered":"In 2024, a study by J.P. Morgan AI Research and Queen\u2019s University found that leading proprietary artificial intelligence…\n","protected":false},"author":2,"featured_media":174491,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[45],"tags":[182,181,507,74],"class_list":{"0":"post-174490","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-artificial-intelligence","8":"tag-ai","9":"tag-artificial-intelligence","10":"tag-artificialintelligence","11":"tag-technology"},"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/posts\/174490","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/comments?post=174490"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/posts\/174490\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/media\/174491"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/media?parent=174490"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/categories?post=174490"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/tags?post=174490"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}