new study<\/a> by researchers from Brown University they presented on at the International Conference on Learning Representations in Rio de Janeiro, Brazil.<\/p>\nThe study looked under the hood of several AI language models to look for signs that they know the difference between events and scenarios that are commonplace, unlikely, impossible, or downright nonsense.<\/p>\n
\u201cThis work reveals some evidence that language models have encoded something like the causal constraints of the real world,\u201d says Michael Lepori, a PhD candidate at Brown who led the work.<\/p>\n
\u201cBeyond just encoding these constraints, they do so in a way that is predictive of human judgments of these categories.\u201d<\/p>\n
Lepori\u2019s research explores the intersection of computer science and human cognition. He is advised by Ellie Pavlick, a professor of computer science, and Thomas Serre, a professor of cognitive and psychological sciences, both of whom are faculty affiliates of Brown\u2019s Carney Institute for Brain Science and coauthors of the research.<\/p>\n
For the study, the researchers designed an experiment to test how language models interpret sentences describing events of varying plausibility. Some statements described commonplace scenarios: For example, \u201cSomeone cooled a drink with ice.\u201d Some scenarios were improbable or unlikely: \u201cSomeone cooled a drink with snow.\u201d Some were impossible: \u201cSomeone cooled a drink with fire.\u201d Some were nonsensical: \u201cSomeone cooled a drink with yesterday.\u201d<\/p>\n
For each input, the researchers examined the resulting mathematical states generated inside the AI model, an approach known as mechanistic interpretability.<\/p>\n
\u201cMechanistic interpretability can be appropriately characterized as something like neuroscience for AI systems,\u201d Lepori says.<\/p>\n
\u201cIt seeks to reverse-engineer what the model is doing when exposed to a particular input. You could kind of think about it as understanding what is encoded in the \u2018brain state\u2019 of the machine.\u201d<\/p>\n
By comparing the differences in \u201cbrain states\u201d generated by pairs of sentences from different categories\u2014commonplace versus improbable, improbable versus impossible, and so on\u2014the researchers could get a sense of whether, and how well, the models internally differentiate between categories. The experiments were repeated across several different open-source language models, including Open AI\u2019s GPT 2, Meta\u2019s Llama 3.2, and Google\u2019s Gemma 2, to get a \u201cmodel-agnostic\u201d sense of how well these types of models distinguish between categories.<\/p>\n
The study found that models of sufficient size do indeed develop distinct mathematical patterns, or vectors, that are strongly correlated with each plausibility category. The vectors could distinguish between even the most similar of categories\u2014like improbable versus impossible events\u2014with roughly 85% accuracy.<\/p>\n
What\u2019s more, Lepori says, the vectors revealed by the study are reflective of human uncertainty about which category a statement might fall into. Take the statement, \u201cSomeone cleaned the floor with a hat,\u201d for example. When people hear that statement, they may disagree about whether it represents something that\u2019s impossible or just unlikely. For the study, the researchers analyzed the vectors to see how ambiguous the AI systems thought these statements were, and compared that with survey results from human participants.<\/p>\n
\u201cWhat we show is that the models actually capture that human uncertainty pretty well,\u201d Lepori says. \u201cIn cases where, say, 50% of people says a statement was impossible and 50% says it was improbable, the models were assigning roughly 50% probability as well.\u201d<\/p>\n
Taken together, the results suggest that modern AI language models can indeed develop an understanding of the real world that is reflective of human understanding. These vectors start to emerge in models with more than 2 billion parameters, the research found, which is fairly small compared to today\u2019s trillion-plus-parameter models.<\/p>\n
More broadly, the researchers say these kinds of mechanistic interpretability studies can help in developing a better understanding of what AI models know and how they came to know it.<\/p>\n
And that, the researchers say, will help in developing smarter, more trustworthy models.<\/p>\n
Source: Brown University<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"Share this Article You are free to share this article under the Attribution 4.0 International license. New research…\n","protected":false},"author":2,"featured_media":412297,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[345,343,344,85,46,125],"class_list":["post-412296","post","type-post","status-publish","format-standard","has-post-thumbnail","category-artificial-intelligence","tag-ai","tag-artificial-intelligence","tag-artificialintelligence","tag-il","tag-israel","tag-technology"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/posts\/412296","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/comments?post=412296"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/posts\/412296\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/media\/412297"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/media?parent=412296"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/categories?post=412296"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/tags?post=412296"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}