{"id":292725,"date":"2025-11-15T07:03:10","date_gmt":"2025-11-15T07:03:10","guid":{"rendered":"https:\/\/www.newsbeep.com\/us\/292725\/"},"modified":"2025-11-15T07:03:10","modified_gmt":"2025-11-15T07:03:10","slug":"scientist-turns-peoples-mental-images-into-text-using-mind-captioning-technology","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/us\/292725\/","title":{"rendered":"Scientist turns people\u2019s mental images into text using \u2018mind-captioning\u2019 technology"},"content":{"rendered":"

\n A scientist in Japan<\/a> has developed a technique that uses brain scans and artificial intelligence to turn a person\u2019s mental images into accurate, descriptive sentences.\n <\/p>\n

\n While there has been progress in using scans of brain activity to translate the words we think<\/a> into text, turning our complex mental images into language has proved challenging, according to Tomoyasu Horikawa, author of a study published November 5 in the journal Science Advances<\/a>.\n <\/p>\n

\n However, Horikawa\u2019s new method, known as \u201cmind-captioning,\u201d works by using AI to generate descriptive text that mirrors information in the brain about visual details such as ob\ufeffjects, places, actions and events, as well as the relationships between them.\n <\/p>\n

\n Horikawa, a researcher at telecommunication company NTT\u2019s Communication Science Laboratories just outside Tokyo, began by analyzing the brain acti\ufeffvity of four men and two women, native Japanese speakers between 22 and 37 years old, scanning their brains as they watched video clips. The participants viewed 2,180 videos without sound that were seconds long and varied in content among objects, scenes and actions.\n <\/p>\n

\n Large language models \u2014 generative AI systems trained on large datasets \u2014 took captions of the video clips and turned those captions into sequences of numbers.\n <\/p>\n

\n Horikawa trained separate, simpler AI models, known as \u201cdecoders,\u201d to match the scanned brain activity related to the video clips to the numerical sequences.\n <\/p>\n

\n He then used the decoders to interpret the study participants\u2019 brain activity while they watched or recalled videos that the AI had not encountered during the training process. Another algorithm was created to progressively generate word sequences that best matched the decoded brain activity.\n <\/p>\n

\n As the AI learned from the data, the descriptive text tool became better and better at using the brain scans to describe the videos the participants had watched.\n <\/p>\n

\n \u201cIt\u2019s just one additional step forward in the direction of what, in my view, we can legitimately call brain-reading or mind-reading,\u201d Marcello Ienca, a professor of the ethics of AI and neuroscience at Technical University of Munich in Germany and president-elect of the International Neuroethics Society, told CNN. He was not involved in the study.\n <\/p>\n

\n The AI model generated text in English, even though the participants were not native English speakers.\n <\/p>\n

\n The method can create comprehensive descriptions of visual content, even without using the activity in language-related regions of the brain, or the \u201clanguage network,\u201d Horikawa said, \u201cindicating that this method can be used even when someone has damage around that language network.\u201d\n <\/p>\n

\n The technology could potentially be used to assist people with aphasia, who struggle with language expression due to damage around the language network; or amyotrophic lateral sclerosis (ALS)<\/a>, a progressive neurodegenerative disease that affects speech, according to the study.\n <\/p>\n

\n \u201cI think this study paves the way for some profound interventions for people who have difficulty communicating, including non-verbal autistic people,\u201d said psychologist Scott Barry Kaufman, a lecturer at Barnard College in New York who was not involved in the study.\n <\/p>\n

\n However, \u201cwe have to use it carefully and make sure we aren\u2019t being invasive and that everyone consents to it,\u201d he told CNN.\n <\/p>\n

\n The success of this method \u2014 which could be applied to decode the thoughts of infants or animals, or the content of dreams \u2014 \u201craises ethical concerns\u201d regarding privacy, with the possibility of disclosing an individual\u2019s private thoughts before they have verbalized them, the study noted.\n <\/p>\n

\n If in the future this technology is used by consumers beyond biomedical purposes, \u201cI think this is the ultimate privacy challenge,\u201d Ienca said.\n <\/p>\n

\n He added that there are many companies, such as Neuralink, Elon Musk\u2019s brain implant startup<\/a>, that are making public claims about soon developing neural implants for the general population.\n <\/p>\n

\n \u201cIf we get there, then we need to have very, very strict rules when it comes to granting access to people\u2019s minds and brains,\u201d Ienca said, highlighting that our brains include \u201csensitive information\u201d such as \u201csignatures of early dementia and psychiatric disorders and depression.\u201d\n <\/p>\n

\n A study published in the journal Cell<\/a> in August suggested that the \u201cleakage\u201d of private inner thoughts during decoding could be prevented by a mechanism in which the user thinks of a particular keyword to unlock the decoding tool only when intended.\n <\/p>\n

\n The \u201cneuroscience is moving fast and the assistive potential is huge \u2014 but mental privacy and freedom of thought protections can\u2019t wait,\u201d said social scientist \u0141ukasz Szoszkiewicz, an assistant professor at Adam Mickiewicz University in Poland and a director of European Affairs at the Neurorights Foundation in New York.\n <\/p>\n

\n \u201cWe should treat neural data as sensitive by default, require explicit purpose-limited consent, and prioritize on-device processing with user-controlled \u2018unlock\u2019 mechanisms. Reliance on AI introduces additional regulatory and cybersecurity challenges and underscores the need for complementary, AI-specific legal framework,\u201d Szoszkiewicz, who was not involved in the study, told CNN.\n <\/p>\n

\n However, Horikawa noted that the method used in his study requires a large amount of data collection, with the cooperation of active participants. So, while the technology is useful for neuroscientific research, it is \u201cnot so accurate for practical use,\u201d he said.\n <\/p>\n

\n Also, the videos used in the study included typical scenes of, for example, a dog biting a man, but not more unusual scenes \u2014 say, a man biting a dog. Therefore, it is not yet clear whether the technique could be used to capture less predictable mental images.\n <\/p>\n

\n As a result, \u201cwhile some people may worry that this technology poses serious risk to mental privacy,\u201d in reality, \u201cthe current approach cannot easily read a person\u2019s private thoughts,\u201d Horikawa said.\n <\/p>\n

\n Sign up for CNN\u2019s Wonder Theory science newsletter<\/a>. Explore the universe with news on fascinating discoveries, scientific advancements and more.<\/p>\n","protected":false},"excerpt":{"rendered":"A scientist in Japan has developed a technique that uses brain scans and artificial intelligence to turn a…\n","protected":false},"author":2,"featured_media":292726,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[45],"tags":[182,181,507,74],"class_list":{"0":"post-292725","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-artificial-intelligence","8":"tag-ai","9":"tag-artificial-intelligence","10":"tag-artificialintelligence","11":"tag-technology"},"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/posts\/292725","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/comments?post=292725"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/posts\/292725\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/media\/292726"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/media?parent=292725"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/categories?post=292725"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/tags?post=292725"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}