WASHINGTON \u2014 The American Medical Association (AMA) today urged federal lawmakers to put stronger safeguards in place as the use of augmented intelligence (AI) chatbots in mental health care continues to grow. The AMA emphasized both the promise and risks of these rapidly expanding technologies.<\/p>\n
In letters to the co-chairs of the Congressional Artificial Intelligence Caucus<\/a> (PDF), the Congressional Digital Health Caucus<\/a> (PDF), and the Senate Artificial Intelligence Caucus<\/a> (PDF), the AMA recognized Congress for advancing conversations about AI\u2019s role in society and mental health. At the same time, the AMA warned that the rapid rise of mental health chatbots\u2014along with reports of risks such as encouraging self-harm and privacy breaches\u2014highlights the urgent need for clear guardrails to protect patients and maintain public trust.<\/p>\n \u201cAI-enabled tools may help expand access to mental health resources and support innovation in health care delivery, but they lack consistent safeguards against serious risks, including emotional dependency, misinformation, and inadequate crisis response,\u201d according to AMA CEO John Whyte, MD, MPH. \u201cWith thoughtful oversight and accountability, policymakers can support innovation and ensure technologies prioritize patient safety, strengthen public trust, and responsibly complement\u2014not replace\u2014clinical care.\u201d<\/p>\n While AI technologies present meaningful opportunities to improve access to care and support innovation in health care delivery, recent congressional hearings made clear that immediate attention is required to ensure these tools do not inadvertently harm individuals seeking mental health support.<\/p>\n To address these concerns, the AMA urged Congress to require stronger safeguards\u2014recognizing they are a starting point, not a limit, and that modernized protections may be needed as these tools evolve. The safeguards outlined in the AMA letter include:<\/p>\n Enhance Transparency:\u00a0Require chatbots to clearly and meaningfully disclose that users are interacting with AI, not a human being, and prohibit systems from presenting themselves as licensed clinicians. Congress should also empower federal regulators to enforce transparency standards and penalize deceptive practices.<\/p>\n Establish Clear Regulatory Boundaries: Prohibit chatbots from diagnosing or treating mental health conditions without appropriate regulatory review. Congress should direct agencies to create a modern, risk-based oversight framework and clarify when AI tools qualify as medical devices. Developers should also be required to implement crisis-detection systems that identify self-harm risk and provide immediate referrals to appropriate resources, along with appropriate de-escalation language to help mitigate potential harms.<\/p>\n Strengthen Oversight and Accountability: Mandate ongoing safety monitoring, reporting of adverse events, and rigorous standards for tools used by children and adolescents, who may face heightened risks.<\/p>\n Limit Commercial Influence: Discourage or prohibit advertising within mental health chatbots, particularly for minors, and ensure that outputs remain free from sponsorship bias or commercial influence.<\/p>\n Protect Privacy and Security: Require strict data protection standards, including limits on data collection and retention, clear user consent for data use, and safeguards against unauthorized access or sharing of sensitive information.<\/p>\n The AMA emphasized that meaningful safeguards are essential to ensure AI tools complement\u2014not replace\u2014clinical care while safeguarding patients from harm. The organization reaffirmed its commitment to working with Congress to advance policies that promote innovation while prioritizing patient safety, clinical integrity, and public trust.<\/p>\n","protected":false},"excerpt":{"rendered":"WASHINGTON \u2014 The American Medical Association (AMA) today urged federal lawmakers to put stronger safeguards in place as…\n","protected":false},"author":2,"featured_media":608654,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[59],"tags":[97,252,253],"class_list":["post-608653","post","type-post","status-publish","format-standard","has-post-thumbnail","category-health-care","tag-health","tag-health-care","tag-healthcare"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/posts\/608653","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/comments?post=608653"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/posts\/608653\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/media\/608654"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/media?parent=608653"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/categories?post=608653"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/tags?post=608653"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}