A 60-year-old man with no history of mental illness presented at a hospital emergency department insisting that his neighbour was poisoning him. Over the next 24 hours he had worsening hallucinations, and tried to escape the hospital.<\/p>\n
Doctors eventually discovered the man was on a daily diet of sodium bromide, an inorganic salt mainly used for industrial and laboratory purposes including cleaning and water treatment.<\/p>\n
He bought it over the internet after ChatGPT told him he could use it in place of table salt<\/a> because he was worried about the health impacts of salt in his diet. Sodium bromide can accumulate in the body causing a condition called bromism, with symptoms including hallucinations, stupor and impaired coordination.<\/p>\n It is cases like this that have Alex Ruani, a doctoral researcher in health misinformation with the University College in London, concerned about the launch of ChatGPT<\/a> Health in Australia.<\/p>\n A limited number of Australian users can already access the artificial intelligence platform which allows them to \u201csecurely connect medical records and wellness apps\u201d to generate responses \u201cmore relevant and useful to you\u201d. ChatGPT users in Australia can join a waitlist for access.<\/p>\n \u201cChatGPT Health<\/a> is being presented as an interface that can help people make sense of health information and test results or receive diet advice, while not replacing a clinician,\u201d Ruani said.<\/p>\n \u201cThe challenge is that, for many users, it\u2019s not obvious where general information ends and medical advice begins, especially when the responses sound confident and personalised, even if they mislead.\u201d<\/p>\n Ruani said there had been too many \u201chorrifying\u201d examples<\/a> of ChatGPT \u201cleaving out key safety details like side effects, contraindications, allergy warnings, or risks around supplements, foods, diets, or certain practices\u201d.<\/p>\n \u201cWhat worries me is that there are no published studies specifically testing the safety of ChatGPT Health,\u201d Ruani said. \u201cWhich user prompts, integration paths, or data sources could lead to misguidance or harmful misinformation?\u201d<\/p>\n Sign up: AU Breaking News email<\/a><\/p>\n ChatGPT is developed by OpenAI, which used the tool HealthBench to develop ChatGPT Health. HealthBench employs doctors to test and evaluate how well AI models perform when responding to health questions.<\/p>\n Ruani said the full methodology used by HealthBench, and its evaluations, are \u201cmostly undisclosed, rather than outlined in independent peer-reviewed studies\u201d.<\/p>\n \u201cChatGPT Health is not regulated as a medical device or diagnostic tool. So there are no mandatory safety controls, no risk reporting, no post-market surveillance, and no requirement to publish testing data.\u201d<\/p>\n An OpenAI spokesperson told Guardian Australia that the company had worked in partnership with more than 200 physicians from 60 countries \u201cto advise and improve the models powering ChatGPT Health\u201d.<\/p>\n \u201cChatGPT Health is a dedicated space where health conversations stay separate from the rest of your chats, with strong privacy protections by default,\u201d the spokesperson said.<\/p>\n ChatGPT Health data is encrypted and subject to privacy protections by default.<\/p>\n Sharing with third parties will happen with user consent, or in limited circumstances outlined in OpenAI\u2019s privacy policy<\/a>.<\/p>\n