\nAI scribes from companies such as Heidi and Lyrebird take notes for practitioners during medical appointments. Image: YouTube \/ Heidi Health<\/p>\n
When Australian artificial intelligence expert Dr Kobi Leins declined a medical specialist\u2019s request to use AI transcription software during her child\u2019s upcoming appointment, the practice told her the practitioner required the technology be used and Leins was “welcome to seek an alternative” provider.<\/p>\n
The specialist\u2019s \u201chigh workload and limited time\u201d meant they were now \u201cunable to perform timely assessments without the use of AI transcribing tools\u201d, the practice said in an email seen by Information Age.<\/p>\n
\u201cThey gave me the choice of proceeding with the AI that they were insisting on, or to find another practitioner,\u201d Leins said.<\/p>\n
The appointment was cancelled, and her deposit refunded.<\/p>\n
In Australia, healthcare providers can decide to not consult with a patient for any reason \u2014 except in emergency situations \u2014 provided they facilitate the patient\u2019s ongoing care in some way.<\/p>\n
The AI system used by the practice Leins had booked was an Australian platform which transcribed sessions to maintain the specialist\u2019s notes about patients and their health, the practice told her.<\/p>\n
It was a system whose privacy and security capabilities Leins had previously reviewed as part of her work in AI governance \u2014 and one she said she would not want her child\u2019s data \u201canywhere near\u201d.<\/p>\n
The rise of AI scribes in the doctor\u2019s office<\/p>\n
AI scribes are becoming more common in Australian medical practices, with around 20 per cent of GPs \u2014 but fewer specialists \u2014 using the technology according to recent industry polls.<\/p>\n
The software is typically marketed around saving practitioners time, improving their engagement with patients, and reducing burnout.<\/p>\n
The generative AI tech can write, organise, and analyse notes about health concerns, and some can even propose follow-up tasks for practitioners, complete administration such as drafting letters and forms, and make follow-up phone calls to patients.<\/p>\n
But unlike most medical devices, AI scribes remain largely unregulated, leaving it up to individual healthcare practices to decide which tools they want to use, and how.<\/p>\n
AI scribes have already presented risks for both patients and doctors in areas such as accuracy and data retention, according to University of Queensland associate professor of information systems, Dr Saeed Akhlaghpour.<\/p>\n
Some clinicians have reported AI scribes making incorrect diagnoses, he said.<\/p>\n
Allowing the software to receive new features “without adequate review or re-consent” from users was also a risk, Akhlaghpour said \u2014 as was vendors potentially processing sensitive data outside of Australia.<\/p>\n
\u201cIf a vendor processes data offshore, patient information may be subject to weaker protections than those under the Australian Privacy Principles,\u201d he said.<\/p>\n
\nAI scribes from companies such as Heidi and Lyrebird take notes for practitioners during medical appointments. Image: YouTube \/ Heidi Health<\/p>\n
Leins, who previously held senior AI and data ethics roles at health insurance company Bupa and the National Australia Bank (NAB), said she had reviewed over 200 AI tools during her career.<\/p>\n
While she was \u201cdisappointed\u201d by the response of the medical specialist she had booked with, Leins said she was more surprised to hear from the Australian Health Practitioner Regulation Agency (AHPRA) that AI scribes were not regulated by the Therapeutic Goods Administration (TGA).<\/p>\n
\u201cThe question is, who has the onus to prove that they\u2019re good enough, or that they\u2019re fit for purpose and legally compliant?\u201d she said.<\/p>\n
Medical practices were likely being approached by AI companies and convinced to purchase their \u201cmagical solution\u201d, Leins suggested, despite often not being qualified or able to review the tools\u2019 privacy and security standards for themselves.<\/p>\n
AI scribes could be reviewed and regulated nationally, she argued, so there was less burden on individual practitioners and fewer risks for patients.<\/p>\n
Should AI scribes be regulated in healthcare?<\/p>\n
A spokesperson for AHPRA did not take a position on whether AI scribes in healthcare should be regulated by the TGA, when contacted by Information Age.<\/p>\n
They acknowledged new AI tools had presented \u201csome new practice management questions for practitioners\u201d and said regulators were working with other agencies to develop further guidance for medical professionals.<\/p>\n
\u201cAHPRA recognises that helping practitioners talk with patients about AI can build public trust and support its safe use in healthcare,\u201d they said.<\/p>\n
The Royal Australian College of General Practitioners (RACGP) also did not have a position on whether AI scribes should be regulated by the TGA, but has previously stated AI systems \u201cmust be evaluated over time \u2026 to ensure they are still fit for purpose\u201d.<\/p>\n
\u201cThere may be a need to establish a new Australian regulatory body to fill gaps in existing government portfolios and legislative instruments,\u201d the organisation stated in its position on AI in primary care<\/a>.<\/p>\n Associate professor Akhlaghpour said regulating AI scribes in healthcare with \u201ctargeted, risk-based national oversight\u201d would help prevent potential issues around privacy, security, and liability.<\/p>\n \u201cWhile AI scribes claim they are not medical devices, some already function in ways that meet the definition, such as proposing diagnoses or treatments, which could require pre-market approval under the Therapeutic Goods Act,\u201d he said.<\/p>\n
\nAustralia’s medical regulator AHPRA says AI scribes raise ‘some new practice management questions for practitioners’. Image: Shutterstock<\/p>\n