Most AI note-takers approved for medical workers by the Ontario government had errors in their testing, the province\u2019s auditor general found in a report released Tuesday.<\/p>\n
Supply Ontario had the bots transcribe two conversations between health-care workers and patients. Most of the vendors that were approved had inaccuracies in their results, including \u201cincorrect information, AI hallucinations and incomplete information,\u201d Auditor General Shelley Spence\u2019s report notes.<\/p>\n
Sixty per cent of approved AI scribes recorded a different drug than what was prescribed, Spence said.<\/p>\n
Seventeen of the 20 approved scribes \u201cmissed key details about the patients\u2019 mental health issues in at least one of the two tests,\u201d Spence wrote.<\/p>\n
And nine of the 20 \u201cfabricated information and made suggestions to patients\u2019 treatment plans, such as referring the patient for therapy or ordering blood tests, even though these steps were not mentioned in the simulated recordings,\u201d the auditor wrote.<\/p>\n
Scribes also hallucinated scenarios about patients\u2019 health, stating that \u201cthere were \u2018no masses found\u2019 or that there was presence of anxiety in the patient, although this information was not discussed in the recordings,\u201d she wrote.<\/p>\n
The province did not put much weight on accuracy in its testing. \u201cAccuracy of medical notes generated\u201d accounted for four per cent of points awarded, while \u201cdomestic presence in Ontario\u201d was weighted the highest at 30 per cent, the auditor found.<\/p>\n
\u201cData privacy\/legal controls\u201d were weighted at 23 per cent and \u201csystem security controls\u201d were at 11 per cent.<\/p>\n
![\"ag-may-12-2026\"](\"https:\/\/www.newsbeep.com\/ca\/wp-content\/uploads\/2026\/05\/ag-may-12-2026.jpeg;w=960.jpeg\"\/)
\nAuditor General of Ontario
<\/p>\n
![\"ag-may-12-2026-2\"](\"https:\/\/www.newsbeep.com\/ca\/wp-content\/uploads\/2026\/05\/ag-may-12-2026-2.jpeg;w=960.jpeg\"\/)
\nAuditor General of Ontario
<\/p>\n
Bidders could have scored zero on system security, bias controls and medical note accuracy, and still meet the minimum score to be approved as a vendor of record, Spence said.<\/p>\n
The tests also did not have to be done live, in front of the evaluators. They were given recordings and allowed to run the system offline, then send the results to Supply Ontario, Ontario Health and OntarioMD \u2014 allowing \u201cvendors to potentially overstate their compliance with security and privacy requirements,\u201d the auditor said.<\/p>\n
\u201cWhen Ontarians see their doctor, they need to share intimate information about their health, their bodies and their personal lives to receive proper care,\u201d Spence wrote in her report. \u201cOntarians expect this extremely personal information to be kept private and confidential. Using AI to assist in providing health care must not come at the cost of compromising privacy.\u201d<\/p>\n