The University of Texas System’s administration building in downtown Austin.
Ken Herman
The University of Texas System is looking to fast-track artificial intelligence use in healthcare through a major initiative to manage and develop AI tools at health institutions across its network.
The UT REAL Health AI initiative aims to safely harness the technology’s benefits at its medical schools and hospitals, including MD Anderson Cancer Center and The University of Texas Medical Branch. The effort aligns with a broader push among nonprofits and governmental agencies to strengthen oversight of technology that doesn’t always work as intended and comes with data privacy risks.
Experts say only a few university health systems have launched such comprehensive plans around AI, with the potential to impact millions of patients, students and faculty.
Article continues below this ad
“There are very few institutions in the world that are doing that,” said Brian Anderson, CEO of the Coalition for Health AI, a nonprofit that develops best practices for responsible AI use in healthcare. “The University of Texas is one of them.”
HEALTH AI: Houston hospitals are seizing on the AI revolution. Is the tech actually helping patients?
UT hospitals already use tools that detect cancer earlier, streamline documentation, flag patients at risk of sepsis and predict hospital readmissions. Under UT REAL, the university system says it will better understand what AI tools its health institutions are using. Security risks, and the intended purpose of the tools, will receive greater scrutiny. The technology will be continuously monitored for evidence of benefits or unexpected harms, and promising AI projects will receive funding.
When fully operational, the system’s efforts to manage the tools will make UT “one of the most transparent and accountable health systems in the country with respect to AI,” according to internal documents obtained by the Houston Chronicle.
Article continues below this ad
However, the initiative is taking shape at a sensitive time in the broader AI governance landscape. Leaders of the world’s largest AI companies, OpenAI and Anthropic, have raised red flags about the technology’s development as their list of hacking incidents grows. In several newly disclosed cases, OpenAI agents attempted to hack into a university digital library and Australian government websites, successfully accessing public and non-public health data, according to news reports.
Publicly available details about the $25 million state-funded UT REAL program remain sparse, leaving one AI expert with questions about whether personal health information will be collected as part of the initiative, and how problems with AI’s performance would be handled, among other issues.
JOB INSECURITY: 1 in 6 Houstonians could see their jobs change because of AI. Are you one of them?
While there are clear benefits of a program like UT REAL, it can also create risks that need to be properly managed, said Numa Dhamani, a machine learning expert who has taught AI courses at UT. For example, Dhamani asks, do vendors have access to large amounts of patient health information? How is that information protected? If one hospital detects a problem with an AI tool used at all UT System hospitals, does that mean all hospitals need to stop using the tool?
“You can share lessons across institutions. You’re able to detect problems that might be invisible at a single hospital. So I can understand why you want to centralize this,” she said. “But … you can have mistakes and weaknesses propagate across the system rather than staying local.”
Article continues below this ad
The initiative’s leaders either could not be reached or declined to answer questions about how this program would achieve its stated aims.
The initiative’s executive director, Nyma Shah of MD Anderson, and its chair, Dr. Peter McCaffrey of UTMB, were not made available for interviews by their respective institutions. Zain Kazmi, the UT System’s chief digital and analytics officer in health affairs, and UT REAL’s executive sponsor, did not respond to emails requesting an interview. Other UT System officials associated with the initiative did not respond to requests for comment.
Expanding AI projects
One of the potential benefits of UT REAL is the ability to expand AI projects that save time — projects like those of Andrew Jamieson, an assistant professor of bioinformatics at UT Southwestern.
Jamieson and his team developed a novel AI system to help grade standardized medical student exams.
Article continues below this ad
Years ago, he and others recognized that the exam, known as an Objective Structured Clinical Examination (OSCE), produced large amounts of audio, video and text files that were time-consuming to grade. The exam involves recorded interactions between students and patient actors at different stations, which simulate different clinical scenarios.
It was the perfect opportunity to use a large language model, he said.
Jamieson’s team started by using GPT-4 to help score the written transcripts from the exam, based on a faculty-approved grading rubric. As the project evolved, the team began using Google Gemini and incorporated the audio and video recordings into the analysis. They also developed checks on the system: AI scores don’t independently determine whether a student passes or fails — the faculty does. Disagreements between human graders and the AI are automatically sent for physician review. Scores below a certain threshold also receive additional human reviews.
Early versions of the technology saved 980 hours of grading time, an 87.5% reduction, Jamieson’s research showed. “Much more time is saved now that we grade the entire 15-min encounter audio/video,” he said in an email.
Jamieson said his team runs the program through a secure cloud-based platform, Microsoft Azure, so that student data is kept private and not used for training. Microsoft is one of several vendors that partner with UT Southwestern. The school has been working closely with the company to use AI on local devices, without using cloud-based software, to give universities greater privacy and control over the data, said Jamieson.
Article continues below this ad
Not every school in the UT System has the infrastructure to safely test and deploy AI technology, so UT REAL awarded a $300,000 grant to expand Jamieson’s work to five other medical schools, including UT Health Houston.
“Especially with these state-run institutions, they’re like aircraft carriers. They just move so slow,” Jamieson said. “To be able to do this, we need to provide safe avenues for folks to do this.”
Other AI projects are moving through the UT REAL pipeline. Earlier this year, UTMB said it became the first academic health system to launch a HIPAA-compliant web search tool, giving staff access to a more secure alternative to commercial products like ChatGPT. The tool is powered through Qualified Health, an enterprise AI platform, and Anthropic.
According to an undated training slideshow obtained by the Chronicle, a Qualified Health chat tool has already been deployed at UTMB, UTSA and UTHealth Houston. The technology can condense documents, draft emails and “act as a thought partner for complex cases, projects and brainstorming sessions,” the slideshow says.
NEW CHALLENGES: Texas Medical Center wrestles with promise, perils of AI
Dhamani said the chat tool is a good example of how a program like UT REAL can improve data security, especially if staff members were entering sensitive information into publicly available search tools. But there are still questions about how institutions monitor the technology.
“I would want to know things like: Does it retrieve information from authoritative and current sources? … How are incorrect and misleading answers reported? Are you logging it?” she said. “What HIPAA compliance tells you is how patient information is handled. It doesn’t really tell you if your clinical answer is correct.”
Creating an inventory of AI tools
UT System health institutions are using AI in hundreds of ways, according to a UT REAL slideshow obtained through an open records request.
Among eight medical schools and hospitals, there are 571 AI “use cases,” or real-world scenarios in which the technology has been applied, according to the presentation. Generative AI, or systems that rely on machine learning to create new content, account for 12% of those cases, according to the slideshow. Another 150 AI applications are in production.
Knowing the full inventory of AI tools across the UT System, and whether each tool is successful, is one of the primary goals of UT REAL.
Research shows there are a number of safety concerns associated with the technology. Machine learning models can be prone to gender or racial bias. They can make mistakes in patient diagnoses.
Under UT’s initiative, “lessons learned at MD Anderson will automatically benefit UT Health San Antonio,” according to internal UT REAL documents. “A safety signal detected at one hospital will trigger a system-wide review. Every AI tool will have a record as complete as a patient’s medical chart.”
UT REAL leaders were not available to answer questions about who will have access to AI records and whether individual patient data will be collected for oversight efforts. UT plans to partner with a vendor to help with AI management, internal records show, but it’s not clear whether an agreement has been reached.
Hospitals can limit vendor access to patient data that AI systems analyze or collect. That may involve removing identities from the information or preventing the company from using it for training.
But “just because data isn’t used for model training doesn’t mean that data isn’t leaving the institution and that no data is retained,” said Dhamani.
She said it would be important to know what health information might be leaving UT’s controlled environment, and what information, if any, is retained by an outside company. A centralized platform that contains patient health information could become an attractive target for cyberattacks, she said. In general, the risk “just depends on if they’re actually ingesting any sort of underlying clinical data,” she said.
UT System officials have recently scrutinized the data-use practices of AI vendor partners, according to emails obtained through an open records request.
In one May exchange, Jennie Kennedy, UT System’s privacy and data protection officer, asked an executive of a company that makes an AI-powered search engine about how they share information with advertisers. In a lengthy response, the executive outlined the company’s policies and said there was no cause for concern. Neither could be reached for comment.
Health systems are having “challenging” conversations with outside vendors about who should be held responsible in the event of a data leak or an error, said Anderson of the Coalition for Health AI, who helped lead data analysis for the White House COVID Task Force and Operation Warp Speed.
The recent incidents involving rogue AI tools only emphasize the need for a more robust system of oversight, like what UT REAL proposes, he said.
“Candidly, so much of what they’re doing in the AI space is fundamental to what CHAI is — sharing and learning from each other,” he said. “So when we bring our health systems together, The University of Texas is one of the biggest contributors to developing that shared understanding of what responsible, trustworthy AI looks like.”