Hannah Cliburn, psychology graduate student, left, and David Gerkens, professor of psychology
What happens when artificial intelligence tries to interpret human emotion? For Cal State Fullerton psychology graduate student Hannah Cliburn, that question became the focus of a mentorship-driven research project that explores how AI evaluates emotional distress and where it falls short.
With a background in positive psychology, Cliburn had not planned to include AI in her research. But when she joined psychology professor David Gerkens’ lab, she was invited to explore a new project examining how large language models rate emotional content in autobiographical memories.
“I wasn’t very familiar with AI research, and it wasn’t on my radar,” Cliburn said. “But I decided to dive in, and it turned out to be a fascinating and timely area that I could not have anticipated.”
The project focused on comparing human ratings of emotional distress with those generated by the AI.
As Cliburn explained, “We were looking at whether the AI would rate these memories at similar levels of distress as an average person would. Could it sense the emotionality from the written memories, like a human would?”
Each of the 10 autobiographical memory statements used in the study was classified as moderately distressing. Human participants then rated each memory on a scale from zero to 100. The model was also asked to rate the same memories. The researchers treated the AI tool like a human participant, feeding the memories into the models at different times of day to capture potential variability in responses.
The results were surprising.
“What we found was that the AI’s emotional distress ratings were all significantly higher than what humans would rate them,” Cliburn said. “At first, it was shocking. I assumed AI would be logical and unemotional — maybe even underestimate emotional distress — but we found the opposite. The AI tools we were using consistently rated memories as more emotionally distressing than humans did.”
The experiment also revealed AI’s inconsistencies. When Gerkens asked the LLM tools to explain the criteria behind their ratings and then rate another memory using the same standards, the model offered slightly different criteria each time.
“When you ask it to evaluate autobiographical memories, it is very unclear what criteria it is using. That uncertainty is why research like this matters,” he said. “One of the funny things was the AI telling me how you can’t always use the same criteria because the way people experience emotion is personal and subjective. A language model explaining human emotion? Sorry, that is crazy.”
Cliburn also reflected on the potential implications. “If AI is overestimating distress, it’s a little scary to think about someone in need of a mental health professional turning to a chatbot instead,” Cliburn said. “I’m hesitant to embrace AI in academic and mental health settings. I think it could be used well, but will we use it well?”
Mentorship and the Research Journey
Cliburn said the project sparked a deeper interest in exploring how humans and AI perceive emotion. She credited the guidance she received from Gerkens with helping her develop the skills to investigate these questions rigorously.
“Being able to work directly with someone who has published research and has a broad perspective on the field has been invaluable,” she said. “I learned how to conduct psychological research on a subject that impacts my own focus in positive psychology, which is helping me shape my thesis and future career.”
They were able to conduct this research thanks to support from the Engaging Graduate Students in Research, Scholarly and Creative Activities program, which provides graduate students and faculty mentors with stipends to work together on hands-on research projects. The program is administered by the Undergraduate Research Opportunity Center within the Office of Research and Sponsored Programs.
“EG-RSCA allowed us to commit fully to this research,” Cliburn said. “We felt motivated to do the best work we could in the time we had.”
Gerkens likewise emphasized the value of sustained mentorship for the project.
“I run a large lab with many projects happening at once,” he said. “This approach was different. It allowed me to work closely with a student on a single project from start to finish, and I can see how impactful that was.”
Looking ahead, Cliburn hopes to continue research on AI and mental health. “It is part of our responsibility as researchers to understand how these tools will impact our field,” she said.
She also stressed the importance of human judgment. “Critical thinking is humanity’s biggest strength,” she said. “AI can be helpful, but it is not the same as talking to a human, and it shouldn’t replace that.”
For more information on EG-RSCA, or to find other faculty-student mentorship opportunities, please visit the UROC website.