Researchers at the University of Texas at Dallas have built a new augmented reality interface that lets users type on everyday objects. The technology, called PropType, overlays a virtual keyboard onto items such as bottles, cans, and books.
The system aims to fix a persistent AR issue by addressing slow typing and arm fatigue caused by floating interfaces.
PropType uses a handheld object as the base for a keyboard, giving users real tactile feedback. “By integrating objects already present in the user’s surroundings,
PropType reimagines text input in AR, creating a seamless connection between the physical and virtual worlds,” said Dr. Jin Ryong Kim, assistant professor of computer science.
“PropType capitalizes on the tactile feedback provided by the objects themselves, offering better key confirmation and reducing reliance on visual cues.”
The research team presented the system earlier this year at major human-computer interaction conferences in Japan and South Korea.
Turning props into keyboards
Virtual keyboards often frustrate users because they demand mid-air typing and constant visual focus. PropType tries to fix that by shifting input back to the physical world. It places a keyboard layout on a real object and adapts to its shape.
The system works on curved or irregular surfaces, including water bottles, soda cans, coffee cups, and thick books.
The team studied how sixteen people held everyday objects. They examined grip strength, hand positioning, and finger movement. These insights helped them build custom keyboard layouts that felt natural on different items.
Users can also design their own key layouts. The editing tool allows changes in size, spacing, and visual effects.
Researchers say this helps PropType work in many settings where external keyboards are not practical.
“PropType leverages the familiarity of handheld objects to offer a more intuitive and accessible alternative to traditional keyboards, particularly in mobile or hands-free scenarios where conventional input methods are impractical,” Kim said.
Awards for haptics work
PropType extends Kim’s broader research on haptics, which explores touch, vibration, and thermal signals in digital systems. “Touch carries a lot of information; it’s another form of communication that is underexplored in virtual and augmented reality,” he said.
Kim’s lab recently earned recognition for work on thermal masking and thermal-tactile integration. These experiments study how the brain interprets heat and vibration.
For example, if a heat actuator sits on the forearm and a vibration actuator sits a few inches away, the combined signals can trick the user into sensing heat at both spots.
The team presented a paper on this at the 2024 ACM CHI Conference, following demonstrations at major IEEE haptics events.
Researchers say this line of study could change virtual reality and medical simulation tools by creating sensations that feel more real.
Kim said the goal is to build interfaces that respond with precision and create stronger immersion.