“I also hope the future of accessible technology includes affordability and distribution,” Marshall says. “We’ve watched emerging technologies like smart glasses evolve primarily as consumer products, while many of their most meaningful possibilities are accessibility-related.” Her conclusion is the one product teams, funders and policymakers should sit with. “Innovation only goes so far if the people who could benefit most can’t afford or access it.”
Justin Kaneps, photographer for the inaugural All in Frame Collection
Last week, ahead of its Made by Google event, Google DeepMind announced SL2T, an AI model that translates sign language directly into written text. It powers a new sign-to-text feature in Gboard and Live Transcribe across the Pixel 11 lineup. American Sign Language users can now sign into their phone’s camera anywhere they would normally type: a search bar, a message, a document, a prompt to Gemini. In Live Transcribe, a Deaf user can sign a reply during a face-to-face conversation instead of thumbing out a response while the other person waits.
For anyone who has spent a career watching accessibility technology get announced, underdelivered and sometimes shelved, this one deserves a close look. It is great work, and perhaps even revolutionary.
Translating ASL is a fundamentally different problem than transcribing speech. ASL is a full language with its own grammar, and meaning is carried simultaneously through the hands, arms, torso, head and face. There is no clean word-for-word mapping to English.
Previous approaches tried to solve this by first converting signs into written labels called glosses, then translating the glosses. Google’s team abandoned that step, arguing that glosses flatten exactly the non-manual markers and spatial constructions that carry meaning. Instead, an on-device computer vision system tracks 130 points across the signer’s face, body and hands, converting the video into a moving map of coordinates. The raw footage is discarded on the device. Only the coordinates travel to Google’s servers for translation.
The model was trained on more than 100,000 hours of data spanning over 50 sign languages, with roughly a quarter of that in ASL. It was deliberately trained to handle one-handed signing, which matters when you are holding the phone with your other hand, and to handle left-handed signers.
The concept came from Sam Sepah, a Deaf Googler. Google convened an AI Sign Language Advisory Committee of Deaf organizations and subject-matter experts, and ran evaluation through Deaf user studies.
To understand why the technical choice to track the face and torso matters, and why the governance around this launch matters more, you have to know what has been done to this language before.
ASL was born in a school. In 1817, Thomas Hopkins Gallaudet opened the American School for the Deaf in Hartford, Connecticut, alongside Laurent Clerc, a Deaf French teacher he had recruited in Paris. What emerged there was a blend of French Sign Language, the home signs students brought from their own families, and the signing already in use in communities like Martha’s Vineyard, where hereditary deafness was common enough that hearing and Deaf residents alike signed as a matter of course. Within a generation, that blend had become a distinct language with its own grammar, transmitted through Deaf schools, Deaf clubs and Deaf families across the country.
Then, in September 1880, a room in Milan voted it out of existence.
The Second International Congress on the Education of the Deaf was organized by the Pereire Society, an oralist body, and the guest list was assembled to produce the outcome it wanted. Of the twelve speakers, nine argued for oral-only education. The Congress resolved that the oral method should be preferred and that sign language should be removed from Deaf education. Of 164 delegates, only the five Americans and a single English delegate voted against. There were, by most accounts, one or two Deaf people in the room.
The consequences had ripple effects and in the United States, the share of Deaf students taught through oral methods rose from 32 percent in 1887 to 70 percent by 1917. Deaf teachers were pushed out of the profession and replaced with hearing speech instructors. Children were punished for signing. An entire pedagogy was built on the premise that a Deaf child’s native language was an obstacle to be removed.
ASL survived anyway, and it survived in exactly the places institutions were not looking. It passed between children in dormitories after lights went out. It was kept alive in Deaf clubs, at Deaf sporting events, around kitchen tables in Deaf families. It was transmitted peer to peer for eighty years while every official structure insisted it was not a language at all.
The vindication came in 1960, when William Stokoe, a hearing linguist at Gallaudet, published Sign Language Structure and demonstrated that ASL had phonology, morphology and syntax like any other natural language. That paper is the reason it is now obvious that ASL is not English performed with the hands. It was not obvious then, and Stokoe was widely mocked by his own colleagues for saying so.
What followed was slower than it should have been. Gallaudet, chartered by Abraham Lincoln in 1864, did not have a Deaf president until students shut the campus down in the Deaf President Now protests of March 1988. The Americans with Disabilities Act arrived in 1990. And the International Congress on the Education of the Deaf did not formally repudiate the Milan resolutions until it met in Vancouver in 2010, one hundred and thirty years after the vote.
Technology has been part of this story throughout, mostly as a series of well-meaning misunderstandings. The most instructive is the sign language glove, an idea that resurfaces every few years in university engineering departments and crowdfunding campaigns, usually accompanied by a press release about breaking down barriers. Deaf linguists have explained each time why it cannot work: gloves capture the hands and nothing else, and ASL is built out of facial expression, head position, eye gaze, body shift and the use of space. A glove that reads only the hands is reading roughly a third of the language and confidently reporting the result.
Which brings the history directly to the point. Google’s decision to track 130 landmarks across the face, torso and hands is not an incremental engineering upgrade. It is the first time a major consumer product’s architecture reflects what Deaf people have been saying about their own language since 1960.
That recognition matters most to the people who will actually use it. “This is groundbreaking technology, and as a Deaf ASL user, I’m excited to see ASL recognized computationally as the complex language it is: from handshapes and classifiers to spatial grammar, facial expression, and non-manual markers,” says Chrissy Marshall, a Deaf writer and director. “But I can’t help thinking about how this technology fits into our existing accessibility landscape.”
That landscape has always run on Deaf labor. “Deaf people already carry so much of the communication burden: watching captions, filling in what was missed, correcting errors, advocating for accommodations, all while trying to participate in the conversation,” Marshall says. “As a society, I think we are far too comfortable expecting Deaf people to rely on imperfect communication tools. Especially when they are ‘futuristic’ or ‘cool.’”
Google and its advisory committee describe SL2T as strictly an assistive input tool for low-stakes communication. The company states plainly that it does not fulfil legal requirements for reasonable accommodations. It is not designed for medical appointments, legal proceedings, classrooms, job interviews or government hearings. The report explicitly warns institutions against using the software to replace qualified human interpreters or to sidestep their accessibility obligations.
The model can struggle with regional signs, slang, complex ASL grammar, rapid fingerspelling and meaning conveyed primarily through facial expression. It can produce what Google calls ghost text, words the user never signed, when someone pauses or a second person walks into frame. Accuracy degrades in low light, when a signer is partially out of frame, or when there is insufficient contrast between hands, face, clothing and background.
Those limits leave open a question the launch materials do not answer. Whose ASL is the system learning to read?
“ASL isn’t monolithic,” Marshall says. “There are regional signs, Black ASL and other dialects, generational differences, slang, signing styles, and differences in mobility all matter. Facial expression also carries grammatical meaning, raising questions about how the technology recognizes people with facial differences or disabilities affecting movement. Will I have to take off my glasses? Will POC be able to communicate and feel understood equitably?”
Human interpreting involves the same negotiation, and Marshall points out that it is never fully settled there either. “Even with interpreters who have decades of experience, there can be linguistic and generational nuances we have to navigate together,” she says. “In the entertainment field I work in, we adapt and create signs for efficacy and flow of work.”
Which leads to the question that should sit at the center of every evaluation of this technology. “Will an AI model understand the way I sign,” Marshall asks, “or will I have to adapt my language to be understood by the technology?”
The answer determines whether the product is translating ASL or asking ASL to become more legible to a machine. A system that performs best when the signer standardizes toward it has moved the work back onto the Deaf user, which is where it has sat since 1817.
There is a practical version of the same concern. “The current need for specific lighting, framing, and phone positioning also makes me question how practical it will be for spontaneous interactions,” Marshall says. “And in formal settings, I would still prefer a qualified ASL interpreter.”
Every accessibility technology that arrives with a caveat eventually meets an institution that ignores the caveat. Automatic captions were released with clear guidance that they were not a substitute for CART or professional captioning. Within a few years, universities and employers were treating auto-captions as compliance, and Deaf students and employees spent years litigating and negotiating their way back to what they were entitled to in the first place.
Marshall has lived both sides of that history. “I know imperfect technology can still be incredibly useful because I don’t have an interpreter with me every day,” she says. “Live transcription and captions, for example, can be used spontaneously in everyday life, and emerging caption glasses can help me maintain eye contact instead of constantly looking down at a phone.” She has also watched the substitution happen to her. “I’ve already experienced automatic captions treated as ‘good enough’ in academic and professional settings where better access exists.”
Then there is the variable accessibility technology gets wrong most reliably, which is who can actually get their hands on it. SL2T is tied to the Pixel 11 lineup, so access to the most sophisticated sign language model ever shipped is gated behind the purchase of a particular phone.
“I also hope the future of accessible technology includes affordability and distribution,” Marshall says. “We’ve watched emerging technologies like smart glasses evolve primarily as consumer products, while many of their most meaningful possibilities are accessibility-related.” Her conclusion is the one product teams, funders and policymakers should sit with. “Innovation only goes so far if the people who could benefit most can’t afford or access it.”
This is the most significant sign language technology to reach a consumer product, built with Deaf people rather than around them, and Google was straight about what it cannot do. That combination is rarer than it should be.
Marshall puts the standard where it belongs. “The technology is remarkable. Its greatest value should be giving Deaf people more tools to communicate, and I’m always open to that. Innovation should expand access.”