Google Celebrates GAAD With New Enhancements to TalkBack, Expressive Captions, More

Google marked this year’s Global Accessibility Awareness Day late last week by publishing a blog post wherein the Mountain View-based company announced numerous accessibility-oriented updates for its myriad platforms. The post was written by Angana Ghosh, who’s Google’s director of product management for Android.

“Advances in AI continue to make our world more and more accessible,” Ghosh wrote in the post’s introduction. “Today, in honor of Global Accessibility Awareness Day, we’re rolling out new updates to our products across Android and Chrome, as well as adding new resources for developers building speech recognition tools.”

Ghosh’s post begins by discussing “more AI-powered innovation with Android,” with Google’s screen reader, known as TalkBack, getting expanded Gemini integration such that users can ask the chatbot about imagery and get answers. Ghosh cites an example of a Blind user asking about a picture of a friend’s guitar, writing the user can ask for details about the musical instrument such as its color and manufacturer. Likewise, users also are able to query Gemini about product sales in their favorite shopping app(s) so they can be more informed about discounts and their overall buying power.

Google first brought Gemini to TalkBack last year, according to Ghosh.

Elsewhere, Expressive Captions, which uses AI to not only telegraph what people say but how they say it, is being updated such that Deaf and hard-of-hearing people can “understand mooooore of the emotion behind captions.” Ghosh notes Google has added a new “duration” feature to Expressive Captions that’s useful for times when, for instance, a sports announcer excitedly boasting about an “amaaazing shot” during a game. What’s more, there are new labels for sounds like whistling or throat-clearing. The updated version is available on devices running Android 15 or higher, with localization in English in Australia, Canada, the United Kingdom, and the United States.

In other news, Ghosh writes Google has expanded availability of its Project Euphonia, announced in 2019 as a way to make speech recognition more accessible for those who have non-standard speech pattern (like yours truly). Google is making its own-source codebase available of the Project’s GitHub repo, as well as working with the University College London’s Centre for Digital Language Inclusion to strengthen speech recognition technology for non-English speakers in Africa. On the educational front, Google announced accessibility improvements to ChromeOS and the Chrome web browser, including more accessible PDF reading and page-zooming functionality.

I interviewed Ghosh back in December about building Expressive Captions.