Google's Live Captions could be revolutionary (one day)
Google went out of its way during I/O to emphasize the ways it's trying to make its products more accessible. One project that could have a huge impact: Live Captions, a new feature coming to Android Q.
The feature automatically surfaces captions to any video or audio content on your phone in real time, regardless of whether you're online or off.
That would make just about all video and audio content -- including video calls, audio messages, and podcasts -- accessible to people who are deaf or hard of hearing. It would also benefit everyone who occasionally watches videos with the sound turned off. That this can happen instantly on your phone without a data connection is all the more impressive -- provided Google can pull it off.
SEE ALSO:What will Google call Android Q? We have some ideas.Google hasn't given it a launch date, though it says the feature will be available as part of Android Q. But I got the chance to try it out and I walked away mostly impressed, though it's clear there are some issues that need to be addressed.
When the feature rolls out, the idea is for Live Captions to be available throughout your phone regardless of what app you're using, so you can call them up any time there's audio playing. Whether you're watching a video or listening to a podcast, all you do is press the volume down key to bring up the Live Captions button.
One of the most impressive aspects of the technology is that captions begin appearing almost immediately, even when you're offline. (Like many other new features Google announced at I/O, Live Captions does all of its processing on the device itself, so no connection is needed.) My entire demo was on a phone that was in airplane mode and not connected to WiFi.
As for the actual captioning, well, it's clearly still very much a work in progress. When my colleague Raymond Wong and I tried it out with videos of us speaking, we got mixed results.
Tweet may have been deleted
When we were in a quiet room, and kept the phone's microphone close, the results were damn near flawless. Though the captioning system struggled with a word here or there ("hashtag" seemed to throw it off), its accuracy was otherwise impressive.
But, move the phone a few inches away from the speaker, and things were much less consistent. It misidentified a number of words even though the audio in the video was clear. These issues were even more pronounced when we tried out Live Captions in a room with a lot of background noise. In one instance, the captioning system generated significantly different text for the same video within the span of a few seconds.
Tweet may have been deleted
A Google spokesperson reiterated that it's still early for Live Captions and that the team behind it is still making improvements. That's true of any pre-release software, but it will be especially important for Google to address the issues we experienced.
Not all audio is perfectly crisp and clear. Some people mumble or have accents. We share video from noisy places or with many people speaking. While it's understandable these issues would be more difficult to address, it's important they are fixed, as it's more representative of the way people actually use their phones. If Live Captions can't work reliably in these scenarios, it will be much less useful and risks becoming an afterthought.
Still, if Google can make it reliable, Live Captions would make a dramatic difference for the people who need it most. Let's hope they get it right.
Featured Video For You
Get Google Assistant to talk in John Legend's voice
相关文章
- 市民填写献血相关表格8月22日晚,雨城区正黄时代天街广场天桥旁停靠着一辆献血车,身穿红马甲的志愿者们正向路过的市民发放无偿献血宣传手册,不少市民停下脚步,参与无偿献血。这是雅安市中心血站开启夜间献血模2024-09-21
Irish politician Matt Carthy hacked by own children
Hacking facial-recognition systems is so easy a kid could do it. One Irish politician found this out2024-09-21YouTube CEO promises to do more for creators amid scandals
It’s been a tough quarter for YouTube, and its most popular video creators are worried. Betwee2024-09-21The most hacked band password is 'Blink
The world is awash in '90s nostalgia — and it's even showing up in our passwords.A new study f2024-09-21- 8月25日中午,烈日当头,热浪滚滚。环卫工人王芳走进青江街道汉碑路社区户外劳动服务站点汉驿站,从冰箱里拿出今天的午饭。“我一上午在马路两侧来回保洁,到了中午能到驿站休息一会儿,喝点凉水,感觉非常舒服。2024-09-21
New York launches new investigation into Facebook
Another day, another investigation for Facebook to deal with. New York Attorney General Letitia Jame2024-09-21
最新评论