What just happened? If you've ever watched a video dubbed into English and been put off by a speaker's lips moving out of sync with the words, you might love Google's new service. Not only does it offer real-time translations, but it also edits video so the lips closely match the translated audio.
One of the many new products Google unveiled at its I/O conference yesterday was its experimental "Universal Translator," part of its AI push.
The tool works by taking a source video – in this case, part of an online college course lecture created in partnership with Arizona State University recorded in English – and translating it into another language using next-generation translation models.
Google's product does more than simply translate; it also imitates the voice and tone of the speaker in the translated language. But the highlight is the way the video changes the speaker's lips to sync with the new language, though it's not perfect just yet.
Google says its Universal Translator is an enormous step forward in learning comprehension, and it has already seen promising results with an increase in course completion rates.
Editing a person in a video and putting words in their mouth is, essentially, a deepfake, which is why Google is limiting access to its Universal Translator to authorized partners for now. The tech giant admitted that while the tool can be enormously beneficial in many cases, the underlying technology has the potential to be misused by bad actors.
Google talked a lot about taking an ethical approach to AI during its I/O event, such as creating tools for helping users spot online misinformation, metadata tagging for AI images, and guardrails for undisclosed deepfakes. It comes at a time when China's Tencent is now offering to create convincing deepfake videos of anyone for $145.
You can check out the entirety of Google's I/O event right here. In addition to all the AI-based announcements, the company unveiled the Google Pixel 7a, Pixel Fold, and Pixel Tablet.
