Watch amazing Meta AI translate language in near real-time

Seamless also retains the original speaker’s tone and emotion.
Sign up for the Freethink Weekly newsletter!
A collection of our favorite stories straight to your inbox

Meta has unveiled Seamless, a new language translation AI that can quickly convert speech in one language into another while preserving the original tone and emotion.

State of the art: Back in August 2023 (what feels like ages ago in the field of AI), Meta got a major step closer to its goal of creating a universal language translator, unveiling SeamlessM4T, an AI capable of translating recordings of audio or text in nearly 100 languages into text in any of the languages (or speech in 36 of them).

“In the future, we want to explore how this foundational model can enable new communication capabilities — ultimately bringing us closer to a world where everyone can be understood,” wrote Meta at the time.

What’s new? Meta has now unveiled an updated version of that language translation AI, SeamlessM4T v2, which it says “demonstrates performance improvements for automatic speech recognition, speech-to-speech, speech-to-text, and text-to-speech capabilities.”

It has also shared two new AIs built on the updated model: SeamlessStreaming and SeamlessExpressive.

A typical language translation tool will wait for a speaker to finish a sentence before beginning to translate it. This helps overcome the challenge of different languages relying on different sentence structures — many languages have different subject-verb-object orders, for example — but it can also lead to delays that make conversations feel less natural.

SeamlessStreaming begins translating speech while the speaker is still talking. This means the listener can hear the translation with a delay of just a few seconds.

SeamlessExpressive, meanwhile, focuses less on what the speaker is saying and more on how they’re saying it, helping ensure the translation preserves the rhythm, emotion, and style of the original speech.

Open source: Meta combined SeamlessStreaming and SeamlessExpressive into a single language translation AI, Seamless, and made the code, models, and datasets used to create everything freely available on GitHub.

“By publicly releasing our work, we hope that researchers and developers can expand the impact of our contributions by building technologies aimed at bridging multilingual connections in an increasingly interconnected and interdependent world,” Meta researchers write.

We’d love to hear from you! If you have a comment about this article or if you have a tip for a future Freethink story, please email us at [email protected].

Related
LLMs are a dead end to AGI, says François Chollet
AI researcher François Chollet thought we needed a better way to measure progress on the path to AGI — so he made one.
Meet Thresh, the world’s first professional gamer
Was Elon Musk any good at Quake? “He’s a legit gamer,” but…
You’re thinking of the metaverse all wrong, says Matthew Ball
Rumors of the metaverse’s demise have been greatly exaggerated.
Perplexity, Google, and the battle for AI search supremacy
AIs that generate answers to user queries could transform search, but only if someone can get the tech and the business model right.
How AI is rewriting Silicon Valley’s relationship with the Pentagon
Silicon Valley is warming to the Department of Defense as it works to get new AI systems developed and deployed en masse.
Up Next
A robot is working in a factory with a lot of bottles.
Subscribe to Freethink for more great stories