The "universal translator" which was part of Star Trek and a lot of other Sci-Fi I was exposed to as a kid was something I was really fascinated with. My Dad worked as a simultaneous French->English translator and sadly spent long hours away from home and, as a kid, I started trying to build a translator so that it could do his work and he could be home more.

Translation is important work and one that could help a lot of people. It's my hope that we get to the point where these models work entirely on locally carried resources.

I did several experiments recording from all the microphones I could on my iPhone and AirPods while out in the wild. My conclusion: it's impossible right now for that hardware given the microphones we have and what they pick up.

So much of what's spoken is at a combination of (a) high distance (b) low volume (c) background obscuration. Something that was clear as day to my ears would barely register on the mics. While context is of course an issue, the raw audio didn't have enough to even translate.

The one caveat is that there might be low-level (i.e., Apple-only) access to headphone microphones that capture the environment to do noise cancellation. I'm not sure though---I couldn't find them on any API.

For cases where you do have clear audio, existing apps (e.g., Google Translate) are so close to achieving this, but don't let you specify audio outputs with enough fine grained control. By default, it will start screaming out of your phone what you were attempting to silently translate.

"Can you stand up?" would be translated differently into Japanese depending on whether you're implying you need them to move their butt off your cell phone versus directly inquiring as to the function of their legs after a car accident. If you speak English and hear it as a background without the rest of the context being picked up, your brain instinctively knows it can interpret it either way, no problem.

But if you're Japanese and the AI picks a specific way to translate it, then you are completely unaware of the ambiguity because the AI resolved it with a 50% chance of being wrong.

That is, they are able to translate (in all directions) novel languages that were not previously heard[0]. It is an open question, with likely a negative answers, that there is a universal grammar even among humans[1] (the definition itself is vague but even the most abstract version is suspect and highly likely to not be universal across species). I think no one will be surprised if it is always impossible to interpret an entire language based on only a few words (let alone do it in real time)

This isn't a knock down, because even a trained device is insanely useful, it's just a note about limitations and triage. This is awesome stuff and I can't wait for the day we have transnational headphones. It's an incredibly complex problem that I'm sure is not short of surprises.

[0] There are a few exceptions such as Star Trek TNG's episode Darmok, S5E2, where the Tamarians' language is unable to be translated due to its reliance on cultural references (the literal words are translated but the semantic meanings are not). It's a well known episode and if you hear anyone saying "Shaka, when the walls fell" (translates to "Failure") they are referencing this episode (often not using the language accurately but who cares (nerds. The answer is nerds)).

[1] https://en.wikipedia.org/wiki/Universal_grammar

> The Babel fish is small, yellow and leech-like, and probably the oddest thing in the Universe. It feeds on brainwave energy received not from its own carrier but from those around it. It absorbs all unconscious mental frequencies from this brainwave energy to nourish itself with. It then excretes into the mind of its carrier a telepathic matrix formed by combining the conscious thought frequencies with the nerve signals picked up from the speech centres of the brain which has supplied them. The practical upshot of all this is that if you stick a Babel fish in your ear you can instantly understand anything said to you in any form of language. The speech patterns you actually hear decode the brainwave matrix which has been fed into your mind by your Babel fish.

“The argument goes something like this: ‘I refuse to prove that I exist,’ says God, ‘for proof denies faith, and without faith I am nothing.’

“‘But,’ says Man, ‘the Babel fish is a dead giveaway, isn’t it? It could not have evolved by chance. It proves you exist, and so therefore, by your own arguments, you don’t. QED.’

“‘Oh dear,’ says God, ‘I hadn’t thought of that,’ and promptly vanishes in a puff of logic.

“‘Oh, that was easy,’ says Man, and for an encore goes on to prove that black is white and gets himself killed on the next zebra crossing.

“Most leading theologians claim that this argument is a load of dingo’s kidneys, but that didn’t stop Oolon Colluphid making a small fortune when he used it as the central theme of his best-selling book, Well That about Wraps It Up for God.

“Meanwhile, the poor Babel fish, by effectively removing all barriers to communication between different races and cultures, has caused more and bloodier wars than anything else in the history of creation.”

I think it's totally doable but you'd need many more microphones in order to deal with real world noise. As MEMS microphone quality improves, this should eventually be possible with a combination of smartphone/headphone/some other device like something around your neck.

So no matter what, conversations in your native speech would have to be delayed before translation.

I'm reminded of Mark Twain complaining about verbs arriving at the very end of sentencess in German (among a myriad of other complaints)

"The Awful German Language* -Mark Twain https://faculty.georgetown.edu/jod/texts/twain.german.html

Pretty sure there is nothing universal there though as you say.

By keeping creating speaker specific tonal ranges and profiles you maintain the better cohesion on the final product.

I grew up bilingual outside the US, and speak English with a hybrid British/Indian/Middle Eastern accent (with some of my personal quirks, and mixing increasing amounts of various American accents over time). I can understand English in nearly any accent (Singaporean, Chinese, Vietnamese, Indian, Nigerian, eastern European) as long as the words involved are globally used and the grammar is passably queen's. Especially after hearing it for about an hour. And people who natively speak English with these various accents usually can understand my English better than they can an average American accent. Yet in this country, my accent is belittled, despite being perfectly understood and more versatile. Even by others who don't speak with the American accent!

This is the problem of the "default accent" anywhere being referred to as "no accent", and therefore anything deviating is considered "having an accent". This makes "accent" a negative trait, scaling from 0-bad to heavy-bad. But if the vernacular were such that we said "American accent" instead of "no accent", then noone's accent is bad, just not used to.

Most of my non-American peers who were raised on English have a better command of the language than my American ones, yet they are mocked for their accents as if they don't know the language, when in reality it's the Americans lack of familiarity with the language (as its used globally) preventing them from comprehending the language.

So yes, put in more work, the world is shrinking and English is the global language (for better or worse). What you're saying is spoken from a position of privilege because the culture allows you to mock others' accents and imply your version of it is the correct one that everyone else should put in work to provide you with, rather than the other way around.

Every time you hear English with an accent other than British, American or Australian, remember that it usually means the speaker knows at least one entire other language as well, probably one that you would sound like an idiot if you tried to speak it. Don't be rude or dismissive of their command of English.

In fact, you were so close — you called it a "no accent am-english", when you could have just called it what it is — "an american accent".

And I mean, yes, there are people who know they don’t sound like whatever ideal accent they have in mind, and there are people who will make fun of accents - but, and I can’t stress this enough, depending on the context literally any accent can be made fun of, sadly. I’ve had people mock my “American” accent while travelling, for example. It sucks, but it’s not easy to single out any accent as “default” unless it’s literally enforced by a government and taught that way in schools. Last I checked, the US is not one of those countries and English is not as centrally controlled as e.g. French can be.

What accent? Whose accent? Brits are as diverse accent wise as Americans, London, cockney, New England, Southern...

A lot of Indians that I know have a very "proper" British accent, one that maybe a bit aristocratic, its quite an irony for a former colony. https://www.bbc.com/future/article/20220915-what-the-queens-...

The context matters, but so does history.

There are many american accents. Your suggestion makes the sentence much less clear.

And by specifying "american" they're already making it clear there is no such thing as a universal base accent for english.

I could of been more specific, but my request for the tech to vary, I think would lead to specific options for different people.

And actually to be even more.. not sure the word.. I want 'the Chicago accent' I think it's called, or midwest / no accent. Personally as much as I enjoy some entertainment from Jersy / NY accents, I would not volunteer to watch tutorials on tech taught by the Sopranos cast - as funny as that might be (and I get if you are from the NE, you may be learning just fine being taught with such a language style).

As annoying some of the Cali style of language is, I can understand the words and meanings without squinting my ears and spending double the brain cycles trying to understand the words, while then interpreting the meaning, and then trying to put together concepts for understanding new ways of coding or using tech.

I've run into folks in Louisana that I could not understand at all and had to ask for an interpreter at a gas station. From Florida to Chicago to Seattle down to Miss and Ala - I can hear what people are saying and learn without spending lots of extra energy trying to understand.

With that being said, I understand there are parts around Miami where accents may be thicker (or not) - and with some folks even if using the rights words and grammar, I may need to slow down the speech to actually learn if they were teaching a class.

The slow down and speed up options already exist with youtube.

"So yes, put in more work"

- I do try a bit. I don't mind accents with some folks and media.For example I can listen to and enjoy Shankar sharing via the 'hidden brain' series, partially because his accent is limited but also because the media requires less thought intensity.

I have tried many youtubes, and bought a few courses taught from folks in India and other places where I just could not muster the energy. I literally squint with my ears and feel like my head gets hot trying to decipher what is being said, translate into what is meant, and how it should create new patterns of understanding in my brain.

I can only do that for so long and I am done. Now I just skip any learning video that has non-am English speakers. When I consider courses to sign up for or buy, I have to research the authors / speakers and find video of them to hear the audio, because I just can't learn well that way.

"other than British," - True story, a few years ago I had to call an ISP in Britain(?) and the person I got to to file an issue with, I could not understand them. I had ask 'what did you just say' many times. I laughed at myself for even thinking of saying 'can you slow down and speak clearer English please' - I mean, crazy... I was paying by the minute for the long distance at the time and it ended up being a 25 minute call that could of been 10 if I had a magic translate without accent device.

"a position of privilege because the culture allows you to mock others' accents"

- This is truly not about mocking accents, this is truly about my lack of ability to learn well.

Yes, I would defintely sound like an idiot trying to speak another language. Like I said, I do not learn as well as some others.

Truly not my intent to be rude. I apologize if the shortness came off that way, I was trying to be brief in the hope that there's a chance that some tech like this exists and someone here could point me to it. Before I posted, I DDG'ed it and found a couple of things attempting to be in that space with a 'speak to sales' type of 'you'll never afford this' button for info.

I will never be dismissive of anyone's command of English, or other spoken language, or computer language or anything like that. There is no way for me to know someone else's situation and circumstances led them to their current command of whatever language. If someone is trying to learn more at any age; I applaud and encourage them - being rude or dismissive does not encourage more learning.

"no accent am-english", when you could have just called it what it is — "an american accent". - Well maybe, but actually I meant to be more specific, as mentioned a bit above - I mean '"no accent" American accent' - because there are plenty 'American accent' types that I would want removed by a magic earpiece to make it easier for me to understand and learn.

There is no "no accent". An accent is a baseline feature of intelligible human speech, like a voice, or a volume, or a language. You can't say stuff without those features. When you say "the Chicago accent", or the "Midwest accent", that's an accent! Not "no accent".

I understand it's common usage to refer to the default "radio accent" as "no accent", but in a country like America, all kinds of people with all kinds of accents speak English. Reinforcing an expectation that a certain (usu. majority-white-spoken) one is the "default" by referring to it as "no accent", implicitly suggests all others are erroneous affectations, even if I trust that is not your personal intent.

All that said, I think your idea for a translation device capable of revocalizing what is said with an unfamiliar accent into one you are used to is not a bad one, and likely easier than translating between languages while retaining expressiveness.

Wow, you just keep digging in don’t you? When these Americans you deride say “no accent”, do you think they are referring to the “majority-white-spoken” Scottish accent?

No, of course not. Get that race baiting out of here.

I have a pair and have only asked it that so far...

Kind of like having an expert next to you all the time.

I’ve been in mixed language communities in which I wasn’t sure who spoke what, and I have found this to be quite effective when done right.

Good time to reference st:ng “darmok” episode and quotes like “darmok and jalad at tanagra”.

In my other languages that I am actually fluent in, it’s kinda the same — you use specific suffixes to soften or embolden your point and so on. Maybe add “exclamation making sounds in specific language” too. Eventually your nouns and verbs end up in different languages, with different suffixes where it “makes sense”, yet the person whom you’re talking to will “get it”.

Would be curious to try the new Seamless model on such speeches.

The only thing that doesn’t work is if you talk to people too young to remember Skype. Then you feel old.

I don’t know for cockney but verlan is very alive.

Question is, what will happen to the tribal / local languages? Will they survive?

This might be required to get full buy in for a unified language, which is a bit sad but makes some sense - if you ensure it's taking up more and more of media and culture more people know it from immersion, and other languages are reduced to being spoken at home / with friends and that's going to cut into how many people really are fluent in them.

Everyone gets a personal tutor for hours a day.

I would absolutely love a VR game where I just need to work in China or Mexico all day and pick up the language that way.

I think a lot of people are working on similar things right now. I know of one called http://yourteacher.ai

[1] https://apps.microsoft.com/detail/9NC624PBFGB7

> coming soon: "We've been getting a lot of these tourists here lately, they're eerily fluent, but all seem to have the same minor speech impediment"

Haha, if that were to pass, that would still be a far better outcome than our current situation of completely blind machine translation (this is especially for various Asian languages that are very sensitive to phrasing) and mispronunciation by non-native speakers.

Ah, that is called an accent.

This would be something quite novel as the speech irregularities would not have their origin in people

I don't know what you would call it but it needs at least some adjective before accent to differentiate it IMO

I don't have the expertise to judge the quality of Mandarin pronunciation myself, being a beginner. But it sounds OK in English and it's made by native Mandarin speakers in China so I expect that it sounds better in Mandarin than English.

i’m hoping a voice as realistic as this becomes a local app soon, but i’ve not found anything that’s nearly as natural sounding yet. (also, honorable mention to chatgpt’s “sky.” she pronounces mandarin with a funnily american accent, but it sounds natural and not as robotic as the open-source alternatives i’ve tried)

I resisted using Duolingo because I knew their speaking feature sucks. But the only reason I need an app rather than books or audio tapes is that I need something to correct my prononciation.

https://fluent-forever.com/product/fluent-forever-pronunciat...

I'd love to learn French and the game would take place in locations all around modern France.

It would have to a good story. Maybe something in the style of Professor Layton series could be interesting, or something more open world.

https://www.proseable.com/

You end up with the Duolingo problem where you know to say the names of 20 different fruits but not how to introduce yourself.

Not sure if this is a duolingo problem. There are of modules in duolingo specifically for saying your name. I think its the travel module.

https://github.com/adrianmfi/gpt-tutor

I look forward to testing if switching to Seamless can improve it further, Seamless supporting nearly 100 languages is a nice improvement.

Yes! Better yet, you're a spy, or a hostage negotiator, or the leader of any kind of enterprise (army, business, aid organization) ...

Programming games like that will resemble directing improv theater. You can't program every response; you'll have to instead fit each character with beliefs and motivations.

I can hardly wait.

What would be really cool is something that can autodub videos or audio into your target language. The hardest problem learning languages that aren't English is often finding content to consume in them.

Disclaimer : I am Krashenist so this take is biased

Obligatory reminder that the movie itself explains that people are what they are not because of their lifestyle, but because of the time spent in low-gravity environment.

The parents suggestion is that if we don't have to learn languages that will lead to us all laying down drinking big gulps while robot slaves take care of us. Their take is the extreme example. People have literally made this same suggestion about every technological advance and it never comes true.

But there's a point in language learning where you can come to express yourself directly in a new language without intermediary "thinking" in your first tongue. The communicative and expressive potential of that mode is much higher than trying to squeeze one's intent through any kind of translation, machine or internal.

Plus, you know, it's fun.

However, translation has a great deal of subjectivity embedded in it, particularly when there aren’t 1:1 translations. Case-in-point: there are many English translations of the Christian bible, all similar enough, but there are enormous variations in some cases. And there are at least as many branches of Christianity as there are English translations of the Bible. Some of them strictly recommend the same translation, and they still disagree on the meaning of various passages.

Besides the problems inherent to translation, learning another language gives you another paradigm of thinking. The words we use, the way we construct sentences, etc., all impact our view of the world. Here’s a paper that discusses the impact of the over-reliance on English in cognitive sciences, and how this has downstream effects: https://www.sciencedirect.com/science/article/pii/S136466132...

Learning languages as an adult also has protective benefits. It reduces the probability of Alzheimer’s (maybe dementia, overall?).

https://www.parcero.ai/

I could integrate this instead of Polly pretty easily.

He ended up rolling his own solution by standing up Whisper in one of our clusters and writing a basic front end and API to take his laptop’s mic input and chunk it every few seconds to send to the model and get back text in pseudo-realtime. We got him a pretty beefy Alienware so he wouldn’t be tied to the cluster GPUs. I can’t wait to see what he does with these new models!

We need more support for employees like this!

Also, what about Apple’s latest M3 series chips? Are this in the same realm as Alienware in terms of AI compute?

It reminds me of the Homer Simpson quote, "I don’t mind being called a liar when I’m lying, or about to lie, or just finished lying, but NOT WHEN I’M TELLING THE TRUTH!" I would be equally critical if it was warranted, but when it isn't it's deeply unfair to the accused.

If the person wanted to build their own ramp, and the employer let them do it on the clock, that's a completely different scenario than the employee having to come in during their off-hours to build the ramp just so they can go to work.

As someone who’s profoundly deaf myself, another less technical approach is to install Rogue Amoeba’s Loopback, and use it to pipe audio from a given app into a tool like Google Meet or Otter.ai using the Loopback device as the audio source. This effectively provides real time captions for anything running on your existing machine.

The nice thing about Chrome feature is you can move the caption box around and keep it in the foreground while doing other things, although styling options seem limited (the text might be a little small for some).

[1] on desktop, not sure about mobile

[2] via chrome://settings/accessibility -> Live Caption

Additionally, it supports multiple languages (only one at a time sadly), so I also use it for Japanese captions and it's equally great there.

But hearing loss does not impair standing up servers and software. They can pay the employee who probably is the expert at this, the guy with the hearing loss, or go task Emil to go do it to ... avoid 'appearances'?

- Whisper processes 30 second audio chunks. So if you process 5 seconds of audio you have to pad it out with 25 seconds of silence. Hence a loss of efficiency with wasted CPU / GPU cycles on 25 seconds per chunk in the case above.

- Whisper most likely can't handle hundreds of commands much less than a thousand performantly.

- Whisper doesn't handle short commands very well with a degree of accuracy post processing commands from free dictation utterances.

Command dictation should be weighted higher than general dictation when decoding.

I work with a little under 1500 of commands dragon naturally speaking. DNS is hot garbage as a program despite it has the best accuracy to date with the feature of commands and dictation in one utterance. You get to pay $750 for the privilege m

I've yet to see a free and open source speech recognition engine that can handle both dictation and commands with a high degree of accuracy.

Please please let me know if there's alternatives out there. I would definitely pay to support an open source project like this that focuses on command and dictation.

Most solutions out there that are open source nowadays focus so much on iot command recognition with intents. That's not well suited for controlling your computer with grammars containing voice commands.

> Input audio is split into 30-second chunks, converted into a log-Mel spectrogram, and then passed into an encoder.

https://openai.com/research/whisper

Simply voice to text is not what's needed for dictating commands. Unless I can load commands of on the fly and decode utterances that may be useful.

The client would need to be able to send its commands to the server on the fly.

Computer: webcaptioner.com Android: Live Transcribe (g.co/livetranscribe) iOS: Live Caption with the 'mic' icon enabled.

Web conferencing: Meet, Zoom, Teams all support realtime CC, which is pretty good.

There you go. https://github.com/dictation-toolbox/dragonfly

Probably this.

I will note though that I feel safer getting an occasional bad word than I do having a translator straight up deceive me.

For example, "what the fuck" in English->Spanish is giving "qué diablos" output. Definitely toning down the meaning there.

If someone says something mean to me, I want to know it.

Would be interesting to see some work stress-testing the ability to convey ill-intent across multiple languages. Accurately conveying ill-intent is safety-critical for the person being threatened.

I told her then that the industry would be disrupted by AI before she retired.

Glad she pivoted. Really impressive results.

Yes. I just discovered there is a text-to-speech addon [1] (now a few months old) for World of Warcraft that adds voices for every NPC in the game... It is so impressive and game changer (pun intended) that I naively asked in the chat of the Twitch stream I was watching "when did Blizzard add voices to the NPCs??". For an instant I really thought Blizzard contracted actors, but no, someone like you and me just used AI to generate realistic voices for every character in the game. I don't think it's ready yet to completely replace actors in video games (surely it will in the near future tho) but voice acting is something so expensive to do that I can see studios and developers in 2024 already use this tech for all the optional dialogues and secondary characters' voices.

[1] https://www.curseforge.com/wow/addons/voiceover

https://youtu.be/kZ87wiHps9s

Typesetting. Music engraving. Bookbinding. The quality of all these fields have been materially harmed by advancements.

Computer typesetting has, by and large, been a significant regression, though the gap has largely been made up now if you make the right choices.

Published music scores used to be set by experts. Now they’re set by novices using software that is mechanical in method and generally quite insipid. Most are atrocious compared to the old masters, and mediocre at best compared to the typical published scores from a hundred years ago; and very few popular scores are really good (… and if they are, there’s a reasonably high chance they’ve used GNU LilyPond, which has focused on this problem). But the barrier for entry is so much lower, and people have got used to the inferior results, so I don’t know if anyone engraves music the old way, and even people that know better largely just shrug and make do with the new. Like with computer typesetting, there is hope because things have slowly improved. But most will continue to be mediocre.

Books used to be bound with cold glue. It takes time to set, but the results are very good, supple and long-lasting. Then along came hot-melt glue, and it’s just so much friendlier for cheap manufacturing because books are finished within a few minutes instead of a day or two, that I don’t think anyone produces books the old way any more, even though the results are abysmal in comparison (compare the binding and reading experience of a paperback from the ’40s or ’50s with one from the turn of the century; no one after tasting the old will desire the new; for he says, the old is good). But they’re just (barely) good enough. Unlike the other two, I don’t think there’s any hope here—the regressive advancement crowded out the superior but dearer option so that no place was found for it.

What seems to universally happen is, the market bifurcates - one part is in a race to the bottom, the other (much smaller) aims for super premium tier (overpriced quality), because only those two positions are sustainable, once the race-to-the-bottom side drags all the economies of scale with it. So as a consumer, you get to chose between cheap low-quality garbage that's barely fit for purpose, and rare, super-expensive, professional/elite high-end products. There is no option for "good value for reasonable price".

This has been happening to everything - software, furniture, construction, electronics, vehicles, food, you name it.

Sure the final result sounds slighty robotic. 99% of people wouldn't care, and you can get more training videos done, faster for a fraction of the cost.

[Edit] And I'll add the difference from 6 months ago is noticeable to today. I imagine every 6 months we can just re-download updated voiceovers and every 6 months will sound just slightly more polished..

a few more years of improvements if they happen could be disruptive

I'm the most excited for an open source one though, and it would be incredible if this could become it. I do 95% of my compute on desktop linux and it sucks being behind.

(quote from the video presentation in the link)

I thought it was meant to be "toxic words, hallucinations, etc" in the script

It would be nice if they could layout what exactly is missing in terms of data to make a language work better, while the actual AI bit is out of reach for most of us maybe we could provide more data.

There is also a 60 sec limit and wonder if this is HuggingFace limitation or Seamless?

If you want to contribute by recording yourself speaking Swahili, https://commonvoice.mozilla.org/sw is the place to go. Although Meta has access to much larger data sets, they nonetheless use Common Voice as a "known good" source. E.g. the paper on their SONAR speech encoder reports experiments on Common Voice data, coincidentally involving Swahili https://ai.meta.com/research/publications/sonar-sentence-lev...

As a student in the mid-90s I worked on a system called Verbmobil at the German Research Center for AI and it did speech-to-speech for English, German and Japanese in very limited domain.

This was done via "classical" NLP: You had to model the domain with concepts, you needed sentence parsers, semantic engines, speech-to-text hand-crafted for 3 languages etc.

As it turns out, this approach is/was a dead-end.

Yeah, I kinda agree with the spirit of your comment; it sure would be nice to see a major Indian language like Telugu on their landing page for sure. But that's just my Indian-person bias speaking.

is there an example notebook that shows how we can use this model with own sample audio and text? thanks!

I don't think this would fool anyone that I was a real native speaker of the target language, but for casual conversation this would work pretty much perfectly. It basically avoids all of the traditional pitfalls of machine translation, like the unnatural robotic voice that it outputs, the slow translation speed and huge latency for realtime conversation, and the loss of emotion.

Just text to speech has gone too far. Audio books would be mainly generated on the fly like this?

I think some RPGs in some 5 years time might have something like this:

- A text file that outlines characters and a lose plot/Story line. Human written.

- 3D Mesh Generation based on character description via Transformers based models. Auto generated.

- Dialogues for each NPC via LLM.

- This TTS engine again based on such models.

Result - almost unlimited replayability. Or even edit text file, have a new world based on a new story line with characters having different personas.

[0]. https://news.ycombinator.com/item?id=38335255

That said, I suspect that real-time language translation is always going to be somewhat imperfect due to its nature. Non-real-time translation of literature is still a subjective art form even at the very high-end of human expertise.

It's all a trade off.

Either way it's extremely exciting that we get to even discuss this stuff as real possiblities.

This may be apocryphal but I’ve heard that in formal settings (e.g. UN) they won’t translate it and will instead give instruction on when to laugh.

I’ve been learning german since 8 years, and the amount of expressions and different ways to say things around the country is impressive. There’ll be a “interpretative” real-time translation, but it won’t guarantee fully understanding in so many cases, maybe ever.

Other thing, and we have this in common with all languages, is the context and this is difficult to address i believe.

Nevertheless, it’s impressive how far we’ve reached and i acknowledge the usability of these tools. However, human knowledge will be always crucial and primordial if we want to guarantee full understanding.

"Since", as used here, would lead me to guess you are not a native English speaker?

So it can't be trusted at all then

Also: "This research demo is not open to residents of, or those accessing the demo from, the States of Illinois or Texas"

It's not dissimilar to some kind of a "ch'ti" / "chtimi" accent or a belgian-french accent (which is not dissimilar to the french ch'ti accent, heard in some part of the north of France. "Ne partez pooooo" (with a longer "a" which sounds nearly like an 'o': that's not proper french at all) instead of "Ne partez pas".

That's said I'll take the non-expressive accent any day over subtitles for when watching video in a language I don't understand: it's clearly good enough.

Besides the ACCEPTABLE_USE_POLICY, there's a CC BY-NC 4.0 (NonCommercial) license, a 'SEAMLESS_LICENSE' (NonCommercial), but also an MIT license? It would seem these other licenses contradict the MIT license, could somebody help clarify how these all interact in practice?

https://github.com/facebookresearch/seamless_communication#l...

https://seamless.metademolab.com/expressive/?utm_source=meta...

Interesting mix.

Facebook has had to pay out hundreds of millions of dollars in settlements for related class-action lawsuits, and rather than trying to get informed consent, they’re deciding not to collect biometrics from residents of those states.

In Texas it seems to be part of AG Paxton's culture war stuff. https://www.texastribune.org/2022/05/12/texas-face-filters-i...

> According to the Settlement Administrator, payments to class members between $200 to $400 started going in the mail May 9.

I got a $0.19 check from an iTunes settlement once, but this wasn't one of those cases.

Edit: I can’t find the weights but if I’m reading the paper right anyone could train their own detector.

Yes, we chose not to release the watermark detector to safeguard against adversarial attacks. This decision helps prevent any attempts to erase the watermark by malicious users.

The watermark generator and detector are trained together, one can use the information in our paper to train your own generator and detector model, however in this case the watermark signature created will be distinct from the one we use to protect our seamless translation models. This approach ensures each model maintains its unique security features.

It runs but any audio input (you will need to provide wav not mp3's) I tried (tried 20s/40s/300s) I get just one short sentence returned in target language that seems not related at all to my audio input (i.e. Tous les humains sont créés égaux).

Seems like some default text but it runs on full GPU for 10 minutes. Tons of bug reports in GitHub as well.

Text Translate works but not sure what is the context length of the model. Seems short at first glance (haven't looked into it).

Oh and why is Whisper a dependency? Seems not need if FB has their own model?

My ultimate goal is to have realtime translations of video conferences. I've moved to a new country, and while I'm super privileged that most of my colleagues speak English, we still have a number of "all hands" meetings that I get lost in pretty easily.