Skyrim modders have a new machine learning tool that turns text to realistic NPC speech

[Cal Jeffrey]

In brief: The modding community has a new tool to help them create original dialog for their mods. The app is called VASynth and is available on the NexusMods website. VASynth uses AI to convert text into realistic synthesized speech.

VASynth creator Dan Ruta used NPC voices from various Bethesda video games to train the tool's machine learning AI models. He published multiple flavors of the app, including variations for Skyrim, Fallout 4, Oblivion, Fallout New Vegas, Morrowind, and Fallout 3. Each has a selection of models to download optionally.

Ruta also plans to release a Starfield version "soon." He presumably means sometime after Bethesda launches the game, which still doesn't have a release date. In the meantime, the modder will continue adding additional models and work on training existing voices to improve quality.

Ruta posted a demo (below) of how VASynth works, which is narrated by several voice models. The demo is relatively crude, and the speech often sounds unnatural. However, the characters explain and provide examples of how to improve the spoken words using punctuation, spelling tricks, pitch, and speed controls. Ultimately, users can record realistic voices with a little effort.

Another modder posted a Skyrim trailer (top) as an example of what kind of results are possible with VASynth. While some phrases don't sound quite right, many others are very convincing. Fearing that some might mistake the voices created with VASynth as the actual Bethesda voice actors, Ruta warns users not to misuse the app.

"To keep things fair, avoid using the tool in an offensive/explicit manner," he cautions in the notes of each version. "Make it obvious where you can in descriptions that the voice samples are generated, and not from the original voice actors. Any issues you cause with this are on you."

One of the first examples of a gameplay mod using VASynth is called "Positive Undressed Reactions." As then name suggests, the mod causes Skyrim NPCs to complement players if they walk around nude. It's a rather silly application of the tech, but one can easily see this tool being used to create scripted quest expansions for any of the supported Bethesda games.

Found is a TechSpot feature where we share clever, funny or otherwise interesting stuff from around the web.

Permalink to story.

https://www.techspot.com/news/88353-skyrim-modders-have-new-machine-learning-tool-turns.html

 

[Bulllee]

Great that the modders are getting a mention on here.
True heroes of the gaming community who put the time in.

 

[noel24]

"I used to be an adventurer like you, but then I got cease and desist order from Microsoft."

 

[captaincranky]

Hopefully toward your greater gaming enjoyment, an ode to "Skyrim".......

 

[terzaerian]

This is big enough news it has me weighing getting back into Skyrim modding again. The limits imposed by voice-acted lines put a ceiling on what was possible and seamless but this removed the very concept of limits, assuming it doesn't get DMCA'd.

That said, of course the first thing they do is find out how to integrate it with a nude mod, lol. Never change, Skyrim mod community

 

[mbk34]

It's a shame they can't do something like this for a new Kindle text to speech facility. They had it on the old kindle 2 but it was very poor. I still miss the feature though.

 

[mbrowne5061]

This is big enough news it has me weighing getting back into Skyrim modding again. The limits imposed by voice-acted lines put a ceiling on what was possible and seamless but this removed the very concept of limits, assuming it doesn't get DMCA'd.

That said, of course the first thing they do is find out how to integrate it with a nude mod, lol. Never change, Skyrim mod community

I doubt it will get DMCA'd. Bethesda practically survives on the good will of their modding community. They know they can release a buggy game with a terrible Ux/UI, and the modders will fix it for them. I half suspect this is why they refuse to abandon the Creation engine, they are afraid they will lose their more experienced modders if they replace it with something more capable and robust (but more complicated to develop for, most likely)

 

[terzaerian]

I doubt it will get DMCA'd. Bethesda practically survives on the good will of their modding community. They know they can release a buggy game with a terrible Ux/UI, and the modders will fix it for them. I half suspect this is why they refuse to abandon the Creation engine, they are afraid they will lose their more experienced modders if they replace it with something more capable and robust (but more complicated to develop for, most likely)

The whole area of AI-generated content is going to become a huge area of legal contention in the future, I imagine, as the companies creating and running the AIs and the rightsholders of whatever content is being fed to them start getting into fights. As soon as something AI-generated in this fashion becomes a big hit in the mainstream, it's going to heat up quickly. The internet made IP-holders prickly already but this has the potential to crank things up to 11.

The scary part is going to be the judges deciding these cases, especially as the quality of them has likely taken a huge dive in this century in particular as the process has become more politicized. Most of them probably couldn't start their computer up in the morning without an intern helping them.