Call of Duty's new chat moderation AI sees major success: 43% drop in toxicity

[Daniel Sims]

In brief: Much speculation has arisen over the possible benefits that technologies like generative AI and machine learning might bring to video games. While image reconstruction has proven useful and generative AI has reportedly entered the production process, AI-based analysis has begun to impact voice chat moderation in competitive gaming spaces.

Activision Blizzard reports that exposure to toxic voice chat in Call of Duty has declined by 43 percent since the beginning of this year. The publisher credits the recent implementation of AI-based moderation for the results, which have convinced it to expand its use when Call of Duty: Black Ops 6 launches on October 25.

The publisher introduced the moderator, ToxMod, when it launched Call of Duty: Modern Warfare III last November. According to ToxMod's website, the system analyzes text transcripts of in-game voice chat and detects keywords based on multiple factors.

To tell the difference between trash talk and harassment, ToxMod monitors keywords and reactions to comments, recognizing player emotions and understanding the game's code of conduct. Furthermore, the system estimates the perceived age and gender of the speaker and recipients to understand each conversation's context better.

ToxMod can't ban anyone on its own, but it can quickly flag violations for review by human moderators. Activision then decides whether to warn or mute players, only issuing bans after repeated infractions. The number of repeat offenders in Modern Warfare III and Call of Duty: Warzone fell by 67 percent since Activision implemented improvements in June 2024. In July alone, 80 percent of players caught violating voice chat rules didn't re-offend.

All regions except Asia currently support ToxMod, and Call of Duty uses the system to moderate voice chat in English, Spanish, and Portuguese. When Black Ops 6 launches, Activision will add support for French and German.

Meanwhile, text-based chat and username moderation expanded from 14 to 20 languages in August. Community Sift has been Call of Duty's text moderation partner since the first Modern Warfare reboot game launched in 2019, blocking 45 million messages since Modern Warfare III's November release.

Using AI to moderate player behavior is far less controversial than employing the technology for in-game assets. Late last year, AI-generated art appeared in content skins for Modern Warfare III. Amid the historic number of gaming industry layoffs occurring around that time, some fear publishers will try to replace artists with AI models.

Permalink to story:

Call of Duty's new chat moderation AI sees major success: 43% drop in toxicity

 

[Theinsanegamer]

Good thing we fought the TOXICITY in the chat of our murder game! Wouldnt want to hurt anyone's feewings when you're brutally gunning each other down and stabbing each other in the neck!

I'm sure its just a coincidence that negative reactions to Activision's decisions get picked up as "toxicity" and any players that are notably better pick up bans because of butthurt noobs. Total coincidence, I'm sure.

 

[MSIGamer]

This sounds pretty cool actually. I don't play the game, but does it really work?

 

[maxxcool7421]

Good thing we fought the TOXICITY in the chat of our murder game! Wouldnt want to hurt anyone's feewings when you're brutally gunning each other down and stabbing each other in the neck!

I'm sure its just a coincidence that negative reactions to Activision's decisions get picked up as "toxicity" and any players that are notably better pick up bans because of butthurt noobs. Total coincidence, I'm sure.

im glad you agree with the bans!

 

[Squid Surprise]

I’d like to know what metric(s) they used to come up with 43%…

How can one objectively describe toxicity anyways?

 

[Theinsanegamer]

I’d like to know what metric(s) they used to cone up with 43%…

How can one objectively describe toxicity anyways?

From my experience, it can be anything from trash talking to saying "GGs" in chat to a particularly pissy teammate or opponent. Most people I know who still play online never talk to randos anymore because of the fear of the banhammer, an stick to private chat lobbies.

 

[Megalomaniac]

Well I'm definitely not playing. I refuse to be censored.

 

[emmzo]

I can take toxicity, I get it, most people are venting off when their a$$es are getting kicked, however constant harassing is extremely annoying and can throw you off your game. I haven't played CoD in a while, so idk if you have this, but I would give the player the option to mute and block people who he/she doesn't like.

 

[axiomatic13]

Remember in Battlefield games of yore when we would all PTFO and remain civil? Good times.

 

[Theinsanegamer]

I can take toxicity, I get it, most people are venting off when their a$$es are getting kicked, however constant harassing is extremely annoying and can throw you off your game. I haven't played CoD in a while, so idk if you have this, but I would give the player the option to mute and block people who he/she doesn't like.

Yes, the games have had the mute option for well over a decade. Which begs the question, why invest all this money into AI monitoring systems instead of telling people to just use mute? Wonder what kind of info they are after, and what daddy MS wants to collect.

 

[umeng2002]

Killing people = fine
Saying bad words = UNACCEPTABLE

 

[Lounds]

Jeez MW2 back in 2009/2010 was pretty toxic but we survived .

 

[yRaz]

Talking s*** was part of the fun of gaming. I miss having dedicated servers with their own rules. I remember in CoD4, I had a private server that I played on until probably 2013-2014. Get off work and game with the same ~50-60 people. I think the pop was limited to 28, but you know what I mean. It was the gaming equivalent of going to a bar.

 

[cfenton]

Killing people = fine
Saying bad words = UNACCEPTABLE

People aren't actually killing anyone. People are actually yelling slurs of all kinds at people. It's awful and makes lots of people avoid using chat.

 

[yRaz]

People aren't actually killing anyone. People are actually yelling slurs of all kinds at people. It's awful and makes lots of people avoid using chat.

Perfect, those sound like the type of people who have nothing interesting to talk about anyway.

 

[holydiver]

Only whiny snowflakes needed this kind of system, I never used voice chat or text chat to start with.

 

[holydiver]

People aren't actually killing anyone. People are actually yelling slurs of all kinds at people. It's awful and makes lots of people avoid using chat.

Then yell back or turn it off. Don't be a snowflake crying for mommy.

 

[cfenton]

Perfect, those sound like the type of people who have nothing interesting to talk about anyway.

Is your position really that people yelling slurs have more to contribute to a conversation than people who don't want to hear slurs?

 

[yRaz]

Is your position really that people yelling slurs have more to contribute to a conversation than people who don't want to hear slurs?

it's that people who complain all the time are no fun to be around

 

[cfenton]

Then yell back or turn it off. Don't be a snowflake crying for mommy.

Not wanting to listen to people using slurs doesn't make you a snowflake, it makes you a reasonable adult.

 

[Fanboiism101]

Game's gone

 

[cfenton]

it's that people who complain all the time are no fun to be around

I don't complain all the time. I do complain when people call me the N-word.

 

[umeng2002]

People aren't actually killing anyone. People are actually yelling slurs of all kinds at people. It's awful and makes lots of people avoid using chat.

Wussy

 

[yRaz]

I don't complain all the time. I do complain when people call me the N-word.

Words have power because people give them power. When you experience an emotion from someone calling you anything, it gives that person power over you. Rather than focus inwards on yourself to take away the power others have over you, you complain until someone else solves the problem for you. People are offended because they are weak and have no desire to become stronger.

That is as accurate of a way I can sum up the mix of emotions that cause my frustration on this issue.

 

[m3tavision]

You have to be 18 to play COD. At what age are you an adult/enter draft..?
Why does a game that comes with a language and adult content warning, moderating for toxicity?

This is nothing more than censorship campaign. If someone is being a bigot/racist it's very easy to report/record such an incident... so what is Ai moderating exactly?