Google API that detects insults, harassment and abusive comments is now available to developers

[Shawn Knight]

The intent of online forums and message boards is to foster the sharing of ideas regarding a particular topic although as anyone that has scrolled past a story or video will tell you, things often turn from amicable to flat out vicious in the blink of an eye.

The problem has only compounded over the years as more and more people make their way online but it’s not going unabated.

On Thursday, Google offshoot Jigsaw made available to developers an API called Perspective that aims to keep online trolling in check.

Perspective uses machine learning to determine the “toxicity” of online speech, thus allowing it to identify insults, harassment and abusive comments. As Wired highlights, the automatic moderation works much faster than a human moderator and is far more accurate than a simple keyword blacklist.

Those wanting to see Perspective in action are encouraged to check out the newly launched demonstration website. On it, you’ll be able to enter in phrases and observe how the tool rates them for toxicity.

A number of large publications including The New York Times are planning to use it as a first defense in their comments section. The Guardian and The Economist, meanwhile, are said to be already experimenting with the API to see how it might be able to improve their feedback platforms.

Of course, not everyone is onboard with this level of censorship. Feminist writer Sady Doyle told Wired last summer when the broader Conversation AI initiative launched that people need to be able to talk in whatever register they talk.

What are your thoughts on the matter? Should Google and others be working to clean up the trolling and hateful nature of comments sections or should this organic bit of conversation be left unaltered?

Lead image via Kirill M, Shutterstock

Permalink to story.

https://www.techspot.com/news/68280-google-api-detects-insults-harassment-abusive-comments-now.html

 

[CloudCatcher]

*comment deleted by Perspective API*

 

[yRaz]

Censorship, plain and simple. There is nothing to stop people from labeling criticism as an insult or abusive comment. We also have a ton of people who simply can't handle anything negative for whatever reason, none of them justified. See, we live in this world with so many other things we need to worry about, like going to work and paying bills. If someone tries to insult me or say something negative about me, I don't really care. Heck, what they're saying might even be true. The thing about it, though, is that I have this thing called priorities. I have to take care of myself before I can start to worry about other people.

 

[Toque]

Internet lost its fun! My youtube comments don't show up! It was fun while it lasted...FU GOOGLE!

 

[tonylukac]

I hope they put it on google play, because it's in a type of chaos.

 

[Evernessince]

Google isn't forcing this on anyone so I don't see a problem with this. If a company wants a better way to moderate their forums, I say let them have it.

It is pretty fun to play around with. For example, you can easily find what words or word combinations are currently associated with negative perceptions (what google labels "toxic"). For example, Typing in just "Obama" gives you a 6% match rate to negative comments and "Hillary" 7%. "Trump" on the other hand has 16% (and "donald trump" a 22%), more than double either Hillary or Obama. 16% of the time anyone mentions Trump, it is in a negative fashion. For comparison, "Bannon" has a 34% rate "Ajit Pai" a 30% rate and "putin" a nearly 50% rate.

 

[LordFox]

So how does blocking a persons voice not constitute a violation of free speech ?
Google is violating the law ..... what he said was offensive so we blocked it
This item sucks don't buy it .... we find this to be an alternate truth we blocked it
our government is harassing minorities similar to Germany ... we found this to be negative, BLOCKED .....

 

[Skidmarksdeluxe]

I hope they put it on google play, because it's in a type of chaos.

Google Play app comments are one thing. Have you ever tried reading through some of the comments on YouTube? Sometimes I can't believe that something loosely termed as "intelligent life" actually made them. And I'm not talking about the derogatory, insulting comments, they should be deleted by the mods. I'm talking about some of the ridiculous questions they asked.

 

[Mr Dude]

So how does blocking a persons voice not constitute a violation of free speech ?
Google is violating the law ..... what he said was offensive so we blocked it
This item sucks don't buy it .... we find this to be an alternate truth we blocked it
our government is harassing minorities similar to Germany ... we found this to be negative, BLOCKED .....

Pitchfork much?

Obscenity actually isn't protected by freedom of speech, among a number of other things. Such as good ol' "fighting words" and speech intentionally meant to cause offense.

I'm genuinely surprised by the number of people who don't know this.

In any case, the system should be alright if it allows you to view the content, at your own judgement.
For everyone else, it should prove a healthier experience.

That said, I don't think there's enough processing power in the solar system to actually keep this thing running in the internet.

 

[mailpup]

So how does blocking a persons voice not constitute a violation of free speech ?

Because strictly speaking the first amendment does not apply to private companies. Read it and see for yourself.

 

[Kibaruk]

Censorship, plain and simple... We also have a ton of people who simply can't handle anything negative for whatever reason, none of them justified.

My dear prince, telling someone that they are plain stupid is not just being negative, I think more than censorship is how you can contribute to a topic or thread or conversation without killing it. When you call someone stupid, you get called other things back, and then the thread is all about who can make the biggest insult -when you are 13 yr old talking to another 13 year old- until a mod comes in and breaks it lol

There are multiple and more subtle ways to call someone stupid without actually saying they are stupid.

 

[yRaz]

My dear prince, telling someone that they are plain stupid is not just being negative, I think more than censorship is how you can contribute to a topic or thread or conversation without killing it. When you call someone stupid, you get called other things back, and then the thread is all about who can make the biggest insult -when you are 13 yr old talking to another 13 year old- until a mod comes in and breaks it lol

There are multiple and more subtle ways to call someone stupid without actually saying they are stupid.

You're stupid

 

[Kibaruk]

You're stupid

Yo' mamma!! (The queen I presume)

 

[Evernessince]

So how does blocking a persons voice not constitute a violation of free speech ?
Google is violating the law ..... what he said was offensive so we blocked it
This item sucks don't buy it .... we find this to be an alternate truth we blocked it
our government is harassing minorities similar to Germany ... we found this to be negative, BLOCKED .....

Freedom of speech does not apply to private entities. They can do whatever they want with the services they provide. Freedom of speech only allows you to talk trash about the government and not be thrown in prison. Remember, our founding fathers wanted to live in a country where the people could openly criticize the government without persecution.

 

[yRaz]

Yo' mamma!! (The queen I presume)

My mom is transgender so I actually have two dads

 

[Camikazi]

So how does blocking a persons voice not constitute a violation of free speech ?
Google is violating the law ..... what he said was offensive so we blocked it
This item sucks don't buy it .... we find this to be an alternate truth we blocked it
our government is harassing minorities similar to Germany ... we found this to be negative, BLOCKED .....

They are a private company, they can do this, maybe they shouldn't but they are not violating any laws.

 

[yRaz]

They are a private company, they can do this, maybe they shouldn't but they are not violating any laws.

Aside from first amendment....

 

[tonylukac]

Google Play app comments are one thing. Have you ever tried reading through some of the comments on YouTube? Sometimes I can't believe that something loosely termed as "intelligent life" actually made them. And I'm not talking about the derogatory, insulting comments, they should be deleted by the mods. I'm talking about some of the ridiculous questions they asked.

I think some people are newly genetically engineered people starting from scratch and the brain was too complex. Joking but seems that way. Some people's children don't even look like them.

 

[Skidmarksdeluxe]

I think some people are newly genetically engineered people starting from scratch and the brain was too complex. Joking but seems that way. Some people's children don't even look like them.

True, true.

 

[captaincranky]

You're stupid

Yo' mamma!! (The queen I presume)

Yeah tho I walk through the valley of the shadow of the trolls, I will fear no insult. They're all friends of mine...(y)

 

[captaincranky]

My mom is transgender so I actually have two dads

@Kibaruk An' dey beez gunna kik yo azz....

 

[captaincranky]

@ Google Please don't take all this fun and camaraderie away from us, stay thy program..

 

[Yynxs]

The idea that Google is releasing this should give pause to anyone. They are already known for steering search results. Imagine adding the capability to steer conversation on everyone else's website.

Blizzard only allow 50 total trolls to be ignored in World of Warcraft because their target audience is 13 year olds and if you ignore 13 year olds they go elsewhere and don't pay the subscription fees. So, no way to ignore them off Blizz servers en masse. This shows in the creepiness of trade chat.

I've dealt with trolls across the net for years and the most wonderful thing to have is a userid 'ignore' button, especially one with unlimited ignores. No matter how many times the slimes spawn a new username, in short order they're off the forum posts to anyone that gets tired of them. I haven't see it, but it would be especially useful if the troll got an auto'ignored' count on their posts as then, it might be a teaching mechanism.

Do not be misled, Google is trying to put mechanisms in place to influence available information and behavior by not letting posts in. Trolls are 'paedophiles reviled by all' of the net. It's just their first step. Look for AI assisted 'intelligent post' filtering next, so you "really only have to read the important stuff".

 

[Camikazi]

Aside from first amendment....

The first amendment protects you from the government and allows you to talk negatively about them (and a few other things) without them arresting you, it actually does not protect you in any other way. We have elevated Freedom of Speech to a huge level but really there isn't all that much protecting a normal person's freedom of speech when it comes to private entities since they have their own rights. Basically, on a forum or on any site that is not owned by you, you do not have any freedom of speech, they control what can and can't be posted. Most companies will allow most types of speech since they, somewhat, respect freedom of speech but few will allow all types of talk, most will moderate and limit certain topics.

 

[yRaz]

The first amendment protects you from the government and allows you to talk negatively about them (and a few other things) without them arresting you, it actually does not protect you in any other way. We
have elevated Freedom of Speech to a huge level but really there isn't all that much protecting a normal person's freedom of speech when it comes to private entities since they have their own rights. Basically, on a forum or on any site that is not owned by you, you do not have any freedom of speech, they control what can and can't be posted. Most companies will allow most types of speech since they, somewhat, respect freedom of speech but few will allow all types of talk, most will moderate and limit certain topics.

Okay, so I'm assuming you read the rest of the thread so I'm going to quote one of the replies

@ Google Please don't take all this fun and camaraderie away from us, stay thy program..