In the fight against online extremism, most companies rely on their users to flag up inappropriate content, which is then reviewed and usually deleted by employees. But according to Reuters, big internet firms such as Facebook and Google have secretly introduced an automated process for removing extremist material.
It's claimed the process uses technology that was originially developed to identify and remove copyrighted material. It works by comparing “hashes” - unique digital identifiers that companies automatically assign to specific videos - against a database of previously banned content. A similar technique has already been used to detect online images of child abuse.
It isn’t clear what level of automation the system can operate at, or exactly how the banned content database is put together. Reuters said the companies that use the process are reluctant to discuss it due to concerns that terrorists may learn to manipulate the system.
“There's no upside in these companies talking about it,” Matthew Prince, chief executive of content distribution company CloudFlare, told Reuters. “Why would they brag about censorship?”
While the process stops the reposting of content that has already been banned, it is unable to identify new extremist material. And, as is often the case when extremism is concerned, there’s the issue of whether it should be down to a company to decide what constitutes offensive content and what falls under free speech.
“It’s a little bit different than copyright or child pornography, where things are very clearly illegal,” said Seamus Hughes, deputy director of George Washington University’s Program on Extremism.
As terrorist organizations such as ISIS continue to use the web as an effective propaganda and recruitment tool, more companies are pushing back against the practice. Facebook and Twitter remove terrorist-related accounts as quickly as they are created, and Microsoft announced last month that it had officially banned all “terrorist content” from its consumer services, including Outlook, Xbox Live, and Docs.
