Almost half of all web traffic is bots, and they are mostly malicious in nature

[Alfonso Maruccia]

In a nutshell: A recent report by Akamai estimates that 42 percent of overall web traffic is generated by bots, with 65 percent of these bots having clearly malicious intent. Akamai's study primarily focuses on "scraping bots," which are designed to harvest data and content for use in cyber-crime operations. These bots can significantly impact web-based businesses, and AI algorithms are making life easier for hackers and criminals.

As one of the largest content delivery networks on the market alongside the likes of Cloudflare and Amazon AWS, Akamai knows pretty well what's happening on the internet right now. And the internet as we know it is mostly bots.

That is one of Akamai's conclusions according to a 2024 report on web scraping that is further supported by similar numbers seen in 2022 and 2023 reports by different sources. Akamai reports that the e-commerce sector has been most affected by scraping and "high-risk" bot traffic. While some types of bots can benefit businesses, web scraper bots typically have a negative impact on both the bottom line and the overall customer experience. Currently, web scraping bots are used for competitive intelligence, espionage, inventory hoarding, scam site creation, and other criminal activities.

Patrick Sullivan, Akamai's CTO, explains that bots pose significant challenges, creating "multiple pain points" for web apps, services, and API owners. Scraping bots can easily obtain product images, descriptions, pricing information, and other data. Cyber-criminal gangs can then exploit this data to create fake websites that impersonate well-known brands or retail services.

It happens to content websites as well like TechSpot and many others, when bots are used to scrape content, spruce some slightly changed text (made easier today using Gen AI) and replicate our content without permission for the purpose of ranking on Google, stealing traffic, and making money through ads.

The scraper landscape is evolving due to AI. Bots using artificial intelligence algorithms are becoming much harder to detect. AI botnets can work well even with unstructured data, Akamai says, and they can employ actual business intelligence to provide scammers with a much refined "decision-making process." Thanks to AI, criminals can collect, extract, and process data more efficiently than ever before.

Bots are also useful to create new fake accounts, which can then be used to target real people and their finances. Even non-malicious scraping bots can degrade a website's performance, impact search engine metrics, and increase computing and hosting costs.

Companies now face increasingly sophisticated bots that use AI algorithms, headless browser technology, and other advanced solutions. These new threats require novel, more complex mitigation approaches beyond traditional methods. A robust firewall is now only the beginning of the numerous security measures needed by website owners today.

Permalink to story:

Almost half of all web traffic is bots, and they are mostly malicious in nature

 

[tonymontana1984]

Yep...they take bus and train seats in world wide web and they simply don't care about real people...it's disgusting...the day they get into uber or taxis the Internet we know it will be mo more!

 

[Underdog]

Isn't AI wonderful!

 

[VitalyT]

Wait until the day when AI will be manufacturing the entire Internet content to fit your profile. That day is almost here. The entire internet will become pure garbage, much more than it already is. It will be like The Matrix, but just for the Internet.

 

[Trapped Nowhere]

The bot farms that are rampant on social media seem to run about unchecked and constantly throw poop until something makes it into the larger followers to influence individuals and politicians, even as recent as this week

 

[Uncle Al]

It should be possible (in my limited opinion) to require all bots be registered and identifiable by code. All others should be blocked and their producers arrested and charged with the appropriate crimes. Enforcement would be difficult BUT it should be possible to trace backwards their origins and if nothing else, block out that IP address forever .....

 

[Dr Roboto]

So, is there anything that can be done about this? Are these bots "illegal" or is that even a thing on the WWW? Think of the resources these "bots" use up. The networking bandwidth needs to be 2x or larger than needed, more servers, more hardware, more everything just to support non-human bots, and for no meaningful gain to society. Pretty much everything on the internet now a days sucks with the bombardment of ads, fake pages, trolling, scams, etc.. It will probably become completely useless in 10-20 years.

 

[Ecurb]

Every time I pull up a web page on my browser I've technically 'scraped' that page. That somebody thinks my web site has information of interest and so accesses it -- isn't that what I wanted?

The notion that someone can only use my web page in a manner I specify "please read the article and click on my ads only' is humourous.