AI bots are now a new category of network users and a source of traffic, and bans do not hinder them

Artificial intelligence already controls a significant portion of web traffic

According to the latest report from Akamai, published in *Wired*, AI-based bots are taking an increasingly larger share of site visits. The report describes how blocking data from such bots and their sophisticated actions turn into a real “arms race” on the internet.

What’s happening
- New types of visitors

Experts note that AI bots do not just copy content – they become a new class of users. Owners of large portals are trying to limit access to materials that bots can collect and use to train their models.

- Real‑time data collection

Some bots pull information from the network instantly and use it to improve their own services: current product prices, movie theater schedules, short news summaries, etc.

Growth statistics
Period Share of bot visits
1–3 March 2025 1 out of 200 visits
IV quarter 2025 1 out of 50 visitors

- Violations of robots.txt

More than 13 % of bots ignore the directives in *robots.txt* files. From QII to QIV such behavior increased by 400 %, while administrators’ attempts to block them rose only by 336 %.

- Masking

Some bots spoof headers and imitate ordinary browser requests to bypass protection.

Services and solutions
Company What it offers
TollBit Platform for contracting between AI developers and publishers, allowing payment for content.
Cloudflare Bot‑blocking tools created by unscrupulous developers.
Bright Data Claim that their bots collect only publicly available information.
ScrapingBee Emphasize the principle of an open internet for people and machines.
Oxylabs Specialize in data collection; claim that site protection does not distinguish between good‑faith and malicious traffic.

There are many legitimate purposes for collecting materials: cybersecurity, investigative journalism, and other tasks. However, most defensive mechanisms are still applied uniformly across all scenarios.

Market trends
- More than 40 companies now offer bots for gathering data needed to train AI.

- AI search engines and agents such as OpenClaw are becoming active.

- Some firms already help companies find materials for AI agents and develop tools to optimize content for machine learning.

These developments foreshadow the emergence of a “full‑blown marketing channel,” where search, advertising, media, and commerce will be integrated into a single ecosystem.