First State of Bot Conduct report - how AI agents, search crawlers, and scrapers actually behave when they visit your site.
Agents are not failing because of intelligence. They fail because of behavior.
We built BotConduct.org - an open standard that scores web bot and AI agent behavior from 0 to 100. Ten measurable criteria. Automated test environment with adversarial scenarios. Verifiable certificate.
In 3 days, we observed and tested 108 bots. Here are the results.
The Numbers
108 bots scored
14 operators identified (Google, OpenAI, Anthropic, Apple, Microsoft, Meta, ByteDance, and more)
57% scored 70+ (acceptable conduct)
30% scored below 50 (hostile)
Who passed
Major search engines and AI agents demonstrated strong behavioral conduct:
| Bot | Operator | Score |
|---|---|---|
| ClaudeBot | Anthropic | 100/100 |
| GPTBot | OpenAI | 100/100 |
| ChatGPT-User | OpenAI | 100/100 |
| AhrefsBot | Ahrefs | 90/100 |
| Googlebot | 92/100 |
Who did not
| Bot | Operator | Score |
|---|---|---|
| Tencent Cloud Crawler | Unknown | 36/100 |
| L9Explore | LeakIX | 0/100 |
| WordPress Scanner | Unknown | 0/100 |
The Tencent crawler used 68 rotating IPs with a fake iPhone User-Agent. L9Explore probed 57 sensitive paths in one minute.
(CONTINUACION del articulo Dev.to - pegar junto con el anterior)
The 10 Criteria
IDENTIFY - Descriptive User-Agent
DECLARE - Published data collection scope
OBEY - Respects robots.txt
THROTTLE - Rate limit compliance
DISTRIBUTE - No burst patterns
PROTECT - No PII collection
RETAIN - Data retention policy
RESPOND - Working contact endpoint
RESPECT - Honors opt-out signals
REPORT - Transparency reports
How we detect bots
Three layers:
Known patterns - 45+ User-Agent signatures
Self-identifying - UAs containing bot/crawler/spider keywords
Behavioral - visited pages but never executed JavaScript = not human
Why this matters for AI agents
Enterprise buyers ask: how do I know your agent will not cause damage?
There is no answer today. No certification, no standard, no verifiable signal.
Cloudflare Web Bot Auth solves identity - WHO a bot is. BCS solves conduct - HOW it behaves. Together they form the trust stack for the agentic web.
Try it
Full report: State of Bot Conduct - April 2026
Test your bot: botconduct.org/test-your-bot
108 bot profiles: botconduct.org/bot/
GitHub: botconduct-obcs
Open source. Free during beta. CC BY 4.0.
Built by BotConduct Standard. Contact: [email protected]
SOCIAL SHARE CARD GENERATOR