Source. Every page request to mojodojo.io whose user agent looks like a crawler, logged at the web server with the visitor's IP. One site, so read the numbers as a case study rather than a census. Tracking since 6 June 2026. IPs were recorded from 7 Oct 2026; hits before that are classified by user agent only and counted as unverified.

Categories. AI training bots collect pages for model training (GPTBot, ClaudeBot, CCBot). AI search bots build the index behind AI answers (OAI-SearchBot, Claude-SearchBot, PerplexityBot). AI user fetches are live reads on behalf of a person (ChatGPT-User, Claude-User, Perplexity-User).

Verification. A hit counts as verified when its IP sits inside the ranges the operator publishes (OpenAI, Anthropic, Perplexity, Google, Bing, Apple, Common Crawl, DuckDuckGo, Ahrefs), or passes forward-confirmed reverse DNS. Ranges refresh daily. A hit that fails the check is marked spoofed and left out. Bots whose operator publishes no method are counted as unverifiable.

Cite this. Mojo Dojo, “AI Crawler Index”, mojodojo.io/ai-crawler-index, accessed 8 Oct 2026. Free to reuse under CC BY 4.0 with a link back.