Agent Analytics

See how AI crawlers and assistants access your site

Open Insights → Onsite → Agent Analytics to see how AI systems reach your site: which engines crawl your content, whether they're crawling for search or training, and when an AI assistant sends a real person to a page. Agent Analytics draws on your CDN request logs, so there's no analytics script to install.

Agent Analytics is available once you connect a supported CDN or hosting provider.

How to Configure

1. Choose the guide for your provider:

   • Cloudflare Logpush
   • Cloudflare Worker
   • CloudFront
   • Netlify
   • Vercel
2. Complete the connection steps in the provider guide.
3. Return to Insights → Onsite → Agent Analytics to review data as requests arrive.

Core Metrics

Metric	What it counts
Total Crawls	Every automated AI crawl of your site during the selected period (web search and training crawls combined). A higher number means AI systems are fetching more of your content.
Web Search Crawls	Crawls from AI search engines fetching a page to ground a live answer, such as ChatGPT search and Perplexity. These track content being pulled into real-time AI answers.
Training Crawls	Crawls that collect your content to train AI models, such as GPTBot and ClaudeBot.
AI Agent Visits	Visits from an AI assistant's browser acting on behalf of a person, for example when someone in ChatGPT or Claude follows a link to your site. Each visit is one request, counted when it happens.
Top Engine	The AI engine that crawled your site the most during the selected period.

Charts

Chart	What it shows
Crawl activity vs. AI agent visits	How crawling and AI agent visits move over time, so you can connect a rise in crawls to the visits that follow.
Engine breakdown	Training, web search, total crawls, and AI agent visits for each AI engine, so you can see which platforms are most active on your site.

Per-page metrics

These metrics are also available for each URL in the Pages view. Every page row shows its crawl and AI-agent-visit counts, and opening a page shows the same metrics and trends for that single URL.

Filters

• Date: Select a date range for your analysis
• Data sources: Choose which connected CDNs to include