# AI traffic

> How we see the AI crawlers reading your site and the people assistants send you, and what data leaves your site.

The **AI traffic** pages answer two questions: **which AI crawlers read your site** (and what they get back) and **how
many people AI assistants send you**. The data comes from your own site, through an integration you install once.

## Why it needs a server integration

AI crawlers (GPTBot, ClaudeBot, PerplexityBot, OAI-SearchBot…) **do not run JavaScript**. A script in the page, like
Google Analytics or our beacon, never sees them. Only the server answering them does, so that is where the
integration runs:

| Platform | What you install | Sees AI crawlers | Sees people from AI |
|---|---|---|---|
| [WordPress](https://survelium.com/docs/ai-traffic/wordpress.md) | the Survelium plugin | yes | yes |
| [Laravel](https://survelium.com/docs/ai-traffic/laravel.md) | the Composer package | yes | yes |
| [OpenCart](https://survelium.com/docs/ai-traffic/opencart.md) | a folder next to `index.php` | yes | yes |
| [Any other PHP site](https://survelium.com/docs/ai-traffic/php.md) | the PHP core | yes | yes |
| [Shopify, Wix, Webflow, static sites](https://survelium.com/docs/ai-traffic/other-sites.md) | the JavaScript beacon | **no** | yes |

Every package is open source (MIT licence), at [github.com/survelium](https://github.com/survelium).

## How the data reaches us

1. You create a key in **Brand settings → Integrations** and put it in the integration's settings. The key ties the site
   to the brand.
2. On every page view the integration looks at the **user agent** (who is asking for the page) and the **referrer**
   (where the visitor came from), **after** the page has gone out to the visitor. Nobody waits on it.
3. Matches go into a local queue, sent to us in batches every few minutes.
4. We match every hit against our crawler list again and [verify its IP](https://survelium.com/docs/ai-traffic/crawlers.md#verification)
   against the ranges the operator publishes.

If the site cannot reach us (network, revoked key, plan without AI traffic), the data stays queued and goes out
later. Hits older than 7 days are no longer accepted.

## What gets recorded

| | Sent | Never sent |
|---|---|---|
| AI crawler hit | time, user agent, page path, status code, IP (to verify it is really that crawler) | — |
| Visitor from an AI assistant | time, assistant domain (e.g. `chatgpt.com`), page path, user agent | IP, full referrer, cookies |
| Every other page view | one total per day, for the [AI traffic share](https://survelium.com/docs/ai-traffic/traffic-share.md) | anything about the visit |

Crawler IPs are deleted after **30 days**. We store no IPs of people.

## The AI traffic pages

- **Overview** — [crawler hits](https://survelium.com/docs/ai-traffic/crawler-hits.md) by purpose, [people from AI](https://survelium.com/docs/ai-traffic/people-from-ai.md)
  and the [AI traffic share](https://survelium.com/docs/ai-traffic/traffic-share.md), with the change against the previous period.
- **Pages** — every page of the site: who read it, how many people landed on it and [what errors crawlers got](https://survelium.com/docs/ai-traffic/pages.md).
- **AI crawlers** — every [crawler](https://survelium.com/docs/ai-traffic/crawlers.md), with its request log.
- **Referrals** — people sent by assistants, per day, per assistant and per landing page, with a
  [CSV export](https://survelium.com/docs/ai-traffic/people-from-ai.md#csv-export).
- **Integration** — keys and install instructions. The "Connected" badge shows when data last arrived.

## Rules on every page

- **Days are UTC days**, like the daily total your site sends. Charts can be a few hours off your local time.
- **Only verified crawlers count by default.** The "Include unverifiable" switch adds the crawlers that cannot be
  verified. Requests pretending to be another crawler never count.
- **There is no model filter**: crawlers and visitors are not tied to the models your prompts monitor.
- The period (7, 28 or 90 days) is compared with the same length of time just before it.