# Capture AI crawlers on Fastly.

> Connect Fastly to Virent AI Crawler Analytics. Stream request logs from a Fastly CDN service to Virent's authenticated HTTPS endpoint.

Canonical: https://virent.app/docs/integrations/fastly
Last updated: 2026-10-19

Stream request logs from a Fastly CDN service to Virent's authenticated HTTPS endpoint.

An HTTPS logging endpoint on your Fastly service streams one JSON line per request to Virent, including cached responses.

You need your site ID and server secret from **Settings → Connection** (see [Hosting integrations](/docs/integrations/overview#before-you-start)). In the dashboard, **Connect analytics → Hosting integrations → Fastly** shows these steps with your site ID, domain and secret filled in. Below, replace `<your-site-id>`, `<paste-your-server-secret>` and `example.com` with your own values.

### Set up with your coding agent

Paste this prompt into Claude Code, Codex or another coding agent. Replace the site ID and example.com with your own.

```text
Configure Fastly so example.com sends reliable AI crawler request telemetry to Virent.

Site ID: <your-site-id>
Use the full Virent server secret from the Connect dialog. Do not use the masked preview "vha_sk_••••" as the real value.

Use the current provider documentation: https://www.fastly.com/documentation/guides/integrations/logging-endpoints/protocol-based-and-self-hosted/log-streaming-https/

Steps:
1. Create an HTTPS endpoint: On the Fastly CDN service, add an HTTPS logging endpoint. Virent answers Fastly’s ownership validation challenge at this destination.

https://virent.app/api/ingest/fastly

2. Authenticate delivery: Set Content type to application/x-ndjson and configure this custom header. Keep Placement on Format Version Default.

x-api-key: <paste-your-server-secret>

3. Set the request log format: Paste this exact log format. It emits one Virent-compatible JSON object per request and safely escapes request headers.

{"timestamp":"%{begin:%Y-%m-%dT%H:%M:%SZ}t","status_code":%>s,"request_method":"%m","request_path":"%{json.escape(req.url.path)}V","query_string":"%{json.escape(req.url.qs)}V","content_type":"%{json.escape(resp.http.Content-Type)}V","client_ip":"%a","hostname":"%{json.escape(req.http.host)}V","user_agent":"%{json.escape(req.http.User-Agent)}V","referrer":"%{json.escape(req.http.Referer)}V"}

4. Activate and verify: Create the endpoint, activate the service version on Production, send an AI crawler request to example.com, and verify it in Virent.

Keep the secret server-side, preserve 100% request coverage where the provider supports sampling, and verify both normal site delivery and a crawler request after activation.
```

## Steps

1. **Create an HTTPS endpoint**: On the Fastly CDN service, add an HTTPS logging endpoint. Virent answers Fastly’s ownership validation challenge at this destination.

   ```text
   https://virent.app/api/ingest/fastly
   ```

2. **Authenticate delivery**: Set Content type to application/x-ndjson and configure this custom header. Keep Placement on Format Version Default.

   ```text
   x-api-key: <paste-your-server-secret>
   ```

3. **Set the request log format**: Paste this exact log format. It emits one Virent-compatible JSON object per request and safely escapes request headers.

   ```text
   {"timestamp":"%{begin:%Y-%m-%dT%H:%M:%SZ}t","status_code":%>s,"request_method":"%m","request_path":"%{json.escape(req.url.path)}V","query_string":"%{json.escape(req.url.qs)}V","content_type":"%{json.escape(resp.http.Content-Type)}V","client_ip":"%a","hostname":"%{json.escape(req.http.host)}V","user_agent":"%{json.escape(req.http.User-Agent)}V","referrer":"%{json.escape(req.http.Referer)}V"}
   ```

4. **Activate and verify**: Create the endpoint, activate the service version on Production, send an AI crawler request to example.com, and verify it in Virent.

Check Fastly's [current documentation](https://www.fastly.com/documentation/guides/integrations/logging-endpoints/protocol-based-and-self-hosted/log-streaming-https/) if its screens differ from these steps.

## Verify

Send a [test crawler request](/docs/analytics/ai-crawlers#verify) to your site. The setup dialog in Virent shows "Virent is receiving AI crawler traffic" once the first request arrives.