Can AI assistants read your website?
Free check. Enter a page and see whether the crawlers behind ChatGPT, Claude, Perplexity, Google's AI answers and Copilot can find and read it, and what to change if they can't.
No sign-up. We don't store the address you check. Results are kept for 10 minutes so repeat checks are quick.
What this check looks at
- robots.txt: whether your rules allow each AI crawler to read the page. We read the copy your server gives each crawler, because some servers answer each one differently.
- Your server and firewall: we request the page once as a normal browser and once under each crawler's name. If the browser gets the page and a crawler doesn't, something is blocking that crawler.
- noindex: whether the page tells search engines not to index it. AI search features only quote pages that may be indexed.
Which crawlers matter for being found by AI
| Assistant | Finds pages with | Opens a page for a user with | Trains models with |
|---|---|---|---|
| ChatGPT | OAI-SearchBot | ChatGPT-User | GPTBot |
| Claude | Claude-SearchBot | Claude-User | ClaudeBot |
| Perplexity | PerplexityBot | Perplexity-User | – |
| Google AI Overviews and AI Mode | Googlebot | – | Google-Extended (Gemini) |
| Copilot (and ChatGPT search, through Bing) | Bingbot | – | – |
Search crawlers decide whether an assistant can find and quote your page. OpenAI says sites that opt out of OAI-SearchBot will not be shown in ChatGPT search answers, though they can still appear as navigational links. User crawlers open a page when someone asks about it; OpenAI and Perplexity say theirs may not follow robots.txt, so for those two the check shows a robots.txt rule but doesn't count it as a block. Training crawlers only decide whether your pages may be used to train AI models; blocking them is your choice and doesn't stop assistants from citing you.
Why a site blocks AI crawlers without meaning to
- A CDN or host setting that blocks AI bots, such as Cloudflare's "Block AI bots" option.
- A security plugin or firewall rule that refuses unfamiliar crawlers.
- A robots.txt copied from a template that disallows every crawler except Google.
How to let AI search crawlers in
Add these lines to the robots.txt at the root of your site (yourbusiness.com/robots.txt), then run the check again:
User-agent: OAI-SearchBot User-agent: ChatGPT-User User-agent: Claude-SearchBot User-agent: Claude-User User-agent: PerplexityBot User-agent: Perplexity-User Allow: /
If your site is behind Cloudflare, also open Security → Settings → Bot traffic in Cloudflare and check that "Block AI bots" is off.
What this check can't see
Our requests carry each crawler's name but come from our server, not from the crawler's own addresses. Some firewalls check real crawlers by their IP addresses and treat them differently from any imitation. When a site does that, the check says "can't test from outside" instead of guessing.
For AI agents and developers
The same check is free to use from code and from AI agents, with the same limits (10 checks an hour per caller, results cached for 10 minutes):
- MCP server:
https://oooworks.com/mcp(streamable HTTP, no sign-in), toolcheck_ai_crawler_access. In Claude Code:claude mcp add --transport http oooworks https://oooworks.com/mcp - JSON API:
POST https://oooworks.com/api/ai-crawler-checkwith{"url": "example.com"}. Description: OpenAPI.
Read more
- How to check if ChatGPT can see your website (and fix it)
- Nearly 1 in 5 top websites block AI search crawlers
Who made this
OOO Works, which is building done-for-you services for small businesses worldwide, including OOO Visibility: it tracks whether AI assistants name a business for the questions its customers ask, and fixes what keeps it out. OOO Works is opening soon; nothing is on sale yet. About OOO Works.