Unblock the crawlers
- Trigger
- You have never checked your robots.txt against the AI user agents.
- Move
- Allow GPTBot, ClaudeBot, PerplexityBot, Google-Extended and OAI-SearchBot explicitly.
- Why it works
- If the crawler is blocked, the engine describes you from whatever it finds elsewhere, which is usually a competitor's comparison page.