AI Crawler Policy

Speedy Geek Services wants its public website content to be discoverable by search engines, answer engines, and AI-assisted search tools. Our current robots.txt policy allows major search, retrieval, user-request, and training crawlers to access public pages.

Allowed AI Crawlers

Our robots.txt file currently allows GPTBot, OAI-SearchBot, ClaudeBot, Claude-SearchBot, Claude-User, PerplexityBot, Perplexity-User, and Google-Extended. These crawler directives may be updated as crawler standards and business needs change.

Training and Retrieval Are Separate

Some AI crawlers are used for model training, while others are used for search, retrieval, or user-requested page access. We treat these as separate policy decisions and review them independently.

Public Content Only

This policy applies to public website pages. It does not authorize access to private systems, admin areas, unpublished client materials, credentials, forms beyond ordinary submission workflows, or non-public data.

Contact

Questions about crawler access can be sent to speedygeekservices@gmail.com.