# Everything is crawlable. The named groups below grant nothing the wildcard Allow does not
# already grant: they are here so the decision is written down, and so the next person who
# reaches for a blanket Disallow to stop a scraper has to delete a line saying these crawlers
# are welcome rather than cutting off retrieval by accident.
#
# READ THIS BEFORE ADDING A Disallow. robots.txt groups DO NOT INHERIT. A crawler obeys only the
# most specific group matching its name (RFC 9309), so a Disallow added to "User-agent: *" below
# would NOT apply to any agent named in the second group: GPTBot would keep crawling the path you
# just blocked, silently. Any Disallow has to be added to BOTH groups.
# server/geoSurfaces.test.ts fails the build if the two groups ever disagree.

User-agent: *
Allow: /

# Answer engines and assistant crawlers, named explicitly. Keep the rules below identical to the
# wildcard group's.
User-agent: GPTBot
User-agent: OAI-SearchBot
User-agent: ChatGPT-User
User-agent: ClaudeBot
User-agent: Claude-User
User-agent: Claude-SearchBot
User-agent: PerplexityBot
User-agent: Perplexity-User
User-agent: Google-Extended
User-agent: Applebot-Extended
User-agent: Bingbot
User-agent: CCBot
User-agent: meta-externalagent
Allow: /

Sitemap: https://sundaysurvivor.com/sitemap.xml