# ZippyGo — https://zippygo.ai # # ── POLICY (decided 2026-09-29): AEO ON, SCRAPING OFF ─────────────────────── # Search engines and AI assistants, including AI crawlers, may read and cite # the public marketing pages: home, about, blog, connector, download and the # legal pages. That is what answer-engine optimisation (AEO) needs. # # No crawler may read the booking funnel or the API. Those pages are # per-session, and /api/ is the live fare-search surface. # # ── MAINTENANCE NOTES (read before editing) ───────────────────────────────── # # 1. ONE GROUP FOR EVERYONE, ON PURPOSE. A crawler obeys only the MOST # SPECIFIC group naming it. Adding "User-agent: GPTBot / Allow: /" would # let GPTBot ignore every Disallow below, including /api/. If a bot ever # needs its own group, copy all of these Disallow lines into it. # GPTBot, ClaudeBot, OAI-SearchBot, ChatGPT-User, Claude-SearchBot, # PerplexityBot, Google-Extended, Applebot-Extended, CCBot, Amazonbot, # Googlebot, Bingbot and the rest all fall under "User-agent: *". # # 2. NEVER add "Disallow: /assets/". This is a client-rendered SPA: crawlers # must fetch and run /assets/index-*.js to see any content at all. # # 3. This file is a REQUEST, not a control. The API enforces its own rules # whatever this file says: crawler and scripting-tool user agents get 403 # on /api/*, fare routes need browser headers, and each address is # rate-limited (functions/_shared/abuseGuard.ts). The Cloudflare WAF # rate-limiting rule is the edge layer (docs/hosting.md). # # 4. The Terms of Use (/terms-of-use, "Reservation of Rights") still reserve # text-and-data-mining and AI-training rights. That text is owned by # legal/management, and keeping it consistent with this file is their call. User-agent: * Content-Signal: search=yes, ai-input=yes, ai-train=yes Allow: / Disallow: /api/ Disallow: /flights Disallow: /search Disallow: /book/ Disallow: /offer/ Disallow: /checkout Disallow: /trips Disallow: /account Disallow: /resume Disallow: /unsubscribe Sitemap: https://zippygo.ai/sitemap.xml