Free robots.txt generator

Pick whether AI crawlers may read your site and where your sitemap lives. The tool writes a robots.txt you upload to the root of your domain.

Sitemap address

Paths to keep private (one per line)

Let AI answer engines read the site (GPTBot, ClaudeBot, PerplexityBot and others)

Your robots.txt

User-agent: *
Allow: /

User-agent: GPTBot
Allow: /

User-agent: OAI-SearchBot
Allow: /

User-agent: ChatGPT-User
Allow: /

User-agent: ClaudeBot
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: Google-Extended
Allow: /

User-agent: Applebot-Extended
Allow: /

User-agent: CCBot
Allow: /

Runs in your browser. Nothing you type is sent to yaya.

Skip the busywork

Sites built on yaya ship these files and tags for you, and keep them current every time you publish.

Build my site free