robots.txt Generator

Generate robots.txt files with custom rules.

Rule 1

User-Agent

Crawl-Delay (seconds)

Disallow paths (one per line)

Allow paths (one per line)

Sitemap URL

Output

User-agent: *
Disallow: /