top of page
This free tool will help you generate an advanced rotot.txt file for your website. The foll support popular agents like Google, Bing, and DuckDuckGo, along with social media platforms like Facebook and Twitter.
You can also add your own custom user agents. The tool also supports a sitemap.
In advanced settings, you can add crawl delays, host directives, and custom directives to your robot.txt file. The tool is very easy to use with a minimal UI with maximum features.
bottom of page