WordPress robots.txt Generator
The right robots.txt for a WordPress site, with the admin folder handled correctly.
Works in your browser. Nothing you type is sent to us.
Paths for every crawler
One path per line, starting with /. A path matches everything that begins with it; * matches anything and $ marks the end.
AI crawlers
Crawlers that collect pages to train or feed AI models. Blocking them does not affect your search ranking.
Sitemap
Ignored by Google; some other crawlers honour it.
Put the file at the root of the site, so it is reachable at https://www.example.com/robots.txt.
Written in your browser. Nothing you type is sent to us.
WordPress robots.txt Generator at a glance
- Price
- Free
- Data sent to us
- None, it runs in your browser
- Sign-up
- Not needed
- Follows
- Google's robots.txt specification
- AI crawlers
- OpenAI, Anthropic, Google-Extended, Common Crawl and more
- Output
- robots.txt to download or copy
How to use WordPress robots.txt Generator
- Type the site address and pick a startThe sitemap line fills in from the address. Choose WordPress, online shop, allow everything or block the whole site as the starting rules.
- Add paths and AI crawlersList the paths to keep crawlers out of, one per line, and tick the AI crawlers to block, or press Block all of these.
- Download the filePress Download robots.txt and put it in the root folder of the site, so it opens at /robots.txt.
This page starts with the rules a WordPress site needs: crawlers kept out of /wp-admin/ but allowed to fetch admin-ajax.php, which themes and plugins call from public pages. Add your site address so the sitemap line points at the XML sitemap WordPress or your SEO plugin produces, add any other paths, tick AI crawlers to block and download. Upload the file to the folder that holds wp-config.php, or paste it into your SEO plugin's robots.txt editor.
How it works
WordPress robots.txt Generator is a web page that does its work on your own device. Everything you type or choose is worked out by your browser and never sent to a server, so nothing is stored and there is nothing to delete. There is no account, no sign-up and no limit on how often you use it.
Where WordPress keeps robots.txt
WordPress serves a virtual robots.txt when no file exists. A real file in the site root replaces it, and SEO plugins such as the common ones offer an editor that writes the same file for you.
Questions
Should I block /wp-content/ or /wp-includes/?
No. Those folders hold the CSS, scripts and images Google needs to render pages; blocking them can hurt how pages are understood. Block only /wp-admin/, with the admin-ajax.php exception.
Other shortcuts
robots.txt Generator · Block AI Crawlers
More
More Web and SEO tools
See all →
Color Converter
Web and SEO ToolsEvery spelling of a colour at once, with a lightness scale, a contrast check and colours from a picture.
UTM Link Builder
Web and SEO ToolsTag a link with utm_source, medium, campaign and the rest, consistently, every time.
Meta Tag Generator
Web and SEO ToolsEvery head tag a page needs, with the search result and the shared link card previewed as you type.