Robots.txt Tester
Paste your robots.txt, then the address you want to test. The tool applies the same matching rules Google documents and tells you which line decided.
Fill in the fields above and the result shows here as you type.
Good to know
- To see your file, open yourdomain.ca/robots.txt in a browser and copy everything.
- When more than one line matches, the longest one wins. If an Allow and a Disallow are the same length, Allow wins.
- If a crawler has its own section, it follows only that section and ignores the general * rules.
This tool runs in your browser. What you type is not sent anywhere unless you ask for your report by email.
Get every result in one email.
Each tool you use adds its result to this list. When you are done, enter your email and I will send the lot as one report you can keep, forward or hand to whoever looks after your website.
Your results stay in this browser until you press send. The report is sent once. Your email goes to me and nowhere else. Privacy
Nothing here yet. Use any tool and its result is added automatically.
Rather have it done for you?
These tools show you what to fix. If you would rather hand it over, tell me what is not working and I will send back a working piece of the fix.