Free · reads the live site · no signup

Robots.txt checker

Fetch the live robots.txt and see whether the crawlers that train and answer with AI are allowed to read the site.

Enter a page or site URL — for example https://example.com

Score: /100

for

for 


            

        

    


    

    

        
How it works

        
Which crawlers does this check?

        
GPTBot, OAI-SearchBot, ChatGPT-User, Googlebot, Google-Extended, ClaudeBot, PerplexityBot, CCBot, Bytespider, Amazonbot, and Applebot-Extended. A matching group wins over the wildcard group.

        
                

            
Questions people ask

                        

                Is this tool free?
                
Yes. It checks the live URL you enter, with no signup, and nothing you submit is stored.

            

                        

                Does it run JavaScript like a browser?
                
No. It reads the HTML the server returns, the same way a search or AI crawler does. Content that appears only after client-side JavaScript runs will not be seen — which is exactly what you want to test for.

            

                    

        
                

                        AI Visibility Checker
                        llms.txt Generator
                        AEO Grader
                        Schema Analyzer
                    

            





    
    

        

            
Ranking in AI answers starts with your own data.

            
isend connects Google Search Console to show the queries you already rank for, the gaps to close, and AI keyword ideas — then helps you ship the content that gets cited.

            

                Open the SEO dashboard
                All free SEO tools