Prevent ChatGPT Bot From Crawling Your Website

To disallow (or allow) ChatGPT from crawling your website, you can add directives in your robots.txt file for the following user-agents:

For instance, if you wish to prevent both, OpenAI's web crawler and ChatGPT plugins from accessing your website entirely, you can add these directives to your robots.txt file:

User-agent: GPTBot
Disallow: /

User-agent: ChatGPT-User
Disallow: /

This will instruct the bots not to crawl your entire website.
Alternatively, you can restrict access to specific portions
of your website using directives like these:

User-agent: GPTBot
Allow: /directory-1/
Disallow: /directory-2/

It's important to note that in the absence of a Disallow directive, it's generally assumed that a user-agent is allowed to access a section of the website. The purpose of the Allow directive is to explicitly state that a specific directory is accessible to a user-agent, particularly when there might be conflicting rules or a need for clear access permissions.


Need some help? Schedule a call together.