Bad Cheese. Not good cheese, but the other kind.
Want to stop AI from crawling your website?
Initializing search
    • Home
      • Blog
        • 2025
        • 2024
        • 2023
        • 2022
        • 2020
        • 2019
        • 2018
        • 2016
    Back to index
    Steve Webb Steve Webb
    Creator
    • Metadata
      • April 3, 2024
      • 1 min read

    Want to stop AI from crawling your website?

    Pull this robots.txt into your own on occasion.

    NOTE: it's changing frequently, so pull often

    https://raw.githubusercontent.com/ai-robots-txt/ai.robots.txt/main/robots.txt

    Previous
    Too much (corporate) MFA
    Next
    Are you replacing an old Broan NuTone 696 b bathroom ventilation fan?
    Copyright © 1998 - 2024 Steven Webb – Change cookie settings
    Made with Material for MkDocs

    Cookie consent

    We use cookies to recognize your repeated visits and preferences, as well as to measure the effectiveness of our documentation and whether users find what they're searching for. With your consent, you're helping us to make our documentation better.