ContentSignalBot
ContentSignalBot fetches a single URL only when a Content Signal customer or a visitor using the free page checker explicitly requests it. It does not crawl your site on its own or follow links.
User-Agent string
ContentSignalBot/1.0 (+https://getcontentsignal.com/bot)
Every request this crawler makes carries this exact User-Agent header, so it can be matched precisely in logs, firewalls, and robots.txt.
What it does
ContentSignalBot performs a single, on-demand fetch of one URL at a time: either a page a Content Signal customer is monitoring, or a URL someone submits to the free page checker at /tools/page-checker. It reads the response once, extracts basic on-page signals (title, meta description, canonical, meta robots, presence of structured data), and discards the raw HTML afterward. It does not crawl beyond the requested URL, does not follow internal links to discover new pages, and does not run on a recurring schedule beyond what a customer has explicitly configured for their own monitored URLs.
What it never does
It never submits forms, executes JavaScript, stores personal data found on the page, or attempts to reach non-public addresses. Every hop, including redirects, is checked against a guard that blocks loopback, private, link-local, and other reserved network ranges before any request is made.
Allowing or blocking it
ContentSignalBot respects standard robots.txt directives for its User-Agent token, ContentSignalBot. To block it from a site entirely, add:
User-agent: ContentSignalBotDisallow: /
Because each fetch is triggered by a specific request (a customer’s monitored URL or a manual page-checker submission), blocking it only affects that one request rather than an ongoing crawl.
Questions about a request you received
If ContentSignalBot fetched a page on your site and you want to know why, contact us at getcontentsignal@gmail.com with the URL and, if available, the timestamp from your server logs.