WGRIN Verify · User agent
About the WGRIN-Verify bot
If you found WGRIN-Verify/1.0 (+https://wgrin.org/verify/bot) in your logs, this page explains it.
What is it?
WGRIN Verify checks publicly observable technical facts about a website (HTTPS, DNS, security headers, structured data, robots.txt and sitemap) when someone requests a check of that domain on wgrin.org, or when an organization has a WGRIN profile for it.
What does it request?
Only public URLs: the home page (over HTTPS and plain HTTP, to see whether it redirects),
/robots.txt, the sitemap declared there (or /sitemap.xml) and
/llms.txt, plus public DNS lookups. A check makes at most ten requests, follows at most
five redirects and never submits forms, logs in or runs JavaScript.
Does it obey robots.txt?
Yes (RFC 9309). It reads /robots.txt first and follows the group for
WGRIN-Verify, or User-agent: * if there is none, including
Crawl-delay. If the home page is disallowed, nothing but robots.txt is read; disallowed
paths are reported as “not checked”. If robots.txt answers with a server error or cannot be reached,
nothing else is fetched. To opt out entirely:
User-agent: WGRIN-Verify
Disallow: /Profile discovery
When a member of an organisation asks WGRIN to suggest profile information from the
organisation's own website, the same bot reads up to 12 public pages of that domain (for example
the about, services, products and contact pages, and the sitemap), at most 24 requests within a
minute. It obeys robots.txt for WGRIN-ProfileDiscovery first, then
WGRIN-Verify, then User-agent: *, including Crawl-delay.
To allow website checks but not discovery:
User-agent: WGRIN-ProfileDiscovery
Disallow: /Discovery only proposes information to the organisation's own members, who review every suggestion; nothing is published automatically. Personal contact details are not collected.
How can I tell it is really WGRIN?
Every request is signed with HTTP Message
Signatures (Web Bot Auth): it carries Signature-Agent: "https://api.wgrin.org",
Signature-Input and Signature headers made with an Ed25519 key. The public
key is published at
https://api.wgrin.org/.well-known/http-message-signatures-directory. A request claiming to
be WGRIN-Verify without a valid signature is not from us. The bot runs on shared cloud addresses, so
its IP address does not identify it.
How often?
At most once per domain within a few minutes; repeated requests reuse the recent result. Only one check per domain runs at a time.
Where does it connect?
Only to public internet addresses. Private, internal and cloud-metadata addresses are refused before any connection is made.
How do I contact you?
Email [email protected] with the domain and the time of the request.