Google has added a brand new Google crawler, a brand new person agent, to the Google spider listing. Its identify is Google Website Verifier Person Agent. This person agent ignores robots.txt guidelines, by the best way, and I assume it’s used for verifying websites with Search Console and different Google instruments and providers.
Google posted the brand new person agent over right here within the Google Crawlers assist documentation.
The person agent token is: Google-Website-Verification
The total person agent string is: Mozilla/5.0 (appropriate; Google-Website-Verification/1.0)
And Google famous with a “warning” that the “Google Website Verifier ignores robots.txt guidelines.”
So maintain an eye fixed out for it in a really restricted foundation.
Discussion board dialogue at Twitter.
Replace: My assumption that this was for Search Console verification was appropriate however John Mueller of Google mentioned the person agent will not be new, the brand new half is that it was added to the docs:
It is for Search Console verification. Not new, simply newly listed.
— 🌽〈hyperlink href=//johnmu.com rel=canonical 〉🌽 (@JohnMu) September 20, 2022
