Google has added a brand new part to the crawl finances administration assist doc. The brand new part is for the If-Modified-Since request header. The part is titled “Specify content material adjustments with HTTP standing codes.”
Google wrote, “Google typically helps the If-Modified-Since and If-None-Match HTTP request headers for crawling. Google’s crawlers do not ship the headers with all crawl makes an attempt; it depends upon the use case of the request (for instance, AdsBot is extra prone to set the If-Modified-Since and If-None-Match HTTP request headers).”
“If our crawlers ship the If-Modified-Since header, the header’s worth is the date and time the content material was final crawled. Based mostly on that worth, the server might select to return a 304 (Not Modified) HTTP standing code with no response physique, wherein case Google will reuse the content material model it crawled the final time. If the content material is newer than the date specified by the crawler within the If-Modified-Since header, the server can return a 200 (OK) HTTP standing code with the response physique.”
“Independently of the request headers, you may ship a 304 (Not Modified) HTTP standing code and no response physique for any Googlebot request if the content material hasn’t modified since Googlebot final visited the URL. It will save your server processing time and assets, which can not directly enhance crawl effectivity.”
I’m not positive if I’ve any commentary on this, outdoors of the truth that this new part was added to this web page.
Discussion board dialogue at Twitter.
