I noticed this thread on Mastodon the place each John Mueller and Danny Sullivan of Google was serving to web legend Tim Bray with some web optimization points he was having with Google Search. Sure, he obtained two Googlers serving to him, which is uncommon, however has occurred earlier than.
Tim Bray is legendary, properly, web well-known, he has a Wikipedia entry that reads, “Timothy William Bray (born June 21, 1955) is a Canadian software program developer, environmentalist, political activist and one of many co-authors of the unique XML specification. He labored for Amazon Net Providers from December 2014 till Might 2020 when he give up on account of considerations over the terminating of whistleblowers. Beforehand he has been employed by Google, Solar Microsystems and Digital Tools Company (DEC). Bray has additionally based or co-founded a number of start-ups resembling Antarctica Programs.”
So when somebody web well-known complains about Google – Google notices, properly, I believe Danny Sullivan not less than notices.
Tim wrote:
I’ve reported earlier than that Google is dropping its reminiscence – see https://www.tbray.org/ongoing/When/201x/2018/01/15/Google-is-losing-its-memory
It is getting worse. I used to be on the lookout for a weblog piece earlier this 12 months wherein I discussed a motorcycle accident I would had, and remembered that I would been sporting a Bontrager helmet (really helpful BTW) so I looked for “bontrager” by way of
bontrager web site:tbray.org
and Google can’t discover it. DuckDuckGo and Bing can with the very same string.
Search used to matter to Google.
So Danny and John stepped in to assist with some web optimization points with the location and likewise saying they’ll ship the suggestions to the precise groups at Google Search.
Danny first got here in and observed it and wrote, “When you’ve got an instance you wish to share sooner or later, glad to look. That’s completely different from us not having listed a web page — @timbray I see @johnmu gave you a reply on what could be tripping us up on this case”
John then dug in a bit and located some points, he stated:
Hello Tim, I work with the search people at Google. I took a fast look right here, and can move a word on internally.
To chop to the chase, what occurred right here is we listed https://www.tbray.org/ongoing/goto-potd/ out of your web site whereas it was redirecting to your E-Bike article, so we listed that content material beneath the “potd” URL. Then, the contents for “potd” modified (I assume that is on design), and we listed that, and misplaced the E-bike content material.
There are just a few methods to repair this:
– Google might simply determine it out and cope with it on their very own. I handed this on, in order that we will enhance the techniques, nevertheless it’s a bizarre edge-case, imo.
– block the “potd” URL with robots.txt in order that it could possibly’t get picked up by search engines like google.
– use link-rel-canonical annotations on the person pages in order that Google is extra prone to decide these URLs.
If you would like examples of the final two, glad to dig some up.
He then had a little bit of a backwards and forwards on some web optimization questions with Tim, all helpful for us to evaluation as properly. Listed here are a few of these responses:
We usually decide up the robots.txt file about as soon as a day (it relies upon a bit on the location, however since it is a static file, we attempt to cache it to cut back the load on the server). My guess is by “tomorrow” (relying on timezone :-)), it should cease crawling that.
With indexing of that URL vs the article, I think it will take longer for the techniques to understand it must re-evaluate the state of affairs (I am guessing per week or so, nevertheless it’s unimaginable to say).
It will possibly get tough when our techniques suppose they’ve already seen the content material, simply on one other URL (there’s a lot duplication on the net).
Usually although, it is uncommon that we might index all the things from an internet site. This can lead to even pages carefully linked from the homepage not getting listed. I do not wish to set the expectation {that a} technically clear web site will all the time have all the things in search, as a result of it is virtually by no means the case.
Danny additionally answered some normal questions on how Google Search works in that thread:
No, we do not deprioritize older content material (nor was this put up from April 2022 “previous”). We attempt to present as a lot helpful content material as we will. On this explicit case, it is seemingly indexing tripped up for a technical purpose.
We index pages. Previous and new. We additionally rank pages. Previous and new. Typically, it could possibly assist to rank pages which are brisker, resembling if there is a trending concern occurring. I believe that usually is smart. This explains extra about how #Google #search makes use of freshness in rating.
I do imagine when somebody who’s web well-known, is adopted extra and revered extra on-line, complains about Google Search, it does get Google’s consideration greater than a traditional particular person. However on the similar time, it is smart, as a result of others additionally comply with them extra and Google, not less than from a PR perspective, desires to leap on these considerations earlier than somebody like me writes it up. Plus, I believe Danny simply follows Tim, so he in all probability noticed it that means anyway.
However sure, being web well-known would not damage relating to getting assist from Google with web optimization questions. It does not imply Google will press a button to magically make Tim’s web site rank higher, by the way in which…
Discussion board dialogue at Mastodon.
