Inspired by this Hacker News comment: https://news.ycombinator.com/item?id=42002543
The commenter was surprised and annoyed that none of the major search engines appear to have any knowledge of a particular blog post he wrote. For comparison, I tried searching for it on Kagi, and was surprised that it didn't show up there either.
The blog post in question: https://mikewarot.blogspot.com/2007/10/mikes-law-of-intermittent-hardware.html?m=1
Search terms I tried:
- "Mike's law of intermittent hardware"
- "mikewarot.blogspot.com intermittent hardware"
I checked robots.txt and it doesn't have anything surprising or problematic.
Interestingly enough, if I search for just "mikewarot.blogspot.com", Kagi finds the blog and even lists out a few recent posts.
I'm curious why Kagi seems to have know knowledge of this particular page. Is it simply because the content is relatively short? Or does the crawler/indexer somehow mark it as irrelevant?
Expected result: the blog post in question shoes up high in the results when the exact title is used for a search query or the blog's domain is included (with relevant keywords) in the search
