Google’s Latest Indexing Glitch: How & Why It Happened

On August 10, 2020, Google was hit by a massive glitch that disrupted search results for hours. 

Casey Markee, who runs SEO consulting firm Media Wyse, took to Twitter after search results for “baked beans” served up recipes for baked salmon, sausage casserole, and teriyaki meatballs. The issue lasted for several hours, sending people into a temporary panic mode.

While some people in the search community were quick to write off the glitch as just another indexing update, it later turned out to be the result of a bug. 

So What Really Happened?

It’s not the first time that Google’s search engine has experienced issues. Just last June, the search engine was hit with a bug that blocked users from seeing newly published web content. But the recent disruptions show that it was an indexing bug that was causing all the trouble.

The problem: The indexing glitch caused two major issues: poor search results and ranking changes that didn’t make sense. The issue made search results turn haywire, making a Minnesota company rank for a Dallas search phrase.

The carousels are f**ked. Look at this one for “baked beans” it’s all over the place. I have several other examples. Good times. pic.twitter.com/avxxu10Fed

— Casey Markee (@MediaWyse) August 11, 2020

A Minnesota company ranking for “Dallas SEO” lol

— Joe Youngblood (@YoungbloodJoe) August 11, 2020

The timeline: The SEO community started noticing significant ranking changes to the Google search results as 1:30 pm ET on August 10. The issue continued for the rest of the day until Google confirmed via Twitter that it had been hit by an indexing glitch that affected search results. By 12:40 a.m. ET on August 11, Google reported that the issue had been fixed. 

On Monday we detected an issue with our indexing systems that affected Google search results. Once the issue was identified, it was promptly fixed by our Site Reliability Engineers and by now it has been mitigated.
Thank you for your patience!

— Google Webmasters (@googlewmc) August 11, 2020

The scope: A report from Search Engine Journal shares that the bug affected users in Europe, Asia, and other English-speaking countries. The glitch affected search queries ranging from recipes to local services, while e-commerce businesses reported extreme fluctuations in rankings.

What’s next: If you’ve noticed any changes in organic traffic in your analytics and reporting tools, it’s best to ignore them for now. Since Google has already addressed the issue, your rankings and traffic will soon return to the way it was.

Google Caffeine: A Closer Look

Some SEO experts weren’t happy with Google’s short explanation about how and why the glitch happened. The havoc wreaked by the search bug led to even more questions about how Google’s web crawling and index system — Caffeine — works.

Gary Illyes, a Google Webmaster Trends Analyst, cleared the air by tweeting useful info about the indexing system, explaining the role it plays in making search results show up.

The indexing system, Caffeine, does multiple things:
1. ingests fetchlogs,
2. renders and converts fetched data,
3. extracts links, meta and structured data,
4. extracts and computes some signals,
5. schedules new crawls,
6. and builds the index that is pushed to serving.

— Gary 鯨理／경리 Illyes (@methode) August 11, 2020

If something goes wrong with most of the things that it’s supposed to do, that will show downstream in some way. If scheduling goes awry, crawling may slow down. If rendering goes wrong, we may misunderstand the pages. If index building goes bad, ranking & serving may be affected

— Gary 鯨理／경리 Illyes (@methode) August 11, 2020

Don’t oversimplify search for it’s not simple at all: thousands of interconnected systems working together to provide users high quality and relevant results. Throw a grain of sand in the machinery and we have an outage like yesterday.

— Gary 鯨理／경리 Illyes (@methode) August 11, 2020

But before we get lost in the technical aspect of how indexing works, let’s break it down and see how it ties in with making a search query on Google. 

Relevant, Real-Time Data

A search indexing system is important because it allows Google to process data faster. Think of it this way: when you enter a search term on Google, you’re not searching the live web. Instead, you’re searching Google’s index of the web, which functions the same way a table of contents does and points you to the specific info you need. 

A web indexing system is important because content is ever-expanding. It continues to grow in size each day with new blogs, images, videos, and real-time updates, making web pages richer and more complex. To top that off, the average user expects to access the latest data in real-time.

Google Caffeine solves this problem by indexing hundreds of thousands of web pages each day and delivering users relevant search results. The web indexing system was released in 2010, and you can learn more about it here.

A Complex Process

Once you understand how complex indexing is, it’s much easier to understand why things go could go wrong every once in a while. Search engines serve three primary functions: they crawl, index, and rank your site. Anything that goes wrong in each stage will have a subsequent impact on the rest of the process. 

This is why we can’t simply assume that ranking fluctuations are the result of an algorithm update. Here are some factors that you can look into if you notice any unexpected changes in rankings:

• Directive changes – Check whether you’ve made any accidental changes to your noindex tags, blocked pages, status codes, and the like. Sometimes, accidentally noindexing or blocking a page could inadvertently tell Google that you don’t want them to index your content. 
• On-site modifications – Any changes to your site, such as removing an internal link, rewriting your web page copy, or changing from an HTML to a Javascript framework could lead to fluctuations in rankings. 
• Competitive ranking shifts – Maybe your competitor updated their content for better relevance on your target keywords. You can verify this by conducting a simple google search for the keyword you lost the position for and perform competitor analysis.

Algorithm Updates vs Search Index Bugs

Google’s Core Algorithm updates happen almost as frequently as it comes under attack from a search bug. So how do you find out the difference?

A Google Core Algorithm Update indicates that they’ve made improvements to its algorithms or systems to rank the relevance of search results relative to a query. Indexing issues, on the other hand, happen as a result of an accident on Google’s end. These problems aren’t a reflection of anything you did wrong; rather, it’s an issue that Google needs to address. 

To stay in the know, it helps to keep tabs on any updates from the official Google Webmasters and Google SearchLiaison Twitter accounts or the Google Webmaster Central Blog.

Why It Pays to Stay Vigilant

The unexpected happens more often than we realize. Whether it’s an algorithm change or a bug that throws your rankings out of whack for a few hours, it’s bound to happen again at some point in the future. But if the volatility of the SEO landscape teaches us anything, it’s that you need to be prepared for anything that could happen. 

Any change can impact your current marketing strategy, and a change in Google’s algorithm could help or hurt your search traffic, rankings, or revenue. So stay in the know, read credible sources, and make necessary changes to your strategy as you go along.