Is Google Penalizing Websites With Disallowed Pages?
Recently, I had a client come to me with a very interesting Google penalty. They had made a few changes to their site, nothing major, mostly tweaks to usability and layout. After these changes, their site dropped in rank.
Curious, I went through each of their changes, a short list. Moving a button? Not an issue. Condensing a couple pages, properly redirected? All fine. The only possible issue I could see was, as it turned out, their contact page.
This client of mine had decided to be a little fancy with their contact page and put the information in a modal window. As you know, a modal window is hidden most of the time, and contact information contains links and keywords like their business name. The client thought, logically enough, that the presence of links and keywords in otherwise hidden content might earn them a penalty. To avoid that penalty, the client added a noindex entry for the page in their robots.txt.
This, as it turns out, earned them a penalty anyway. It cleared up when the robots.txt entry was removed, while the contact information was still hidden in the modal window. So this taught us a few things.
1: Google is Smart about Hidden Content
Google is intelligent. Google also gives you the benefit of the doubt.
One of the things Google has been penalizing for a long time is hidden content. It doesn’t matter how the content is hidden, whether you’ve put it in a hidden div that is never revealed, or you’ve matched the text and background colors, or you’ve thrown a hundred keywords in your meta data. It’s all content the search engine sees but the user cannot. When that content is overly optimized or working in opposition to the content of the page, the page can be penalized for it.
The thing is, there are a bunch of ways you can use hidden content legitimately (okay, so the text/background matching thing is never really legitimate). Take a look, then, at hidden divs. Make a FAQ page and hide the answers until the question is clicked; that’s hidden content. The same goes for modal windows; they’re content that doesn’t display until a specific action is taken.
Modal windows and hidden divs are both fine as far as Google is concerned, as long as you’re not using them in excess and you’re not using them to game the search engine algorithm. In our case, our client was using a modal window for contact information, which isn’t even all that uncommon. Google has no issues with such a benign use of hidden content.
2: Google Reads Noindexed Content
This is somewhat old news, but the “noindex” tag isn’t a be-all and end-all solution for hiding pages from Google. Google can find those pages, and they can read them. The only thing that noindex tags do is prevent Google from serving the page in the search results. You aren’t hiding the content of the page from Google, you’re just telling Google to hide it from others.
This is important because it means Google knows what you’re putting on that page. It’s also important because different search engines handle the tag differently. That difference in action has been the subject of long debate, but it still exists.
So the takeaway here is that Google reads your noindexed pages. If you want a page to remain hidden from Google, you would need to hide it behind a form, a user login or some other block that a user can pass but a search engine cannot.
3: Google Hates Hiding Critical Pages
Google offers a tool through the Webmaster Tools monitoring system that analyzes your overall site health. It will check for common problems, like a malware-infected site, an important page being removed or an important page being blocked. Note the difference between those last two; again, Google can tell the difference between you blocking the content and you deleting it.
The problem our client faced was that they placed a noindex tag on their contact page. Google considers the contact page an essential page for a website, particularly a business page. You can’t hide your contact information; it hurts you in a number of ways. Among other things, a lack of contact information makes it impossible to place in the local carousel or in local search results. It’s the same reason you shouldn’t put your phone number in a picture rather than text.
If you’re in a position to earn sitelinks in your search results, you’ll probably want your contact information available as well. It’s just one more way you hurt yourself by hiding essential pages.
4: Google Penalizes what it Hates
So, Google hates it when you hide an essential page. They warn you about blocking important pages in your Webmaster Tools site health checkup. Now, it appears, they’re penalizing it as well.
Google wants you to have as healthy and effective a site as possible. They want you to do this naturally, without artificial techniques, which is why they penalize you for everything from unnatural keywords to spam backlinks to duplicate content. Anything that might give you an unnatural advantage is penalized, but, apparently, so are actions that can hurt you.
You might wonder how much of the problem with our client site was due to a Google penalty and how much was due to the natural consequences of being harder to contact. Well, if our client had completely lacked a contact page, we could assume that their drop in rankings came from that issue alone. Fortunately, our clients didn’t want to drop their contact information entirely; that’s why it was still there, after all. They just didn’t want to risk a penalty from hidden content in a modal window.
That caution, coupled with an incomplete understanding of Google’s actions, led to this penalty. It’s entirely understandable, there’s plenty of rhetoric online about Google penalizing people for hidden content, but there’s not much information one way or another about legitimate uses for modal windows, and the page indicating that hiding a critical page is bad is hard to find. The “better safe than sorry” attitude turned out to be both unsafe and sorry.
It appears that a minor warning in the Webmaster Tools, a warning in a section full of warnings you can usually ignore safely, isn’t doing enough to encourage webmasters to keep their critical pages visible. As a consequence, Google appears to be trying to draw more attention to the issue by leveraging minor penalties. Once the page was made visible again – modal window and all – the penalty was gone.