WordPress Sites Will Ruin the Web

Many great websites are suddenly and quietly disappearing out of search results and we blame WordPress for that.

In the last 30 days I’ve seen more websites disappear out of Google’s index than ever and went to investigate in hope to understand what is going on. After closely examining about ten cases I stopped looking any further as they all fell into a single pattern. They were all based on WordPress (typically with /wp-admin accessible) and all had their privacy settings configured to block search engines (noindex, nofollow). Naturally, without allowing search engines to index their pages these sites are excluded from search and carved out of the link graph.

Why is this happening?

A lot of great companies and organisations look forward to improve their online presence by moving onto a CMS platform and updating their design. WordPress, naturally, is a popular choice. Webmasters (and sadly even web design companies) are not always aware of the impact of privacy settings in WordPress asking to block search engines. I asked a few affected website owners and they said that this looked like a good way to protect their website’s content and privacy, which of course is a good, but they didn’t understand the more serious side of being inaccessible to search engines.

Why is this bad for the web?

Just think about how many fantastic websites are wiped out of Google after re-design or move to WordPress. If this trend continues a whole lot of great resources will suddenly and quietly disappear out of search results.

What can WordPress do?

WordPress needs to warn users when they are about to block search engines, reword the privacy settings panel and finally tell the webmasters that their website is disallowing search engines to index it and follow its links and that this translates in nobody being able to find them. This should be in a red box with big bold letters. Some may of course chose to keep search engines away for a good reason, so once closed the message never pops up again, unless you navigate through to the privacy settings again.

WordPress Indexing Warning

What can webmasters do?

Log into your WordPress admin area and in privacy settings select “Allow search engines to index this site.”

WordPress Privacy Settings

Have you seen any great websites which have accidentally blocked search engines in their settings?

Tell us about it.

Dan Petrovic, the managing director of DEJAN, is Australia’s best-known name in the field of search engine optimisation. Dan is a web author, innovator and a highly regarded search industry event speaker.

More Posts - Website

6 thoughts on “WordPress Sites Will Ruin the Web

  1. Good post. The title caught me off guard as I assumed you were coming out against WordPress but then you gave instructions on how to fix the index/privacy settings. This is good education for WordPress site owners. 

  2. Title of the post caught my attention too! Good job. :) Once the site is off your agency’s development server and onto client’s server, the settings for Privacy should be changed to ‘Allow search engines to index this site’. Keeping checklist of things like this ensures that all sites are properly indexed.

    Following this, it is also crucial that the agency submits the site’s sitemap to Google Webmaster tools. This should ideally sit at http://www.yourwebsite.com/sitemap.xml so that Google can see what pages are existing on your site.

    Vineeta @eatwholegrain

  3. I couldn’t agree more. This alert was much more obvious pre version 3.2 or 3.1, now the only place the that WordPress alerts you is in the dashboard here: http://cl.ly/3Y262K3T2D3R3i1I3S0Z It’s just not obvious enough. It can be an easy thing to miss when migrating a website from a staging website to live unless you check it off.

  4. This is not down to WordPress.

    It wouldn’t happen nearly as much if these companies/organisations hired an expert who is familiar with WordPress SEO.