WordPress Sites Will Ruin the Web
Many great websites are suddenly and quietly disappearing out of search results and we blame WordPress for that.
In the last 30 days I’ve seen more websites disappear out of Google’s index than ever and went to investigate in hope to understand what is going on. After closely examining about ten cases I stopped looking any further as they all fell into a single pattern. They were all based on WordPress (typically with /wp-admin accessible) and all had their privacy settings configured to block search engines (noindex, nofollow). Naturally, without allowing search engines to index their pages these sites are excluded from search and carved out of the link graph.
Why is this happening?
A lot of great companies and organisations look forward to improve their online presence by moving onto a CMS platform and updating their design. WordPress, naturally, is a popular choice. Webmasters (and sadly even web design companies) are not always aware of the impact of privacy settings in WordPress asking to block search engines. I asked a few affected website owners and they said that this looked like a good way to protect their website’s content and privacy, which of course is a good, but they didn’t understand the more serious side of being inaccessible to search engines.
Why is this bad for the web?
Just think about how many fantastic websites are wiped out of Google after re-design or move to WordPress. If this trend continues a whole lot of great resources will suddenly and quietly disappear out of search results.
What can WordPress do?
WordPress needs to warn users when they are about to block search engines, reword the privacy settings panel and finally tell the webmasters that their website is disallowing search engines to index it and follow its links and that this translates in nobody being able to find them. This should be in a red box with big bold letters. Some may of course chose to keep search engines away for a good reason, so once closed the message never pops up again, unless you navigate through to the privacy settings again.
What can webmasters do?
Log into your WordPress admin area and in privacy settings select “Allow search engines to index this site.”
Have you seen any great websites which have accidentally blocked search engines in their settings?
Tell us about it.