Monday, October 21, 2013

How to do a 301 Redirect Properly on Apache or Windows Servers

In today’s blog post we will discuss how to implement a proper 301 redirect on Apache or Windows servers from one domain to another. Redirects are technical and we see a lot of sites where 301 redirects are not implemented properly. You might want to do 301 redirects because of a number of reasons: redirecting the non www to www and vice versa, or if you are changing your domain or a file within the same domain.  This is also a great post on ways you can fix your 404 error pages.

Before we enter the technical details, it is important to understand the importance of a 301 redirect from non www to www version of your site (or vice versa). First having two versions of your site can create duplicate content, which may result in your website being penalized by search engines. Secondly and most importantly, when you acquire links it’s always much better to have them pointing at one version of the site versus distributing it among two pages which dilutes the search engine importance to your domain.

301 redirects is the most preferred way of handling duplicate content. Other ways include using the ” rel = canonical” tag (don’t use for cross domain, Yahoo/ Bing still don’t recognize it), blocking files in robots.txt and the meta noindex tag.

Let’s dive into the technical details:

How to do a 301 redirect for an Apache server:

Step 1 : To implement a 301 redirect the file we need to work with is the .htaccess file. To access the file you need to go into your FTP and look into the document root.

Step 2 : If you can’t see it, enable viewing of hidden files since the .htacess file is hidden. If there is still no .htaccess file present , create one with a simple text editor.

Step 3 : Insert this code in the file:

Code example from non www to www:

RewriteEngine On
RewriteCond %{HTTP_HOST} !^www.example.com$ [NC]
[NC] RewriteRule .? http://www.example.com%{REQUEST_URI} [R=301,L]

Obviously you will need to replace ‘example’ with your own domain name.

Also make sure the Rewrite Engine is turned on, you will just need to turn it on once.

Step 4 : Save and Test it!

How to do a 301 redirect for a Windows server:

When setting up a site in IIS, the normal process is to create one account for the site and add both www and non-www versions of the domain name to the host headers for the account. This creates a canonicalization issue;  and the site will then be available at both www and non-www URLs.

Step 1 : Get access to the Windows Server navigation Panel. Navigate your way to the Internet Services Manager (Programs — Administrative Tools — Internet Services Manager).

Step 2 : Create 2 accounts for the site within IIS: one with the www version of the domain in the host header and one with the non-www version of the domain. All of the site files can be placed in the preferred version and a single page in the other.

Step 3 : Right click on the single page you want to redirect FROM and choose Properties. The Properties box will now appear.

Step 4 : Change the redirect option to “A redirection to a URL” and type in the new URL in the box provided.

Step 5 : Be sure to check the box marked “A permanent redirection for this resource”. If you leave this box unchecked, you will create a 302 (temporary) redirect, which is not permanent or beneficial from an SEO standpoint in this situation.

Step 6 : Test it!

Doing a www redirect for Front Page

As I see some of the comments below pertaining to Front Page, it was a matter of time before I had to do one for this God-forsaken MS product myself.  Here’s how I did after some trial and error:

1.  First, you have to identify weather you are running Linux or Windows.  This works for Linux.  Apparently, there is an extension called FollowSymlinks which needs to be turned on, as well as Mod Rewrites, so call your host provider for that one.

2.  FTP uses several .htaccess files – one in the main directory structure, and 3 other .htaccess files called “super files”.  You will find these other .htaccess files here:

/_vt_bin/.htaccess
/_vt_bin/_vti_aut/.htaccess
/_vt_bin/_vti_adm/.htaccess

3.  Make sure this is at the top of all 4 .htaccess files: “Options +FollowSymlinks” underneath “# -FrontPage-”

4.  Underneath this, add your 301 redirect command:

RewriteEngine On 
RewriteCond %{HTTP_HOST} ^yoursite.com$ [NC]
RewriteRule ^(.*)$ http://www.yoursite.com/$1 [R=301,L]

Here, I did a 301 from non-www to the www, because for SEO purposes, most people have more inbound links pointing to the www version.

That’s is – this should work!


Tags : How to do a 301 Redirect Properly on Apache, How to do a 301 Redirect Properly on Windows Servers,  non www to www, redirecting on www, non www to www with .htaccess,  nilesh patelseo services providerbest information of the world,  nilesh patel forum,  fans of photography,  nilesh patel seo

Redirecting non-www to www with .htaccess

If you want to redirect all non-www requests to your site to the www version, all you need to do is add the following code to your .htaccess file:
RewriteEngine On RewriteCond %{HTTP_HOST} !^www\. RewriteRule ^(.*)$ http://www.%{HTTP_HOST}/$1 [R=301,L]
This will redirect any requests to http://my-domain.com to http://www.my-domain.com. There are several benefits from doing that:

This will redirect any requests to http://my-domain.com to http://www.my-domain.com. There are several benefits from doing that:
  • It will avoid duplicate content in Google
  • It will avoid the possibility of split page rank and/or split link popularity (inbound links).
  • It's nicer, and more consistent.
Note that if your site has already been indexed by Google without the www, this might cause unwanted side effects, like lost of PR. I don't think this would happen, or in any case it would be a temporary issue (we are doing a permanent redirect, 301, so Google should transfer all rankings to the www version). But anyway, use at your own risk!

Something nice about the code above is that you can use it for any website, since it doesn't include the actual domain name.

Redirecting www to non-www

If you want to do the opposite, the code is very similar:
RewriteEngine On RewriteCond %{HTTP_HOST} !^my-domain\.com$ [NC] RewriteRule ^(.*)$ http://my-domain.com/$1 [R=301,L]
In this case we are explicitly typing the domain name. I'm sure it's possible to do it in a generic way, but I haven't had the time to work one out and test it. So remember to change 'my-domain' with your domain name!

In this case we are explicitly typing the domain name. I'm sure it's possible to do it in a generic way, but I haven't had the time to work one out and test it. So remember to change 'my-domain' with your domain name!


Tags : non www to www, redirecting on www, non www to www with .htaccess,  nilesh patelseo services providerbest information of the world,  nilesh patel forum,  fans of photography,  nilesh patel seo

How Google is Changing Long-Tail Search with Efforts Like Hummingbird

The Hummingbird update was different from the major algorithm updates like Penguin and Panda, revising core aspects of how Google understands what it finds on the pages it crawls. In today's Whiteboard Friday, Rand explains what effect that has on long-tail searches, and how those continue to evolve.


For reference, here's a still of this week's whiteboard!


Video Transcription

Howdy, Moz fans and welcome to another edition of Whiteboard Friday. This week I wanted to talk a little bit about Google Hummingbird slightly, but more broadly how Google has been making many efforts over the years to change how they deal with long-tail search.

Now long tail, if you're not familiar already, is those queries that are usually lengthier in terms of number of words in the phrase and refer to more specific kinds of queries than the sort of head of the demand curve, which would be shorter queries, many more people performing them, and, generally speaking, the ones that in our profession, especially in the SEO world, the ones that we tend to care about. So those are the shorter phrases, the head of the demand curve, or the chunky middle of the demand curve versus the long tail.

Long tail, as Google has often mentioned, is a very big proportion of the Web search traffic. It's anywhere from 20% to maybe 40% or even 50% of all the queries on the Web are in that long tail, sort of fewer than maybe 10 to 50 searches per month, in that bucket. Somewhere around 18% or 20% of all searches Google says are extremely long tail, meaning they've never seen them before, extremely unique kinds of searches.

I think Google struggles with this a little bit. They struggle from an advertising perspective because they'd like to be able to serve up great ads targeting those long-tail phrases, but inside of AdWords, Google's Keyword Tool, for self-service advertising, it's tough to choose those. Google doesn't often show volume around them. Google themselves might have a tough time figuring out, "hey, is this query relevant to these types of results," especially if it's in a long tail.

So we've seen them get more and more sophisticated with content, context, and textual analysis over the years such that today, with the release of, in August according to Google, Hummingbird, which was an infrastructure update more so than an algorithmic update. You can think of Penguin or Panda as being algorithmic style updates, and Google Caffeine, which upgraded their speed, or Hummingbird, which they say upgrades their text processing and their content and context understanding mechanisms is affecting things today.

I'll try and illustrate this with an example. Let's say Google gets two search queries, "best restaurants SEA," Seattle's airport, that's the airport code, the three-letter code, and "where to eat at Sea-Tac Airport in Terminal C." Let's say then that we've got a page here that's been produced by someone who has listed the best restaurants at Sea-Tac, and they've ordered them by terminals.

So if you're in Terminal A, Terminal B, Terminal C, it's actually easy to walk between most of them except for N and S. I hope you never have to go N. It's just a pain. S is even more of a pain. But in Terminal C, which I assume would be Beecher's Cheese, because that place is incredible. It just opened. It's super good. In Terminal C, they've got a Beecher's Cheese, so they've got a listing for this.

A smart Google, an intelligent engineer at Google would go, "Man, you know, I'd really like to be able to serve up this page for this result. But it doesn't target the words 'where to eat' or 'Terminal C' specifically, especially not in the title or the headline, the page title. How am I going to figure that out?" Well, with upgrades like what we've seen with Hummingbird, Google may be able to do more of this. So they essentially say, "I want to understand that this page can satisfy both of these kinds of results."

This has some implications for the SEO world. On top of this, we're also getting kind of biased away from long-tail search, because keyword (not provided) means it's harder for an individual marketer to say: "Oh, are people searching for this? Are people searching for that? Is this bringing me traffic? Maybe I can optimize my page more towards it, optimize my content for it."

So this kind of combination and this direction that we're feeling from Google has a few impacts. Those include more traffic opportunities, opportunities for great content that isn't necessarily doing a fantastic job at specific keyword targeting.

So this is kind of interesting from an SEO perspective, because we're not saying, and I'm definitely not saying, stop doing keyword targeting, stop putting good keywords in your titles and making your pages contextually relevant to search queries. But I am saying if you do a good job of targeting this, best restaurants at SEA or best restaurants Sea-Tac, you might find yourself getting a lot more traffic for things like this. So there's almost an increased benefit to producing that great content around this and serving, satisfying a number of needs that a search query's intent might have.

Unfortunately, for some of us in the SEO world, it could get rougher for sites that are targeting a lot of mid and long-tail queries through keyword targeting that aren't necessarily doing a fantastic job from a content perspective or from other algorithmic inputs. So if it's the case that I just have to be ranking for a lot of long-tail phrases like this, but I don't have a lot of the brand signals, link signals, social signals, user usage signals, I just have strong keyword signals, well, Google might be trying to say, "Hey, strong keyword signals doesn't mean as much to us anymore because now we can take pages that we previously couldn't connect to that query and connect them up."

In general, what we're talking about is Google rewarding better content over more content, and that's kind of the way that things are trending in the SEO world today.

So I'm sure there's going to be some great discussion. I really appreciate the input of people who have done extensive analysis on top of Hummingbird. Those folks include folks like Dr. Pete, of course, from Moz, Bill Slawski from SEO by the Sea, Ammon Johns, who wrote a great post about this. I think there'll be more great discussion in the comments. I look forward to joining you there. Take care.



Thursday, October 17, 2013

How Google’s Disavow Links Tool Can Remove Penalties

Can using Google’s link disavow tool help remove penalties? Yes, the company says. But when it comes to manual penalties, disavowing links alone isn’t enough. With algorithmic penalties, there may be a time delay involved. Below, more about how both methods work.

Over the past few days, I’ve encountered a couple of cases where people are confused about how the link disavow tool works to remove penalties. So, I figured a clarification post was in order. Here’s the situation, all of which I reverified with Google yesterday.

Disavowing Links: “Don’t Count These Votes!”

If you submit a disavow request, Google will automatically process that request and tag those links pointing at your site in the same manner as if they had the nofollow tag on them, in other words, as if they aren’t actually pointing at your site for link counting and analysis purposes.


This is something that came up again in a Google Webmaster Central hangout video yesterday:


In short, if links are votes, using the link disavow tool effectively tells Google that you don’t want any of those votes counted, for better or worse, toward your rankings.

This all happens automatically, and Google says it still takes several weeks until the disavow request is processed.

Removing Algorithmic Penalties

Now let’s take a situation where you’re hit by an algorithmic penalty related to links, such as the Penguin Update. “Algorithmic” means an automatic penalty, one that involves no human review at all. Rather, Google’s computers have ruled that your site has done something wrong.

To remove that penalty, you need to clean up your links. That’s where link disavow can help. Let’s assume you use it to correctly disavow bad links that were hurting you.

That’s step one, cleaning up the links. Step two is waiting for the disavow request to get processed. That, as I’ve said, may take several weeks.

Step three is that you have to wait until the next time Google runs your site against whatever part of its algorithm hit you. For many, that means Penguin. Even if you’ve cleaned up your links with disavow, you have to wait until the Penguin Update is run again before you’ll see an impact.

For example, let’s assume you were hit by Penguin 3 last October. You used the link disavow tool to clean up your links soon after that. You still have to wait until Penguin 4 happens before you should see a change (and Google has said that more Penguin updates haven’t yet happened).

Now take the same situation, where you file the disavow request just a few days before a Penguin Update. Even though the request went ahead of the update, you still might not get cleared because by the time it’s processed (several weeks), the latest update will have happened. You’ll have to wait for the one after that.

Eventually, if you’ve used the tool, you should see a change. It’ll just take time. But if it was an algorithmic penalty, then it should automatically clear if you file disavow alone (or clean up your link profile in other ways).

Removing Manual Penalties

The situation is different — and potentially much faster — if you were hit by a manual penalty. That’s when some human being at Google has reviewed your site and decided that it deserves a penalty. In virtually all of these cases, it also means you would have received a notice from Google that this has happened.

If the penalty involves bad links, the link disavow tool can help you disown those. However, the penalty won’t automatically be removed because it was placed manually. You have to also file a reconsideration request. This will prompt a human being at Google to check on your site. They can see that the link disavow request has been filed, and if that’s enough, then the manual penalty may get lifted.

You have to do both: disavow links and file a reconsideration request, which Google has said before. And really, you have to do a third thing, which is make a good faith effort to remove links beyond just using link disavow, which Google has also said before (see our Q&A With Google’s Matt Cutts On How To Use The Link Disavow Tool for more about this).

There is one caveat to the above. Manual penalties have expirations dates, Google reminds. This means after a period of time, perhaps a few weeks or a few months, the penalty against your site should expire naturally. That’s why you might see an improvement even if you do nothing. (But note from the comments below, some penalties can go on for two or three years before they expire).

Doing nothing, however, may leave you subject to an algorithmic penalty in the future. In short, if you get a manual penalty, take that as a solid warning you need to fix something, lest you face a longer-term algorithmic penalty in the future.

Google’s Matt Cutts: Guest Blogging Best Done In Moderation

In his latest video, Google’s head of search spam Matt Cutts answers the question, “How can I guest blog without it appearing as if I paid for links?”

According to Cutts, when his team reviews spam reports, there is usually a clear distinction between organic guest blog content and someone who is paying for links.

Cutts identified specific differences between spam and organic guest blogging content, confirming the spam content doesn’t match the subject of the blog itself and will contain keyword rich anchor text.

A true guest blogger, said Cutts, is usually someone who is an expert on the subject matter and doesn’t drop a heavy amount of keywords in their anchor text.

“Guest blogging seems like it’s the fad of the month,” said Cutts, but he says it is best done in moderation. “It shouldn’t be your full-time job.”

He emphasizes that if guest blogging is all you are doing to drive traffic to your site, you’re probably not doing any favors for your site’s reputation.




Ultimate Guide to Google Penalty Removal

Introduction

A few months back, I wrote an article on Moz all about a penalty our web agency received for unnatural links pointing to our website. At first, this was a bit of a shock to the system, but since then, we've learned so much about Google's webmaster guidelines and we've helped lots of companies get their businesses back on track and remove manual penalties associated with their websites.

What did we get hit for?

Cutting a long story short, the main reason we were hit with a manual penalty was for adding followed anchor text into the bottom of clients websites that said 'Web Design Yorkshire by Pinpoint Designs' (both 'web design yorkshire' and 'Pinpoint Designs' linked to our website). At the time, we were just doing this out of habit, but we never varied anchor text, always had followed links and we were basically violating Google's quality guidelines.

After a lot of work and research, we managed to remove the penalty from our website and since then have worked on lots of other clients websites to help them lift penalties. We've worked with clients who've had both unnatural link penalties along with on-site penalties for low quality content, cloaking issues and malware issues. We have a great success rate and are consistently trying to improve our processes to become even better!

What are we doing to improve?

Over the past few months, we've been trying out different tools to aid link recovery, speeding up our process of getting in touch with webmasters, finding new ways to contact webmasters and ultimately just trying to streamline the process of getting spammy links removed. In this guide, I'm going to review a few of the different link removal tools we've tried out and also give you some ideas of how you can carry out link removal work manually to get your website penalty removed.

This guide is mainly for people who've specifically received a penalty for unnatural links warnings. To find out if you've received a penalty for unnatural links, simply log in to Google Webmaster Tools and use Google's new 'Manual actions' tab to see what type of penalty you have. If it's for 'Unnatural links' with a yellow warning symbol next to it, then you're on the right guide!







My aim over the next few months is to try to write guides on different types of penalties in order to help people out further. We're also working on a tool ourselves called Peel, which we're building from the ground up in order to try and deliver exceptional analysis of links. You can sign up to our newsletter for tips and information about the launch of the product using the link above.

Let's get started!

Step 1: collecting backlink data

First of all, we need to pull a list of all the links pointing to your website from a few different sources. A Google employee on the Webmaster Central Forums has recently stated that they recommend you focus on the Webmaster Tool Links.





Whether you choose to believe that though is another question, there's an interesting discussion on this available at the Webmaster Central Forums:


Matt Cutts has also responded recently saying the following:

"It's certainly the case that we endeavour to show examples from the set of links returned by Webmaster Tools, and likewise we prefer to assess reconsideration requests on that basis of those links. However, if there's a really good example link that illustrates a problem, we do leave enough room to share that link, especially because it can help point the webmaster in a better direction to diagnose and fix issues."

Whilst it may be that Google just want you to remove the majority of bad links and not necessarily every single one of them, I would personally recommend doing the job properly and start by collecting as much data as possible from various different link sources:
  • Google Webmaster Tools - Login to Google Webmaster Tools and click into the website with the issues, go to 'Search Traffic' > 'Links To Your Site', then click the 'more' link under the 'Who links the most' tab. Once in here, click the 'Download more sample links' from the top.

    Google Webmaster Tools Export
    • Open Site Explorer - Visit http://www.opensiteexplorer.org and type in your domain name (make sure you get this absolutely correct as having a slight variation of your domain such as missing out the www. can cause a different set of results). Once loaded, click the 'Advanced Reports' tab, select 'Links that come from: External linking page' and leave all the other settings as they are, then click export.
    • Ahrefs - Visit http://www.ahrefs.com and enter in your domain. Click the 'CSV' tab and export the list of 'Backlinks/Ref pages'. Ahrefs is an absolutely brilliant tool that's helped us so many times with link removal campaigns. It allows you to narrow down quickly sitewide links, followed / nofollowed links and more.
    • Majestic SEO - Visit http://www.majesticseo.com, enter your domain name and select 'Historic Index'. This will show all your previous links instead of just the most recent. Click on the backlinks tab, scroll to the bottom of the page and click 'download data'.
Most of the above sites will require you to have subscriptions in order to gather the data. Sites such as Majestic SEO will allow you to use the tool free of charge (or a limited amount) if you have the domain verified in your Webmaster Tools account. That being said, for the sake of one or two month's membership, it's worth paying for the wealth of data you'll get.

Note: Google have previously recommended you add both the www. and the non-www. version of the domain to Webmaster tool and gather links from both sources. It's worth search different variations of your domain to get as much data as possible.

Step 2: documenting your work

Start by creating yourself a Google Docs Spreadsheet where you can track all of your work to see exactly where you are at with the link removal process. This will be extremely useful when you come to submitting a reconsideration request with Google, as they'll be able to see exactly what you've done to sort out the issues with your site.

I should also point out at this point that Google say they don't tend to trust external links from sources they don't trust. For that reason, I recommend only using a Google Spreadsheet when documenting your work as it's a Google product and trusted by them. You can also include a link to this document in your reconsideration request very easily and it's free!

We would usually start by creating tabs at the bottom of spreadsheet for each of our data sources. For example, Google Webmaster Tools, Ahrefs, Majestic SEO & Open Site Explorer. You don't have to separate your data into different sheets, but I personally think it shows more work being carried out and allows more in-depth analysis.

The only disadvantage to doing this is that you'll have lots of repeat links - we do however have this covered!

If you visit the Pinpoint Designs Blog, we've created a Google docs spreadsheet that you can use which helps to combine multiple sheets together and remove duplicates. This is a brilliant asset to use on your link removal campaign.

Google Docs only allows importing of 50,000 characters at a time. This may sound a lot, but it's surprising how quickly you hit this limit. You can import data from a CSV by going to your Google Docs file, then clicking 'File > Import'. You can do this for each document you export from OSE, Ahrefs, Majestic and Google Webmaster Tools and import them into separate sheets within the same document. There's still a limit, but it's higher than 50,000 and will speed up the process.

IMPORTANT: Once you've got all your data into the spreadsheet, make sure you add the following columns to the end of each of the sheets (or alternatively your 'Master Sheet' if you're combining all the data into one sheet):
  • Contact Name
  • Contact Email Address
  • Website Contact Form
  • Twitter URL
  • Facebook URL
  • Google+ URL
  • LinkedIn URL
  • Date of 1st Contact
  • Date of 2nd Contact
  • Date of 3rd Contact
  • Link Status
  • Notes
This may seem like a lot of data, but it's the best way you can document your work. That way, when you're carrying out research on your domains, you'll be able to start populating the spreadsheet and making your life easier in the long term. You'll also be able to show Google all the work you've been doing to get the links removed and making it very easy for them to see everything you've done.

If you don't want to go to the effort of populating all the data above, you could combine all of the different forms of contact method into one cell and just populate that. The chances are, you're not going to need Twitter, Facebook, Google+, LinkedIn, Email and Website Contact Form for every single URL so it's just down to preference. The above recommendation is the best way we've found of documenting data.

Finally, add an additional sheet to your Google Docs file called 'Contact Examples'. In here, you can upload images of a few examples of the emails you've sent out to the webmasters you've been working with. Be careful what you put in here, Google will be reading these messages so make sure you're not badmouthing them.

Don't threaten Webmasters by saying Google will block their site and how they're going to get harmed if they don't remove your links. Instead, say that you're trying to clear things up so that you're fully compliant with Google's webmaster guidelines. You can apologise for the inconvenience to the webmaster and thank them for their help (examples further down in this article). That way, when Google reads them, they'll understand you're genuinely trying to sort things and hopefully be a little more forgiving under the circumstances.

Tip: If you're on a Mac, you can press 'Cmd + Shift + 4'. This command allows you to take screenshots quickly and easily of a specific section of your monitor. Perfect for quickly snapping contact forms, emails you're sending, etc. and uploading them to the 'Contact Examples' sheet in your Google docs file.

Step 3: spotting low-quality links

This is a hugely important section of link removal. It sounds simple, but you have to be extremely careful with what links you attempt to remove. Good links generally take a long time to build and if you ask for them to be removed thinking they're potentially spammy, that hard work may all be for nothing.

Over the past year, we've learnt that the best way to identify spammy links is to manually review each and every one of them. If a link is good, mark it in your spreadsheet so you know not to remove it. Don't delete any links from the sheet, as it's all research to show Google what you've been doing. Either highlight the cell in a colour, or add a note in one of the columns so you know it's safe / genuine.

So, how do we spot spammy links?

Some links are easy to identify. For example, if you've been featured on the BBC News, Guardian or a high quality, authoritative website, you can be fairly sure it's a strong link that doesn't need removing.

The definition of a natural link is fairly hard to summarise. I would class these as links that have appeared from other content you've written naturally. For example, let's say you were involved in the automotive industry and wrote an article all about the history or cars, different manufacturers and really went into great details about every aspect. This type of article is obviously going to end up very big, hopefully interesting and should be a brilliant read. If you've done a good job and shared the article in the right places, you should hopefully acquire links naturally. People will link to your guide / post naturally without you having to ask for it and those types of links are ok.

Throughout this article, I'll link to other articles I've read on the internet that I believe are helpful for link removal. All the people I link to have created good quality content that I believe will help you in removing your penalties with Google. They have written an article for the sake of it, they've written it with an aim in mind so that it's beneficial.

On the same side, it's easy to spot some spam links. For example, if you're listed on a website with 10,000 links on the same page in a list, or if you've commented on a blog with xx,xxx other comments just for the backlink. I realise that's a bit of an extreme situation, but hopefully my point is made. If you know the link has been placed on the site purely for SEO purposes, then it's most likely unnatural.

Some links however are harder to spot, so here are my top tips for identifying lower quality links:

Whether the URL is indexed in Google or not:


If not, remove the links as the site has most likely received a penalty. You can also see if a domain is indexed in Google by searching Google for 'site:yourdomain.com'.
Advanced Search Operators - Google
Page Authority and Domain Authority

Don't pay too much attention to this metric. A new domain name can still be a very strong link even though it has low page and domain authority. A poor quality site can also still have good page authority and domain authority, so just be careful when looking at this.

Site-wide Links:

Site wide links are generally associated with websites you have an affiliation with (friendly links), or links you've paid for. Due to this, it's better to make sure you don't have links across every page of someone's sites.In this case, either nofollow the links, or remove them and only put them on one page. Ahrefs does a brilliant job of spotting site-wide links on websites quickly and easily.
That being said, the same rules still apply, if the links looks spammy, remove all of them.
Link Directories:
Link directories are fairly easy to spot, if they contain words like 'backlinks', 'links', 'seo' etc. in the URL, then the chances are that they are low quality links and need removing. If it lists every category under the sun from Website Developers to Sunglasses, then it's most likely a directory that you need to remove yourself from!
There are some good link directories on the internet, generally, these are targeted to a particular niche, are manually reviewed and sometimes locally targeted. You can look at link directories and think 'Will I ever receive any traffic from this site?' or 'Is this genuinely a valuable link?' if the answer is likely to be no, then the link should be removed. Be fairly honest with yourself on this one, if it looks like spam, then it most likely is.
If you want some more tips, here are a few bullet points:
Remove a link if:
  • The site is not indexed in Google, this would indicate a penalty.
  • The site automatically accepts links without any manual review.
  • The site has lots of spam links pointing to it (type the URL of the directory into Open Site Explorer and see what you can find!)
  • The site has categories for everything imaginable (cars, holidays, sunglasses, websites, hosting, dresses etc.).
  • The site is obviously part of a network where there are lots of related directories with similar names / similar looking websites etc.
  • The site contains keywords like 'best', 'links', 'seo' etc in the name.
If the site shows 'powered by phpLD' or 'PHP Link Directory' in the footer, it's most likely going to be a fairly spammy directory. That's not always the case, but 9/10 times, it's most likely true.

During a recent link removal campaign, we managed to get a webmaster to take down a set of 20 link directories that were pointing to our client's website. They couldn't be bothered to remove the links from each site individually, so instead they took every site offline so everyone's links disappeared!

Forum Profiles:

These are usually very spammy. If you have the odd profile link on a forum that you're very active on, then this is generally fine. However, if you're listed on a forum in Romania (an example only!) and have never posted before, get the link removed.

If you're a member of the forum purely to get a link back to your website, then the link should be removed. It's a very easy to identify spam technique, so stay safe and remove them.

Blog Comments:

Similar to Forum Profiles, blog comments are an easy one to spot. If you've commented every now and again on a blog that you generally feel has helped you, or the blog is in your industry and you've added value to the discussion with your comment (rather than just posting a boring comment like 'Good work dude'), then it's probably ok.

That being said, if you've got a large number of blog comments with very little substance, you should remove all the links. If the site has hundreds and hundreds of comments and you're one of a huge list of spam comments, you should remove the link too.

Social Bookmarks:

Very similar to both the blog comments and forum profiles, social bookmarks are ok if they're genuine. Remember that the penalties you have received are manual actions and when you put in a reconsideration request, the chances are that a Google employee with manually be looking through some of your links. If your social bookmarks look spammy, remove them.

Paid Links:

If you've been paying for links, make sure you remove them or add a rel="nofollow" attribute to the link. When you're writing your reconsideration request, mention the fact that you have previously purchased links and have now rectified the issue by ensuring they are all nofollow / been removed.

Google is getting much smarter at detecting advertorial links, so don't try to trick the system.

Blog Posts:

A slightly trickier one to detect straight off. Usually, you can spot the spammy blog posts generally by looking at the URLs. If they're dynamic URLs which end in something like 'index.php?id=99', then it's usually a sign of a site being launched very quickly. The best way to identify spam blog posts is to load up each blog. Use these tell-tale signs to spot the low quality posts:
  • Does the article make sense? - Is it English, do the sentences make sense? or is it spun, low quality rubbish that benefits nobody.
  • Is the site nicely designed? - Does the website look like a genuine blog that's been looked after, cared for and regularly updated? Or is it using a standard template with awful layouts and content.
  • Is the website content unique? - You can use a website such as http://www.copyscape.com to find duplicate / spun content.
  • Is the article high quality? - Again, this is down to interpretation, but does the article provide any value to your business or is it there for the sake of being there? If it's high quality, but just linked in the wrong way, ask the webmaster to add a nofollow attribute assigned to it.
  • Is the blog too consistent? - Does the blog post about shoes one day, software the following day and medicine the following day? Also, does every post have an outgoing link to a different website, the same length and full of anchor text using 'money keywords'? If so, they're not focused and most likely either automating their posting / trying to build a site for SEO purposes. Avoid these sites and remove the links.
  • Is there an easy way to contact the owners? - Lots of lower quality blogs will remove the contact form, or completely hide who is behind them. If you can't get in touch with the owner, it's likely to be low quality and should be removed.
If the answer to any of the above questions is no, then have the links removed. Other things you can look at are as follows:
  • Are you using money terms and anchor text in the article? - This is ok as long as it's not overdone and as long as it's genuine. You have to use common sense when it comes to looking at blog posts. If it could be considered a spam article, remove it to stay on the safe side).
  • Are there lots of keywords stuffed into the article? - If your article reads like something that's been written for a search engine, remove it
If the answers to the above are yes, then remove the articles.

Link Networks:


Link networks are bad news, but many 'SEO' agencies still use them. You can use some of the metrics gathered in the reports above to detect link networks. For example:
  • IP Address - If the IP address is the same on multiple different domains, this is a sign that you could be part of a link network. You could run a quick whois search on the various domains to see who the owner is. We've carried out link removal campaigns on clients' websites where over 150 domains are from the same IP address.
  • Whois Searches - By running a whois search on a domain name, you can see who the registered owner of the domain is (in most cases). If you find that the same owner owns a large number of domains pointing to your site are from the same owner, this should send danger signals.
  • Google Analytics IDs - Look at the source code of the sites and search for the Google analytics code. These follow the format UA-XXXXXXXX-X (where the X's are replaced with numbers). If you find the same UA- code being used on multiple sites, this shows signs of a link network.
  • Site Design - Does the site look exactly the same as other sites pointing to your domain? This can sometimes be the same links in the footer, a 'sponsored company' or even just the same look and layout. If so, it's possibly built by the same person and an example of a network.
Some of the tools listed further down in this article will help you to speed up the locating of link networks by automatically pulling in Whois Results, IP address and Analytics Details for each domain. You can then use their system, or software such as Excel to manually filter through the data and spot offending links straight away.

Over Optimised Anchor Text:

If you have articles out on the internet which just contain anchor text pointing to keywords you want to appear on Google for, then these need removing. There has been a lot of talk online about the correct 'Anchor text ratio' to have for brand vs commercial anchor text terms, but I don't think you should think of SEO in this way.

Instead, focus on building your brand. If it's appropriate to link to your money keywords, or keywords surrounding the money terms, link to it. If your trying to build your brand, you'll find that you're linking to your brand name and it's URL more regularly and you're going to have a much more natural and organic link profile.

Tip: Use Eppie Vojt's Link Detective tool to see what your current link profile looks like. You'll be able to see very quickly which keywords you've been targeting too regularly.

If your anchor text is only targeting money terms, remove the links. Chances are, it's probably not the best article / content anyway. If it is, it's probably worthy of a money term link.

Link Exchanges:

This is a fairly old tactic these days, but many people still do it. Don't create a page on your site called links and swap these with other peoples 'links' pages. This creates a huge footprint of spammy link building which should be avoided.

Press Releases:

I still believe press releases are a great way to carry out SEO, but only if they're done correctly. If your press release is simply put out there and contains 3-5 links back to your site with a money keyword as the anchor text, you're doing it all wrong.

Press releases should be used to build your brand, shout out to the world about what your company is doing and only used when you have something relevant to tell people. If you've just received investment from a company, helped out your local community or had your best ever years profits, then write a really good quality press release that people find engaging and interesting. You can then put in links to your website under its brand name or the website URL.

If you are adding money terms in, I'd recommend adding a nofollow attribute in order to stay on the safe side as you can bet this is where Google will be targeting very soon. If you've got press releases out there which are blatantly spam with no real value, have them removed.

Malware:


A simple one. If the site contains malware, have your link removed. Alternatively, if it's a site that you know is hugely reputable, contact the owner and have them fix the problem as soon as possible.
The above section hopefully gives you an idea of how to spot some poor quality links. It's not a definitive guide, but it should give you an idea of some of the more common sections we come across when carrying out work for clients.

The biggest tip I would be able to provide is use common sense. Deep down, you'll know if a link is worth having or not. Sometimes likes are good, but they may need a nofollow attribute adding to them. So just work through the list systematically and put yourself in Google's shoes.

All you need to do is go through each link manually and make a note of whether it needs to be removed or not. If you consider it a safe link, mark it so you know not to remove the link. You could do this either by a colour marker, or just by adding a piece of text next to each link. If you've got a lot of colours going on, make sure to make a sheet at the bottom called 'Key' so that the Google employees can see what work you've carried out.

Tools for making your life easier...

There are a lot of tools available online that say they'll help to identify spammy links in bulk. Some of these tools are brilliant for speeding up analysis, but I wouldn't rely on them 100%. I've put some comments below on systems we've used in the past and my thoughts on using them.

Websites / Tools available for full link removal:


This is probably one of the better tools we've used over the past year. The detox tool assigns a risk score to each of your backlinks which it pulls from 22 different link sources. In addition to this, it will pull in lots of statistics about the links and will try to retrieve an email address / contact name for the owner of the offending website. Personally, I've found the contact finder to be brilliant and some of the additional metrics (such as IP address of the website and domain registrant) have helped hugely in discovering link networks. They still have a lot of work to do as far as the link scores are concerned. Sometimes, the 'Very Low Risk' links are obviously spammy and should be removed, so if you use this tool, you need to be very careful and double-check the links that you're removing. Overall, a very good system though.


I've never actually used Link Risk, but I've heard some good things about it. The team behind Link Risk are certainly very talented and know their stuff, so I can't really comment on the results seen with it.


A new tool on the block - Linkquidator offer a 30 day free trial with a limited number of backlinks. I have signed up to look at the tool but haven't had chance to give it a full test drive.


I've used Remove'em on a couple of campaigns we've done for clients as a tool to help us identify bad links. Personally, I found the interface very clunky in terms of outreach to webmasters and a very slow process. As for uncovering bad links, it did a fairly good job of this but didn't show me the safe links so it is hard to see how many it may have missed. I found the outreach to webmasters very slow with this system and would opt to use mail instead for speed.


Rmoov have personally tried not to identify which links are classed as spammy. Personally, the idea behind this system is brilliant, instead, it just helps to speed up the process of contacting webmasters by pulling contact information for each domain, allowing you to create and send emails then following up each of the emails with reminders. It will check the status of your links periodically and record each step of the process in a Google Doc.

Barry Schwartz from Search Engine Roundtable has done a brilliant write-up on link tools identifying toxic links. It really highlights that manual review is still necessary when using tools such as the above and you should never rely on them 100%.

Our agency Pinpoint Designs is currently working on a link removal system called Peel App which we're hoping will launch in 2014. Whilst we still push that manual research is required, we'll be trying to make our system identify spam links in bulk as we do feel there are still algorithms we can use to detect this better than some of the tools already on the market.

For More Info about Google Penalty Removal