One of the biggest challenges in SEO is measuring success in a way that both the client and the marketer agree on. As recently as five years ago, this wasn't a challenge at all. Back then, we had these wonderfully reliable, ironclad numbers called rankings. The marketer would do their keyword research and the client would sign off on X number of keywords.
The marketer would then set to work and regularly keep the client updated on how their efforts are affecting rankings. They would tout statistics such as the clickthrough rate of each position in the top 10, how many people actually advance to page 2 and so on.
Rankings go up = happy client
Rankings go down = mad client
Well, gone are the days of such simplicity. Today, any SEO whose primary benefit statement is along the lines of "We can get you #1 on Google!" quite frankly, doesn't know their shit. Or, even worse, knows their shit and preys on clients who don't.
The fact of the matter is that rankings are a terrible way to measure success in today's search landscape. Allow me to list just a few of the factors that basically make these measurements moot:
- Personalization: In case you haven't noticed, search engines are getting pretty damn good at getting to know you. This is based on several factors which all work together to give you a more personalized experience. For example, you might search the name of a TV show one minute, then start to type in the name of an actor from that show only to discover that Google already guessed who you were going for, despite not being a hugely famous actor. Based on your history and recent clicks, they have made that association.
- Localization: This is a huge one. So much so, that it makes measuring rankings for keywords with any shred of local intent impossible. Sure, searching "pizza" will produce completely different results if searching from Seattle when compared to Atlanta, but even searches like "sunglasses" will generate local results. Additionally, the number of Google+ Local results Google chooses to display (3-, 5- ,7-packs, etc.) varies as well, which can push your normal organic results all over the place.
- Signed In/Signed Out: This goes along with personalization, but whether or not you're signed in to Google impacts your search results as they will reference your web history and habits.
- Social: Yup, social media impacts rankings. The heaviest associations are with Google+ and Google (of course) and with Facebook and Bing. If your friends are engaging with certain pages and brands, then that can influence who you see in the top spots.
With so many factors that skew your search results, how can anyone possibly report on keyword rankings? In order to get the "cleanest" search results (meaning, the least skewed based on these factors), you need to search from a private browser, signed out of Google or Bing, your location set to "USA," your curtains drawn, doors locked...okay you get the point.
There are several rank checker extensions and software that do their best to filter out these factors and give a more vanilla look at your rankings, but none are perfect.
Why do we cling to these metrics? Why do promises of being #1 still grip businesses? Is it like the guarantee on the box?
You figure you put that little ranking under your pillow at night the guarantee fairy will come by and leave a quarter. Amirite, Ted?
A Quick Disclaimer
Warning: Do not assume this means you should never pay attention to rankings.
Of course ranking well leads to more organic traffic. You should keep an eye on your rankings in order to watch out for fluctuation in certain areas which may signify trouble. What I'm saying is don't let your project be bogged down by why one keyword dropped from #2 to #4 and don't use these as your primary gauge for success.
So how, you ask, should we measure success in an SEO project? The best way is to look in your analytics. Specifically, you should look at organic search traffic growth, conversions from organic search and engagement stats such as bounce rate, average time on site, etc.; again, all relative to organic search traffic.
These metrics will give you a much more concrete basis on how your SEO efforts as a whole are impacting your bottom line. In reality, a single keyword bouncing around the top 10 will have little impact on your total traffic and conversions (if it does, you need to look into a more diversified and extensive strategy).
This also better positions you to set more useful goals. For example, a 10% increase in organic search traffic in the next 6 months or a 25% increase in conversions resulting from organic search in the next year is much more bottom-line focused than "I want to be #1 for as many keywords as possible."
Rankings are a vanity metric. Every business owner, CEO and marketing manager like to see their competition below them. However, getting hung up on this can absolutely derail your search engine optimization efforts. Keep an eye on rankings, but set your goals based on analytics.
Update: It just so happens that SEOmoz posted a Whiteboard Friday video this morning on this exact subject. I highly recommend giving it a look.