Alexa Rank – Part Three

This is my third post responding to all the internet grumbling about Alexa being useless. You can view my second post, here.

The Alexa Critics are Wrong

Alexa rank is the mean of your site impressions and site reach (unique visitors). Alexa only counts one page view per page…Per day…For each unique visitor. Alexa Toolbars are matched to IP address. So gaming attempts like Jason Calacanis’s are 100% wrong, as are html refresh scripts. I explained this in more detail in part two. Basically, hoping to game Alexa by running a refresh script is like hoping to earn Google adsense money by reclicking all your ads. In 2007, it ain’t gonna happen.

We know the Alexa algorhythm has improved because Alexa gaming sites like Alexa-Booster, Alexa Surf, and SiteReturn have all gone out of business. These sites flourished in 2004 by charging up to $5,000 to “guarantee” your website an Alexa rank in the top 5,000.

Statistical Probability Sampling

Now in Part Three we get into the mathematics of probability, and why I believe Alexa is far more accurate than people think.

The Gallup Poll’s average sample size is 1,000 people per survey. And yet they’ve predicted every presidential race in the past 50 years with a margin of error of 3%. Gallup estimates that doubling its sample size to 2,000 would narrow the margin of error to 2%, but raise their polling costs by 100%. So they’ve settled at 1,000 as a typical sample size.

Gallup Polls are overwhelmingly accurate due to equal probability of sampling (more on this below). Mathematically, Gallup has spent half a century proving that 1,000 people can accurately model the behavior of 300,000,000 Americans 97% of the time. That’s .0003% of the population!

Nielson samples 5,000 households to determine the TV viewing habits of 99,000,000 households. Nielson ratings make or break TV shows every season. Their statistical sample is .005% of the population!

Now let’s consider that over 10 million Alexa toolbars have been downloaded, and there are approximately 1 billion worldwide internet users. Alexa’s statistical sample is therefore 1%. Percentagewise, this is 33 times bigger than a Gallup Poll Sample.

Equal Probability of Sample

At this point, a statistician might assert that Alexa Toolbar users are a self-selecting sample, while the Gallup Poll panel is randomly selected by phone calls. But as a current member of the Gallup Poll panel, I can testify that becoming a member of the Gallup Poll is a highly self-selecting process. It takes a lot of time to complete the applications and surveys and my understanding is that most people quit the Gallup Poll fairly rapidly when they realize how much work is required. In terms of political passion and awareness, a Gallup Poll panelist is by no means a statistical representive of your average American Joe.

Nielson carefully chooses a 5,000 home statistical sample to match the demography of the American census bureau. Meanwhile, you could assume that the Alexa toolbar is primarily downloaded by webmasters with a site to shill! Furthermore, the toolbar is only available for Internet Explorer and Firefox. However, given that Alexa is drawing from a statistical sample an order of magnitude larger than Nielson’s, I imagine these errors are more than smoothed. When the most cynical SEO geek downloads the toolbar, he is promoting the traffic stats of the hundreds of sites he visits besides his own.

People need to accept the fact that Alexa is at least somewhat useful. And as I’ve stated before, it is the least bad traffic comparison system we’ve got.

*You can download the Alexa toolbar for Internet Explorer and Firefox here*