The Fudging of Internet Audience Measurement

Interesting story in the Register couple of days ago:

http://www.theregister.co.uk/2007/04/27/auditing_the_internet_auditors/

Quote:  "The Interactive Advertising Bureau this week sent letters to the two major Internet audience measurement services, comScore and Nielsen NetRatings, asking them to submit to a third-party audit of their measurement processes... The IAB didn’t spell out what the consequences of NOT agreeing to a third party audit would be, but they can’t be too dire, since both sides have so far resisted it"

Of course they've resisted it...

In the first half of 2004 before Hurricane Ivan turned our lives upside-down, I thought it would be good to get rated (measured) by ComScore. We see unique IP visits coming through the door and our reach placed us somewhere in the middle of the top 50 US properties at the time. I figured if ComScore audited our unique monthly visits, advertisers and agencies of the world would beat a path to my previously unknown company's door!

We were excited,  ComScore was excited.. while they didn't elaborate on how they were counting or what form of methodology they were using, at the end of a week, they were only able to measure 1/30 of our actual US traffic. It was a train wreck.  Why?  Most of our sites were to small to be seen.  ComScore can see one website with 1,000,000 visits a day but they couldn't see 100,000 websites with 10 visits a day. I presume this same dynamic plays at Nielsen and other rating services. In the end, we wound up getting creamed by a hurricane and abandoned the exercise of trying to repair their methodology.

That little vignette isn't the only problem with ComScore.. Traffic Marketing or Arbitrage inflates the traffic numbers across the majority of top US properties and renders a misleading picture.  It's one thing when visitors tune in to view a website on their own, entirely another when you pay third parties to dump visits on your doorstep by the busload. We could spend millions a month buying traffic from Google Adwords or Yahoo Search Marketing and then sell those targeted visits back to others in a revenue neutral manner. The by-product of this paper-trade is a raft of visitors passing across your network.. THAT would move the needle at rating services, but it's such a charade.  None of those visitors came on their own.. they were bought and paid for.

I think the rating services have a vested interest in continuing to report in this manner.  Would you want your the light of truth to shine down on your flawed metrics? I think a standardization or general push to cleanup imbalances is inevitable.. to that end I applaud the IAB for trying "something". Maybe they can publish a blacklist of non co-operative rating services at some point.

Comments

[Shane]

Couldn't Microsoft throw an Alexa clone into IE and get results that were worlds more accurate -- plus, for one of the first times, do something on the Internet that truly was far better than what they were copying?

***FS*** You know, that has 'always' puzzled me. Verisign could do it too from the DNS level.. VRSN knows the actual traffic on everything.. I just don't get it.

[Drewbert]

>VRSN knows the actual traffic on everything.. I just don't get it.

They know the traffic on non-existing domains, but correct results are cached thoughout the DNS - a DNS query for google.com or yahoo.com seldom makes it to one of Verisign's com zone servers - it's answered by the user's own machine (local cache) or the ISP's DNS cache.

If every query DID make it through to Verisign , they might actually have an excuse for a price rise! But they don't.

***FS*** true true Drew.. what about the A root.. couldn't they count at the root? Some percentage must fail-over?

[David Wrixon (aka Rubber Duck)]

There are certainly issues with Alexa in terms of producing representative results outside the USA.

If you want proof of how US centric Alexa is, then go to the foreign lanuage pages on their website.