Posted by: Marshall Sponder | February 16, 2008

Engagement Metrics Methodology and Measurement

I figured it out – did it – but Eric Peterson and Kevin Mannion have done all the legwork – not that my idea is easy (it’s not)- but it is a solution to the problem of How the New Engagement Metrics Can Impact Advertising Decisions where Peterson is quoted in regards to who will produce and verify the data?

“…Probably not comScore or Nielsen,” says Peterson. The problem is the panel approach. Projecting results from a small sample of users is increasingly controversial looking at simple metrics such as reach and composition. It will not work for engagement, as we have described it here. To get at the depth and complexity of this engagement model, we need to look at publisher data. As we discussed in the first article in this series, the startup firm Quantcast holds special appeal. Quantcast has advocated a methodology that normalizes direct publisher data through its “mass inference” algorithm. When I outlined this approach to CEO Konrad Feldman, who views the work of Eric Peterson “with the highest respect,” Feldman said he believed that his company could in fact produce this kind of indexing. Are there other companies who can take the engagement model forward? I welcome all nominations.”

I think what we need is a commonly agreed upon Web Analytics Framework to be adopted by at least one or two of the major web analytics vendors that defines a set of engagement tags that can be embedded into actions of the site in a standardized way – Here’s the overall measures, according to the MediaPost article by Kevin Mannion:

  • Click Depth (content clicked on): Percentage of visitors who exceed average page views in a given content category. If 26% of visitors exceed, say, 3 page views, C = 26%.

let’s tag this event as “egm=1”

  • Loyalty (number of return visits over a longer period of time — say 12 months): see WebMD example above: L = 57%.

let’s tag this events as “egm=2”

  • Recency (number of return visits over a shorter period of time, say 1 month): If 5% of visitors return more than once a month, R = 5%.

let’s tag this event as “egm=3”

  • Duration (time of session): If a category of content sites records a 4.6 minute average session time and 19% of a specific site’s visitors spend more than 4.6 minutes, then D = 19% for that site.

let’s tag this event as “egm=4 – but I’m not sure this needs a tag – then again, the more consistent we are, the better”

  • Interactivity (defined actions taken with content-downloading, posting, attending a video or audiocast, etc): If 32% visitors take any one of these actions, I = 32%, during a specified time period.

let’s tag this event as “egm=5”  – this is really a measurement of a download, or clicking on a button of a widget – even there, identifying what it is and putting an agreed upon tag on it will make it easier to create an automatable engagement metrics that’s industry wide.

  • Subscription (commitment of name and business or personal info): Measures the percentage of visitors who have given registration information. If 21% of a site’s traffic can be identified by name and other submitted information, then S = 21%.

let’s tag this event as “egm=6” – point being – we want to keep all these events together – as part of one continuum – which will make it easier to come up with the score.So, what we’d first have to do, I think, to get this right, is set up a campaign with a defined set of conversion events and then come up with a standardized set of tagging that corresponds – you can almost get to a point where you have a Web Analyst who is certified to do this (I know WebTrends Score is going in this direction, in fact does some of this, but it’s a vendor solution whereas what we need is an industry solution).

And lets face it, the majority of people asking for “Engagement Metrics” … are Marketers and Advertisers – they’re the ones that cooked up the name in the first place, I think.

In fact, the more I think about it, the more convinced I am this “Engagement” mess started when Marketers were trying out their ideas of motivating visitors and tried to find some kind of corresponding Web Analytics underpinning, when there really wasn’t any (sure, there was page views per visit and time spent on site per visit/visitor, but that’s hardly a proxy for the kinds of engagement the brand marketers were looking for).

And then you’d see Marketing diagrams like the one I picked on a couple of weeks back in Social Media Measurement of Blog Success where a lot of what A framework for measuring blog success by Greg Verdino can’t be measaured today via Web Analytics today without some deliberate tagging and methodolgy – and who’s problem is that?

And here’s a chart I put together which might work, if we could just agree on a couple of hazy points:


Is it Web Analytics’s problem or is it Brand Marketers who just haven’t really figured out if what they’ve cooked up can be verified – or if it’s just a mental model of how they think people behave – but it’s unverifiable.

The model that is beginning to emerge and which the WAA and the IAB how to work on together – will probably end up having a set of standardized measures overlaid onto any event you want to measure – which will need to be tagged in some way and then web analytics, to a large extent, will be able to measure customer or visitor engagement.

But till then, I’m sure we’ll get a lot of ideas of what we should measure and what would be nice to measure – except, we’ll not be able to fully measure it – or even agree on the definition – which is where we have been for the last couple of years.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s


%d bloggers like this: