The topic of social media measurement is almost as hot as the topic of social media. With only a few years of consistent data, we still remain in the shadow of the econometric models of the olden days, built for measuring the outcomes of PR and branding efforts.
The novelty and uncertainty of the field certainly haven’t stopped the burgeoning cottage industry of self-inaugurated gurus. This combination of ambiguity and hucksterism might scare off the ROI-driven marketer.
Now I am certainly not a social media marketing nay-sayer. Like most marketers, my gut tells me that there’s great opportunity here. However, the scientist in me demands evidence. And in business, evidence is ultimately in the ROI.
Do ROI and Social Media go together?
I was quite perplexed by one author’s argument that while social media marketing creates value, it may not deliver an ROI. I will leave the debate about whether social media marketing should deliver an ROI in the first place to another time. Today, I wanted to turn to a small sliver of a large study that MarketingSherpa published earlier this year in its Social Media Benchmark Guide.
This chart (click on chart to enlarge) displays how frequently various metrics are utilized by marketers as they attempt to quantify the effect of their social media efforts. My immediate impression was that there were broadly two types of metrics listed here:
- the more traditional website analytics and bottom-line-related measurements and
- buzzword-laden, social media-specific measurements with intuitive, but likely only anecdotal, relationships with outcomes.
What this chart wasn’t telling me was whether marketers were likely to mix these approaches, or were loyal to either one or the other. I enlisted MarketingExperiments’ experienced research analyst and statistics guru, Arturo Silva, to help get a little deeper into the data.
What Marketers Tend to Do
Using principal component analysis, he was able to paint a different picture from the more flat utilization frequency account. Without getting into the technical details of the loading plot, what this diagram (click on diagram to enlarge) shows us is how likely each of the responses above are to be given in conjunction with one another. In other words, which activities these marketers are likely to measure together.
The vectors indeed bunched up quite nicely. Leads Generated, Search Engine Rankings, Visitors and Sources of Traffic, and Sales Conversions or Other ROI Metrics are grouped together toward the top (by the way, the exact direction of the vectors here is irrelevant—what’s important is their confluence).
Network Size, Competitive Share of Coverage, Engagement with Influentials, and Progress toward Social Media Objectives also were tightly grouped. This means that if a marketer was measuring network size, she was also likely measuring the other three items I just listed, and was less likely to measure the first four.
ROI vs. non-ROI Metrics
Altogether, even though the non-ROI metrics are not all plotted next to each other, they stand in stark contrast to the more traditional and ROI-based ones. That is, marketers are typically looking at either one set or the other.
I am sure that a big part of the reason for this separation has to do with the tools that marketers use. Traditional analytics packages have little or no support for social media measurement, and conversely the new crop of social media management tools lack web analytics components and don’t connect with transactional data. The converse may be true as well—marketers choose their tools based on their interest in either side of the story.
Measure what matters most
What concerned me was how poorly some of the metrics that I would consider critical for marketers, like Leads Generated (for B2B) and Sales Conversion (for everyone) compared with measurements like Network Size and Sentiment, which haven’t proven to be predictors of bottom-line outcomes.
Paris Hilton may be considered a highly trusted influencer according to some unscrupulous Twitter data-crunching tools, but aren’t her Twitter stats just a reflection of the pre-existing celebrity status? Twitter stats (and I am focusing on Twitter because its simplicity makes the new metrics easier to understand, not just because it’s an easy target for pundits) are a measurement of reach, but not of impact. Content analysis tools can measure sentiment of comments, but not their effect on the business.
Intuitively we know that more reach means more impact, and nicer comments mean more satisfied customers (who will influence others). However, measuring the impact of each would require either taking a deep dive into the psyches of a large number of social media participants, or (more realistically) looking at how all the metrics, all the way down to resulting changes in revenues and expenses, fluctuate in response to the changes in the social media end (or rather, top) of the funnel.
So how do you determine the ROI of social media?
In today’s live web clinic, MarketingSherpa’s Research Director, Sergio Balegno, will join me in discussing how the value of social media activities can be derived from bottom-line results, giving business-level meaning to intermediate metrics like Quality of Commentary.
I want your feedback as well. Leave a comment and let me know how you measure your social media efforts. Our favorite comment will win a free seat (a $499 value) at an upcoming stop of the 2010 Online Marketing ROI Tour.
UPDATE: Congratulations to Jon Rognerud, our favorite commenter and winner of the free seat at an upcoming stop of the Tour.
This is why most social media marketing is so, well, outright bad. To go back to that study by 
I would suggest deeper-reach strategies, starting from understanding your target audience and getting involved in related Facebook Groups. Through meaningful conversation, you can introduce them to your Page (
In what amounts to five days of epic conversations, presentations and social media events disguised as a “technology conference,” the world (and all of 
”
Direct selling on a social network is difficult. The best way to sell is to replicate the offline world to a certain extent by signing up online agents. The same people who would host a cosmetics party or a Tupperware party are natural networkers who will have large social networks on all of the primary platforms.
I’m a pragmatist. I’ll leave my personal biases at the door any day in favor of solid metrics combined with scientific experimentation that shows what really works.
When using