Demonstrating return on investment (ROI) of social media in education – are we really delivering?

Last week I discussed the question ‘What are we actually trying to achieve with social media?

I always have concerns with approaches that just focus on the number of fans/followers/views/retweets, etc as the only measure of ‘success’ in the use of social media. I promised I would blog more about this and today the prompt certainly arrived.

I was delighted to see some of my favorite US tweeters sharing a blog post bearing the title ‘Universities are out to prove the ROI of social media’. Now we’re getting to the meat, I confidently thought to myself. This is what we’ve been waiting for and working towards! And started reading with high expectation and hope…

I’m sadly, however, left feeling disappointed by the false promise of the title. Instead of really getting to the meat of ROI, it focuses instead on another study of university use of social media. It reports a number of findings of the study, but the majority of those findings are purely anecdotal and not based on any real evidence of whether social media is actually delivering a return on investment for them. In fact, only one of the bullets in my opinion qualifies as coming close to measuring return on investment:

Reduced costs for traditional media are attributed to use of social media. Schools report 33% less spent on printing, 24% less spent on newspaper ads and 17% less spent on radio and TV ads.

This only qualifies for the ROI argument because, if you look at how we measure ROI (i.e. the actual equation to demonstrate ROI as a per cent statistic) then it’s true to say that if expenditure decreases, and income (return) remains the same, then the ROI figure will increase.

Calculating ROI:
(Income or gain – Expenditure)
Expenditure

However, this only assumes a positive ROI if they haven’t spent more on social media than they would have spent on traditional media (it claims they have reduced traditional media spend, but it doesn’t tell us how much has been spent on social media effort so we can’t actually run the ROI equation just yet).

The rest of the summary then focuses on anecdotal evidence, but no actual ROI evidence:

  • One in three schools say social media is more efficient than traditional media in reaching target audience … They say this, but do they know this backed by actual evidence?
  • 92% say it’s worth the investment … But have they quantified the investment and worked out the actual ROI or, again, is this just being reported on a general feeling that it’s a better approach?

I believe more than most people do in the power and benefit of social media. In fact, I’m one of the most passionate advocates of how it can transform (and how it is transforming) education. However, I think we risk the credibility of the communications profession if figures like these are interpreted as the sector leading the way in demonstrating ROI.

So, we come back to the question again of how can we actually measure ROI of social media? In reality, I don’t believe that we can (yet). Why? Because we are dealing with an immensely complex picture and to try to extract just the social media element of our communications activity to measure that component alone is immensely challenging.

Let’s look at the student journey alone…

This is a template that I put together a while ago to help participants in my workshops and ourselves in our strategy development work think about an integrated approach to social media for student engagement, and to also think about measurement and evaluation at different stages in the audience engagement journey and how it all fits together. I believe that the more we break the stages of the engagement journey down, the more we can break down and measure the impact of those activities at each stage, and get closer to demonstrating ROI.

If we try to measure the success of our SEO activities designed to raise awareness of our university or school with people who have never heard of us before, for example, by comparing that spend to the total income from actual admissions, then aren’t we missing the point of the SEO activity in the first place? There the objective was to raise awareness and make the right people know about you, possibly driving them to your website or encouraging them to order a prospectus or attend an open day. Then the objective of your prospectus is to raise their interest in you as an organisation and to make them want to attend a visit day. Then the objective of the visit day is to make them feel passionate about your organisation, to believe you are credible and to want to apply to study there. And so on. See my earlier blog post here on the importance of objective setting.

If we break the journey apart, we can tailor a package of analytics, insights, measurements and evaluation that can ultimately help us to understand that impact on ROI.

A group of admissions officers suggesting that they believe social media works might convince some people of its effectiveness, but any senior manager or finance director worth their salt shouldn’t accept that ‘gut feeling’ as evidence of ROI. We must do more. So much more.

The communications profession is making great and significant steps at the moment to develop a framework for social media measurement. See the recent AMEC update on the (global) work to develop social media measurement standards (follow #smmstandards for more discussion and updates). I also find the AMEC Valid Metrics for PR Measurement work useful for thinking about social media evaluation.

Underpinning all of this is the need to focus on outcomes, not outputs. As the great Mike Stoner just pointed out in a tweet to me, efficient doesn’t equate to effective. We need to measure the effectiveness, not just the efficiencies. And if we’re to persuade senior managers to invest in this, then we need to produce some pretty good evidence, not a gut feeling.

If only we had the funding to try to do some really in-depth analysis for the sector (oh, how I wish, since I already know who’s in my A-Team to lead that study! Now, how do we finance it?)

8 Comments

  1. John Goode
    August 2, 2012

    We’ve recently been investigating enquiries management systems (context = HE). I’m saddened to see or rather *not see* evidence that the well known sector providers don’t have Social Media integration on their roadmaps.

    Many of our key target audiences will want to engage via Twitter and/or Facebook. A university needs to track enquiries all the way through to applications and then on to Alumni. It needs joined-up *systems thinking* but I fear 2 things: 1. we can’t measure unless the systems are in place. 2. No measurement, no ROI.

    HE has to look beyond the “what’s historically worked.” HE has to take the large and difficult to swallow pill called *entrepreneurship* to be taken with a healthy dose of innovation.

    Reason: demanding ROI proof is the equivalent to pulling oneself up with ones own bootstraps.

    Reply
  2. Social media and #highered: Where’s the ROI? « Higher Ed Marketing
    August 4, 2012

    [...] post on his own site. (Tracy Playe of Pickle Jar Communications — @picklejar – also does a fine job of dissecting the Massachusetts Dartmouth study.) Both Stoner’s comment and his blog post are worth [...]

    Reply
  3. Andrew Careaga
    August 4, 2012

    I’m so glad you picked up on this survey and added your perspective. One observation I made in my post about the same study has to do with the bullet point from the study that you shared — the one you said came closest to actually measuring ROI. The implication of that data point is that a reduction in media spend equals or is offset by an increase in investment in social media. But that may not be the case. Also, just because the vast majority of admissions officers surveyed believe that social media is worth the investment doesn’t necessarily mean they’re investing adequately in social media, or for the right reasons.

    Maybe with your post, and Michael Stoner’s, and mine, and perhaps others chiming in, we will actually start to get to the meat of social media ROI for higher education.

    Reply
  4. Tracy Playle
    August 5, 2012

    Absolutely Andrew. I think if there are a group of us out there with a focused effort on doing this right and not settling for meaningless measures, then we have a chance of pulling the rest of the sector along with us. I know some organisations this side of the pond who are really committed to good measurement and evaluation. We can help them apply that to their social media activities and to activities where social media is part of a wider integrated approach too. But I firmly believe that we have to be involved in the planning stages if we are ever to be able to evaluate in a meaningful way. In my mind we have to start with measurement and evaluation, not just think about it at the end.

    Reply
  5. Take Two: Social Media ROI in Higher Education « social @ edu
    August 20, 2012

    [...] social media ROI in higher education is complicated, it does not happen enough and warrants a university-wide effort.  I agree with all these points. Today though, I’d like to present my own take on this important [...]

    Reply
  6. Shannon Kelly
    August 20, 2012

    Tracy, I really enjoyed this post of yours. Thanks for putting it out there, my colleague and I are focusing on this on our blog, http://www.socialatedu, over the course of this week – would love to have your thoughts there as well.

    Reply
  7. 3 Essential Social Media Measurement Tools for Your School
    August 23, 2012

    [...] The return on investment (ROI) of social media initiatives is one of the current Holy Grails in higher ed. With a wide variety of marketing goals (increase lead generation and admissions, enhance retention and reputation, etc.) and what seems to be an infinity of metrics (comments, shared content, views, follow, etc.), it is virtually impossible to track the specific effect of all the social media initiatives going on at your school. So much so that some specialists are now questioning the very idea of a Social Media ROI in higher ed. [...]

    Reply
  8. Morgan Hammond
    October 24, 2012

    What about reach and frequency measures? Since the investment is time in social media, I can see why HE is jumping on the “it HAS to work” bandwagon, but we have to look at what is being measured. Why are we not measuring the reach and frequency of Social Media the same way we measure “leads” at college fairs and booths? Do we really have to know WHO it is we are reaching with our brand and follow them all the way through the system? We don’t get the name of every potential student reached through a flyer/pamphlet at a college fair or a billboard on the interstate and trace them through the system, so why is it any different with social media?

    Reply

Leave a Reply