The market for social media monitoring is saturated and its nigh on impossible to differentiate one vendor from another. Yet some of the results from automated analysis are typically only accurate 50 to 60 per cent of the time.
I recently had the opportunity to ask Metrica’s managing director Richard Bagnall, a veteran of the PR monitoring and evaluation industry, to make sense of the online monitoring market during an interview for the book that Steve Earl and I are writing.
Our conversation was timely. Bagnall had recently finished overseeing a project that analysed and made sense of over 100,000 pieces of social media coverage from a 18-month period for a large IT client that had been supplied by one of the leading US social media monitoring companies.
Bagnall presented the results at the recent PR Moment B2B social media conference. Here’s his presentation.
Utilising the specialist boolean search team from their sister company Durrants and some topic clustering technologies developed in house, Bagnall’s team reduced the number of relevant client mentions by a third. Metrica’s specialist media analysts then read and evaluated all of the remaining coverage with the result that the data reported back to the client showed that just 33,000 of the posts were actually relevant.
Man vs machine
In parallel Metrica invited all of the leading social media monitoring tools to partake in a trial to prove which would be the best company with which to partner. It challenged them all to undertake the same search string over a two week time period and then the Metrica team compared the results. The volumes returned were widely different by as much as 50 per cent.
Next Metrica took 1,500 clips from the search results and had media evaluation researchers analyse them for relevance and sentiment not once or twice, but on three occasions for positive, negative or neutral sentiment. The results were benchmarked against those from all of the social media measurement companies to really get a feeling for which companies were providing the most robust data.
Incredibly, the worst performer was accurate only 29 per cent of the time with their favourability ranking. That’s 4 per cent worst than the 1-in-3 wager of leaving the result to pure chance. The next best got the answer right 52 per cent of the time and the best was 58 per cent accurate.
“If you offer me a lift and there’s a 4-in-10 chance I won’t make it home I’m not keen to get in in your car. Platform based monitoring tools alone aren’t good enough yet for automated analysis. For the foreseeable future we will need humans to provide context and meaning to the language and the conversation,” say Bagnall.
Brandwatch deal
But early this week Metrica invested in Brandwatch, a Brighton based social media measurement tool, via its parent company Durrants. I caught up Bagnall and asked him to explain the apparent about turn.
“PRs working with social media tend to need two things from their monitoring, namelyspeed of sourcing to help them manage potential issues and breadth of content when looking to get a wrap up on a proactive campaign. For both speed and breadth of quality Brandwatch excelled against the competition.
”PRs also need to measure what matters and to do it with meaningful and credible metrics. This is something that Metrica has been providing clients for over 17 years. We will now be providing this value-added service to our clients working with the Brandwatch data to ensure that our clients get the best of all media.
The Brandwatch deal will help propel their already leading automatic metrics as our media analysts provide literally millions of clips worth of data feedback to propel their machine learning algorithms to new levels of accuracy.”
Durrants has acquired Gorkana ![Reblog this post [with Zemanta]](http://img.zemanta.com/reblog_e.png?x-id=2e05389c-ce93-4892-ab49-7d807c5e34e2)
![Reblog this post [with Zemanta]](http://img.zemanta.com/reblog_e.png?x-id=fcb4deb5-e4e5-43b0-9339-2abeb4754958)
