Tapping into Social Media for Business Intelligence

Customer Experience
Customer Experience
Five lessons for successfully harvesting social media data for customer insight.

Social media conversations and the data found within them are growing exponentially, providing businesses with an incredible opportunity to get actionable intelligence from today's real-time Web. While most businesses understand the potential insight and competitive advantage that the real-time Web promises, traditional business intelligence (BI) applications aren't built to access and aggregate unstructured data coming from blogs, forums, Facebook, and Twitter. Additionally, the amount of "noise" in social media poses further problems for businesses to consume real-time data in a way that's actionable and meaningful.

More than 5 billion Web sites produce vast amounts of useful information, including customer opinion on products, services, market trends, and competitors, but they're predominantly found as unstructured text in social media sites. The difficult part is efficiently gathering the right data at the right time, transforming it into actionable intelligence, and loading it into BI tools for analysis. It's no surprise that without an easy way to consume the rapidly multiplying mass of information in an organized fashion, many enterprises simply write off this treasure trove of customer insight for various reasons including either a presumed lack of credibility or reliability, or an underestimation of its value.

With more intelligent, more accurate real-time Web data processing, new methods for working with business information are possible. Marketers and business analysts can then spend their time extracting greater intelligence from the data and less time worrying about collecting or accessing it.

Here are five lessons for harvesting social media data for customer insight and business intelligence:

  1. Data trumps gut feel. Recently, the Dancing with the Stars reality TV show monitored social media conversations to predict the results. For several weeks TV star Kate Gosselin racked up the most negative sentiment of all the contestants (90 percent at one point). Popular opinion was that if she wasn't voted off, the show was surely "fixed." However, the data painted a different picture. Forty percent of all DWTS conversations mentioned Kate, and while 95 percent was negative, Kate's 5 percent positive was still greater than several of the other contestants. Most people predicted her immediate demise -- the gut prediction. But that wasn't the case. Her positive numbers kept her on the show for three more episodes even though she was clearly the worst dancer.
  2. Timing is critical. In this day of real-time Web data, businesses can't wait weeks for market research reports. It is now a simple task for a movie studio like Disney or Universal to collect social media sentiment minutes after a midnight premier of a newly released movie, compare the data to historical trends, and by morning predict the success of the movie.
  3. Eliminate the noise. It's easy to understand trends, changes in momentum, traffic volume, and customer sentiment; however, huge events like the BP oil spill or the Toyota car recall generate lots of collateral "noise." The bigger the event, product, issue, or scandal, the more noise. Businesses need to carefully evaluate the noise factor and establish filters for specific scenarios.
  4. All social media sentiment is not created equal. Depending on the source of data, not all public sentiment for products, companies, movies, or shows should be handled or weighed equally. A lengthy blog post by a respected author cannot be considered equal to a tweet or comment by an unknown user. Organizations need to understand their objective before gathering and analyzing the data.
  5. Don't look at data in a vacuum. Having knowledge of events and circumstances is critical to understanding and extracting the "intelligence" in Web data. Manual review of data ensures data quality and consistency. A balance between automated sentiment analysis and manual review needs to be struck. When using an automated sentiment analysis tool, companies should also weigh keywords differently. Automated tools can't distinguish sentiment as functional, emotional, or behavioral yet.

+ + + + + +

About the Author: Stefan Andreasen is the CTO and founder of Kapow Technologies