Social media may be the world’s largest source of Open Data. Between blog posts, online reviews, and a couple of billion tweets every week, social media produces an unending stream of opinion and information about products, services, and events. Sentiment analysis is a new method for turning social media into computable data that can be used to analyze trends, gauge consumer opinion, or guide political campaigns. It’s a way to analyze social media for the “sentiments” it reflects – the virtual crowd’s opinions, emotions, and feedback about companies, politicians, entertainers, and anything else that’s a subject of social media discussion. Recently, I wrote about how the company newBrandAnalytics is using sentiment analysis to gauge public agencies in Washington, D.C.
To learn more about sentiment analysis, I recently went to Takoma Park, Maryland, to see Seth Grimes, founder of the consulting firm Alta Plana Corporation and a recognized industry guru on the subject. In addition to consulting with individual clients, Grimes runs symposia on sentiment analysis twice a year. The next one will be held in New York on March 6, 2014, preceded by a day of “Human Analytics” workshops covering Intelligent Customer Experience, Digital Measurement, Sentiment Analysis, and Technology & Innovation. The program’s still in formation and should be out in early November, with a full roster of speakers, at http://sentimentsymposium.com
Grimes has a diverse background: he worked for the Organization for Economic Cooperation and Development in the mid-1990s, helped the U.S. government with the 2000 census, and worked on other government projects in the United States and the United Kingdom. Around 2002, he became interested in text analysis, a decades-old field that was showing new potential and that forms the basis of sentiment analysis today. You can listen to a podcast of my interview with him here:
“The idea is to use computer software to do what people do, which is read text and make sense of it in some kind of situational framework,” Grimes told me. “If you’re doing marketing,” he continued, you want to know, “‘What do people have to say about my company’s products? What are the particular flaws in the product? What do they really like? What do they think about the pricing, about the customer service, for not only my products but my competitors’?’ There’s a huge volume of information” in text that can be extracted and analyzed from the Web. Doing that, and doing it well, is the essence of sentiment analysis.
While sentiment analysis is promising, “the technologies are not mature by any means,” said Grimes—and some companies that look into it may be disappointed by simplistic applications. “For a lot of people,” he said, “sentiment analysis just means whether a particular tweet or review on TripAdvisor or article in the newspaper is positive or negative. To me that is a very stilted point of view. If you take the word sentiment, you might picture Jane Austen, and the word sentiment to someone like that is very broad. It encompasses emotion and mood and attitude and opinion.”
To understand sentiment, picture Jane Austen.
Analytic tools that just look for positive or negative words can be entirely misleading if they miss important context. According to Grimes, “What’s positive to you might be negative to me. If you’re with Toyota and General Motors has a broad recall, then that’s actually positive news for you. . . . A more sophisticated analysis,” said Grimes, “will decompose a message or document into particular elements. Those elements could be the names of persons or places or companies or products or concepts or themes. More capable tools will extract those entities or concepts or themes, and they will analyze the sentiment attached to each element.”
Sentiment analysis can also be used for competitive intelligence and strategy. One European consulting firm, for example, has done this kind of analysis for telecom companies. When negative comments about a competitor’s service are spiking on social media, it may be an ideal time for a telecom company to run ads touting its own network’s reliability.
You can use sentiment analysis to study political unrest, antismoking ads, or idiomatic Chinese.
I’m looking forward to the next sentiment analysis symposium in March: the one I went to in New York last May was fascinating. Among other things, the speakers talked about using sentiment analysis to:
- Predict political unrest in Kashmir and the results of a national election in Pakistan
- Determine how effective antismoking scare ads are
- Find and fix problems with online payment systems
- Do text analysis of idiomatic Chinese
- Lay the data-based groundwork for campaigns to appeal to customers emotionally
New tools may make sentiment analysis more accurate in the years to come. Some companies are now figuring out how to analyze audio recordings not only for their words, but also for cues to emotional content like rapid speech, raised voices, or one speaker interrupting another. They’re getting richer insights into communications as diverse as doctor-patient dialogues and customer-service calls. At the same time, several startups are developing facial analysis technology to decode emotion from video feeds.
In a way, sentiment analysis is starting to come full circle. It began by deconstructing written documents to extract atoms of meaning from human communication. It may develop in the future by adding in the cues and signals that make our communication truly human.
- Joel Gurin, Founder and Editor, OpenDataNow.com