The Loudpixel Blog

Does Automated Sentiment Really Work?

When it comes to social media measurement and reporting, automated sentiment is often viewed as a miracle technology—one that saves time by replacing the need for any human intervention. But it’s important to look at this technology with a critical eye before accepting automated sentiment results as fact.

There are a few problems with the nature of natural language processing, which is behind automated sentiment. Even if a technology claims to truly understand the English language, it is still difficult for it to understand sarcasm, misspellings or slang—which are all prevalent across social media posts. I should also note that Biz360, one tool that tracks social media posts, pulled down it’s automated sentiment tracking for Twitter because there are not enough characters to give context for proper processing.

To show a firsthand look at how posts are being categorized through natural language processing, I took a quick look at posts related to “girl scout cookies” (a timely topic). The following are just a few examples of posts that ended up with the wrong categorization.

Marked as Negative:

Marked as Positive:

While the tools for conversation analysis are improving at a rapid pace, it’s important to remember that human analysis is still an important part of the research process.

Posted in Tools by Allie

  • http://www.symscio.com/ Mike Layton

    Hi Allie, you have some great examples here. On the flip side to your twitter examples, when NLP is used for articles and blogs posts, it attempts to apply sentiment as a whole based on the parts. The issue here is that sentiment is not a mathematical equation. I don’t care how many nice things are said about a company in an article, if it mentions that their product malfunctioned and caused bodily harm, it is negative.

    Thanks for sharing, Mike.