Better quality classifiers for social media content: Crowdsourcing with decision trees

A basic task in sentiment analysis is classifying the polarity of a given text at the document, sentence, or feature/aspect level whether the expressed opinion in a document, a sentence or an entity feature/aspect is positive, negative, or neutral. as social media use grows and increasingly becomes a forum for social debate in politics, social issues, sports, and brand sentiment; accurately classifying social media sentiment remains an important computational challenge. Social media posts present numerous challenges for text classification. This paper presents an approach to introduce guided decision trees into the design of a crowdsourcing platform to extract additional data features, reduce task cognitive complexity, and improve the quality of the resulting labeled text corpus. We compare the quality of the proposed approach with off-the-shelf sentiment classifiers and a crowdsourced solution without a decision tree using a tweet sample from the social media firestorm #CancelColbert. We find that the proposed crowdsource with decision tree approach results in a training corpus with higher quality, necessary for effective classification of social media content.