•
The third condition: expand beyond the classification of positive/negative/neutral sentiment and highlight different emotions that users are expressing. To deliver our project goal – “analysis of user emotional responses to identify the opportunities for further development of the target product/company” we needed to capture a broader range of emotions than “good vs. bad.”
3.3.1. Data input methods of the tools Many of the tools we used did not allow large text or data input. Despite the great functionalities of Topsy and Sentiment.VIZ, we discovered that these tools were only suitable for analysis of tweets. We did contact the developers of Sentiment.VIZ to find out if it could support data from other sources or text input, but this functionality was not available. However, we still could take into account the results derived from Topsy and Sentiment.VIZ but only as supplementary information. The tools with sufficient input options were Lexalytics, word-‐processing.com, IMB Many Eyes, and Wordle. LIWC would have been an option, but there were limitations in how much data could be inputted using the demo version so it was not feasible for our analysis.
3.3.2. Validation of Sentiment Analysis We chose our final evaluation tools according to the availability of inputting large texts and how similar they were to human sentiment analysis. According to our first requirement, the tools were narrowed down to Word-‐Processing.com, Lexalytics, and Qdap (R-‐package). Then, by evaluating the quality of the sentiment analysis and their algorithms, we performed a manual sentiment of a random sample from our dataset (approximately 10% or 100 entries) and compared the results of each tool. We then calculated the Euclidian distance (see Figure 14) to the manual answer from each of the tools values to verify their sentiment analysis and evaluate which tool we should move forward with.
3.3.2.1.
Manual Analysis 100 cases
For validation of the various tools we used, we performed a manual sentiment evaluation. We read 10% (100 entries) of our AppStore reviews and gave each a value of -‐1 if the sentiment sounded negative; the value of +1 if the sentiment sounded positive; and 0 for neutral reviews.
3.3.2.2.
QDap – R package
Quantitative Discourse Analysis Package (QDap) is a statistical software tool originally designed for analysis of transcripts but is also suitable for other purposes such as sentiment analysis. Its functional possibilities are frequency counts of sentence types, words, sentences, turns of talk, syllables and other assorted analysis tasks. QDap was only used as a comparison tool for the manual analysis. QDap is one of the tested analysis tools in Table 3.
15