Enhancement of sentiment analysis scoring mechanism – A case study on Malaysian Airline industry / Rayvendran Visvalingam

Rayvendran , Visvalingam (2017) Enhancement of sentiment analysis scoring mechanism – A case study on Malaysian Airline industry / Rayvendran Visvalingam. Masters thesis, University of Malaya.

	PDF (The Candidate's Agreement) Restricted to Repository staff only Download (201Kb)
Preview	PDF (Thesis M.A) Download (2086Kb) | Preview

Abstract

Sentiment polarity calculation is a method to gage the strength of a sentiment extracted from a text. Many tools have been developed with their respective scoring mechanism in order to produce an effective sentiment score. Semantic Orientation Calculator (SO-CAL) is one of the lexicon-based tool that is incorporated with important features (such as intensifiers, negation and etc.) to calculate the sentiment polarity of a text. However, this tool has its limitation in processing misspelled word especially in repeated letters or characters that may lead to sentiment inaccuracy. The accuracy of SO-CAL is when processing social media text that mostly contains misspelled word is low. Thus, an enhanced scoring mechanism (LexiPro-SM) was developed to improve the sentiment scoring considering misspelled word especially on words that contain repeated letters. The LexiPro-SM was tested on the posts that were collected from the Facebook official pages of two major airline industries in Malaysia, which will be referred to Airline A and Airline B respectively. Three important phases were involved the development of LexiPro-SM which are, data collection, data cleaning and data analysis. Data collection was performed with the aid of Facebook Graph API to collect three months’ posts from the both airlines. Data cleaning was performed by removing noise leaving only text that contains alphabets and exclamation mark. Improvement was made on the scoring mechanism and incorporated in LexiPro-SM with the features that can process misspelled word and also other improved features such as negation and exclamation mark. Then clean data of the airline was analyzed with LexiPro-SM and SO-CAL. A web-based portal was developed to visualize the LexiPro-SM’s result of the two airlines, where each airline has own page with overall score chart, polarity group chart and sub-services chart. Sub-services chart is a new idea implemented in this research to categorize the overall services into sub-services such as customer service, price, preflight and facility. This would be helpful for the airline management to improve their service by narrowing down their attention into a particular service. The airline pages are also linked in order to show the comparison results between Airline A and Airline B. Based on these results, a case study was conducted between the two airlines where the observation shows that Airline A achieved a high positive score than Airline B. Moreover, to assess the effectiveness of LexiPro-SM , the both results of LexiPro-SM and SO-CAL was compared by performed evaluation measures using evaluation metrics (such as accuracy, recall, precision and F1-score) with the reference of human expert results. From the evaluation it shows LexiPro-SM achieved higher accuracy (90.7%) than SO-CAL (58.33%). Overall, in LexiPro-SM the improvement made has increased the accuracy of sentiment detection and produced a better result than SO-CAL. This concludes processing misspelled word is an important process in social media sentiment analysis. This is further proved with the reference to the case study, where a conclusion was formed as Airline A providing a better service than Airline B.

Actions (For repository staff only : Login required)