This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Deep Learning for Aspect-Based Sentiment Analysis on Indonesian Hotels Reviews
Corresponding Author(s) : Siwi Cahyaningtyas
Kinetik: Game Technology, Information System, Computer Network, Computing, Electronics, and Control,
Vo. 6, No. 3, August 2021
Abstract
Tourism is one of the fastest-growing industries. Many travelers book hotels and share their experiences using travel e-commerce sites. To improve the quality of products and services, we can take advantage by analyzing their reviews. We can see the good and the bad thing reviews in every aspect of the hotel. However, research to analyze sentiment in every aspect using Indonesian hotel reviews is still relatively new. In this work, we propose to create an Aspect-based Sentiment Analysis (ABSA) using Indonesian hotel reviews to solve the problem. This research consists of four steps: collecting data, preprocessing, aspect classification, and sentiment classification. Our classification process compares with eight deep learning methods (RNN, LSTM, GRU, BiLSTM, Attention BiLSTM, CNN, CNN-LSTM, and CNN-BiLSTM). In aspect classification, we have six classes of aspects which are harga (price), hotel, kamar (room), lokasi (location), pelayanan (service), and restoran (restaurant). In sentiment analysis, we compared two scenarios to classify sentiments as positive or negative. The first one is to classify sentiment in all aspects, and the second one is to classify sentiment in every aspect. The results showed that LSTM achieved the best model for aspect classification with an accuracy value of 0.926. For sentiment classification, our experiments showed that classify sentiment in every aspect achieved a better result than classify sentiment in all aspects. The result showed that the CNN model gets an average accuracy score of 0.904.
Keywords
Download Citation
Endnote/Zotero/Mendeley (RIS)BibTeX
- T. Tran, H. Ba, and V. N. Huynh, Measuring hotel review sentiment: An aspect-based sentiment analysis approach, vol. 11471 LNAI. Springer International Publishing, 2019. https://doi.org/10.1007/978-3-030-14815-7_33
- M. Ady and D. Quadri-Felitti, “Consumer research identifies how to present travel review content for more bookings,” TrustYou, 2015.
- P. Phillips, S. Barnes, K. Zigan, and R. Schegg, “Understanding the Impact of Online Reviews on Hotel Performance: An Empirical Analysis,” J. Travel Res., vol. 56, no. 2, pp. 235–249, 2017. https://doi.org/10.1177%2F0047287516636481
- W. H. Khong, L. K. Soon, H. N. Goh, and S. C. Haw, Leveraging part-of-speech tagging for sentiment analysis in short texts and regular texts, vol. 11341 LNCS. Springer International Publishing, 2018. https://doi.org/10.1007/978-3-030-04284-4_13
- B. Jang, M. Kim, G. Harerimana, S. U. Kang, and J. W. Kim, “Bi-LSTM model to increase accuracy in text classification: Combining word2vec CNN and attention mechanism,” Appl. Sci., vol. 10, no. 17, 2020. https://doi.org/10.3390/app10175841
- M. Afzaal, M. Usman, and A. Fong, “Predictive aspect-based sentiment classification of online tourist reviews,” J. Inf. Sci., vol. 45, no. 3, pp. 341–363, 2019. https://doi.org/10.1177%2F0165551518789872
- H. Peng, Y. Ma, Y. Li, and E. Cambria, “Learning multi-grained aspect target sequence for Chinese sentiment analysis,” Knowledge-Based Syst., vol. 148, pp. 167–176, 2018. https://doi.org/10.1016/j.knosys.2018.02.034
- D. Ekawati and M. L. Khodra, “Aspect-based sentiment analysis for Indonesian restaurant reviews,” Proc. - 2017 Int. Conf. Adv. Informatics Concepts, Theory Appl. ICAICTA 2017, 2017. https://doi.org/10.1109/ICAICTA.2017.8090963
- S. Wu, Y. Xu, F. Wu, Z. Yuan, Y. Huang, and X. Li, “Aspect-based sentiment analysis via fusing multiple sources of textual knowledge,” Knowledge-Based Syst., vol. 183, p. 104868, 2019. https://doi.org/10.1016/j.knosys.2019.104868
- S. Gu, L. Zhang, Y. Hou, and Y. Song, “A position-aware bidirectional attention network for aspect-level sentiment analysis,” Proc. 27th Int. Conf. Comput. Linguist., pp. 774–784, 2018.
- M. Al-Smadi, B. Talafha, M. Al-Ayyoub, and Y. Jararweh, “Using long short-term memory deep neural networks for aspect-based sentiment analysis of Arabic reviews,” Int. J. Mach. Learn. Cybern., vol. 10, no. 8, pp. 2163–2175, 2019. https://doi.org/10.1007/s13042-018-0799-4
- Y. Luo and X. Xu, “Predicting the helpfulness of online restaurant reviews using different machine learning algorithms: A case study of yelp,” Sustain., vol. 11, no. 19, 2019. https://doi.org/10.3390/su11195254
- Y. Setiowati, “Service Extraction and Sentiment Analysis to Indicate Hotel Service Quality in Yogyakarta based on User Opinion,” 2018 Int. Semin. Res. Inf. Technol. Intell. Syst., pp. 427–432, 2016. https://doi.org/10.1109/ISRITI.2018.8864269
- L. Vinet and A. Zhedanov, “A ‘missing’ family of classical orthogonal polynomials,” J. Phys. A Math. Theor., vol. 44, no. 8, pp. 329–334, 2011. https://doi.org/10.1088/1751-8113/44/8/085201
- J. Thanaki, Python Natural Language Processing. 2017.
- P. Prameswari, I. Surjandari, and E. Laoh, “Opinion mining from online reviews in Bali tourist area,” Proceeding - 2017 3rd Int. Conf. Sci. Inf. Technol. Theory Appl. IT Educ. Ind. Soc. Big Data Era, ICSITech 2017, vol. 2018-Janua, pp. 226–230, 2017. https://doi.org/10.1109/ICSITech.2017.8257115
- S. Dolnicar and T. Otter, “Which Hotel Attributes Matter? A review of previous and a framework for future research,” Preterm Birth Prev. Manag., pp. 270–273, 2010.
- A. F. Hidayatullah, S. Cahyaningtyas, and R. D. Pamungkas, “Attention-based CNN-BiLSTM for Dialect Identification on Javanese Text,” Kinet. Game Technol. Inf. Syst. Comput. Network, Comput. Electron. Control, vol. 4, pp. 317–324, 2020. https://doi.org/10.22219/kinetik.v5i4.1121
- SimilarWeb, “Top Apps Ranking.”
- H. Jangid, S. Singhal, R. R. Shah, and R. Zimmermann, “Aspect-Based Financial Sentiment Analysis using Deep Learning,” pp. 1961–1966, 2018. https://doi.org/10.1145/3184558.3191827
- A. Yadav and D. K. Vishwakarma, “Sentiment analysis using deep learning architectures: a review,” Artif. Intell. Rev., vol. 53, no. 6, pp. 4335–4385, 2020. https://doi.org/10.1007/s10462-019-09794-5
- A. Mittal, “Understanding RNN and LSTM,” towards data science, 2019.
- S. Kostadinov, “Understanding GRU Networks,” towards data science, 2017.
- G. Liu and J. Guo, “Bidirectional LSTM with attention mechanism and convolutional layer for text classification,” Neurocomputing, vol. 337, pp. 325–338, 2019. https://doi.org/10.1016/j.neucom.2019.01.078
- P. Chen, Z. Sun, L. Bing, and W. Yang, “Recurrent attention network on memory for aspect sentiment analysis,” EMNLP 2017 - Conf. Empir. Methods Nat. Lang. Process. Proc., pp. 452–461, 2017. http://dx.doi.org/10.18653/v1/D17-1047
- M. T. Luong, H. Pham, and C. D. Manning, “Effective approaches to attention-based neural machine translation,” Conf. Proc. - EMNLP 2015 Conf. Empir. Methods Nat. Lang. Process., pp. 1412–1421, 2015. http://dx.doi.org/10.18653/v1/D15-1166
- D. Bahdanau, K. H. Cho, and Y. Bengio, “Neural machine translation by jointly learning to align and translate,” 3rd Int. Conf. Learn. Represent. ICLR 2015 - Conf. Track Proc., pp. 1–15, 2015.
- J. Xie, B. Chen, X. Gu, F. Liang, and X. Xu, “Self-Attention-Based BiLSTM Model for Short Text Fine-Grained Sentiment Classification,” IEEE Access, vol. 7, pp. 180558–180570, 2019. https://doi.org/10.1109/ACCESS.2019.2957510
- P. Zhou et al., “Attention-based bidirectional long short-term memory networks for relation classification,” 54th Annu. Meet. Assoc. Comput. Linguist. ACL 2016 - Short Pap., pp. 207–212, 2016. http://dx.doi.org/10.18653/v1/P16-2034
- Y. Kim, “Convolutional neural networks for sentence classification,” EMNLP 2014 - 2014 Conf. Empir. Methods Nat. Lang. Process. Proc. Conf., pp. 1746–1751, 2014. http://dx.doi.org/10.3115/v1/D14-1181
- C. L. S.-T. M. Networks, “CNN Long Short-Term Memory Networks,” Machine Learning Mastery, 2019.
- M. Abdullah, M. Hadzikadicy, and S. Shaikhz, “SEDAT: Sentiment and Emotion Detection in Arabic Text Using CNN-LSTM Deep Learning,” Proc. - 17th IEEE Int. Conf. Mach. Learn. Appl. ICMLA 2018, pp. 835–840, 2019. https://doi.org/10.1109/ICMLA.2018.00134
- Z. Rajabi, O. Uzuner, and A. Shehu, “A Multi-channel BiLSTM-CNN model for multilabel emotion classification of informal text,” Proc. - 14th IEEE Int. Conf. Semant. Comput. ICSC 2020, pp. 303–306, 2020. https://doi.org/10.1109/ICSC.2020.00060
- M. Rhanoui, M. Mikram, S. Yousfi, and S. Barzali, “A CNN-BiLSTM Model for Document-Level Sentiment Analysis,” Mach. Learn. Knowl. Extr., vol. 1, no. 3, pp. 832–847, 2019. https://doi.org/10.3390/make1030048
References
T. Tran, H. Ba, and V. N. Huynh, Measuring hotel review sentiment: An aspect-based sentiment analysis approach, vol. 11471 LNAI. Springer International Publishing, 2019. https://doi.org/10.1007/978-3-030-14815-7_33
M. Ady and D. Quadri-Felitti, “Consumer research identifies how to present travel review content for more bookings,” TrustYou, 2015.
P. Phillips, S. Barnes, K. Zigan, and R. Schegg, “Understanding the Impact of Online Reviews on Hotel Performance: An Empirical Analysis,” J. Travel Res., vol. 56, no. 2, pp. 235–249, 2017. https://doi.org/10.1177%2F0047287516636481
W. H. Khong, L. K. Soon, H. N. Goh, and S. C. Haw, Leveraging part-of-speech tagging for sentiment analysis in short texts and regular texts, vol. 11341 LNCS. Springer International Publishing, 2018. https://doi.org/10.1007/978-3-030-04284-4_13
B. Jang, M. Kim, G. Harerimana, S. U. Kang, and J. W. Kim, “Bi-LSTM model to increase accuracy in text classification: Combining word2vec CNN and attention mechanism,” Appl. Sci., vol. 10, no. 17, 2020. https://doi.org/10.3390/app10175841
M. Afzaal, M. Usman, and A. Fong, “Predictive aspect-based sentiment classification of online tourist reviews,” J. Inf. Sci., vol. 45, no. 3, pp. 341–363, 2019. https://doi.org/10.1177%2F0165551518789872
H. Peng, Y. Ma, Y. Li, and E. Cambria, “Learning multi-grained aspect target sequence for Chinese sentiment analysis,” Knowledge-Based Syst., vol. 148, pp. 167–176, 2018. https://doi.org/10.1016/j.knosys.2018.02.034
D. Ekawati and M. L. Khodra, “Aspect-based sentiment analysis for Indonesian restaurant reviews,” Proc. - 2017 Int. Conf. Adv. Informatics Concepts, Theory Appl. ICAICTA 2017, 2017. https://doi.org/10.1109/ICAICTA.2017.8090963
S. Wu, Y. Xu, F. Wu, Z. Yuan, Y. Huang, and X. Li, “Aspect-based sentiment analysis via fusing multiple sources of textual knowledge,” Knowledge-Based Syst., vol. 183, p. 104868, 2019. https://doi.org/10.1016/j.knosys.2019.104868
S. Gu, L. Zhang, Y. Hou, and Y. Song, “A position-aware bidirectional attention network for aspect-level sentiment analysis,” Proc. 27th Int. Conf. Comput. Linguist., pp. 774–784, 2018.
M. Al-Smadi, B. Talafha, M. Al-Ayyoub, and Y. Jararweh, “Using long short-term memory deep neural networks for aspect-based sentiment analysis of Arabic reviews,” Int. J. Mach. Learn. Cybern., vol. 10, no. 8, pp. 2163–2175, 2019. https://doi.org/10.1007/s13042-018-0799-4
Y. Luo and X. Xu, “Predicting the helpfulness of online restaurant reviews using different machine learning algorithms: A case study of yelp,” Sustain., vol. 11, no. 19, 2019. https://doi.org/10.3390/su11195254
Y. Setiowati, “Service Extraction and Sentiment Analysis to Indicate Hotel Service Quality in Yogyakarta based on User Opinion,” 2018 Int. Semin. Res. Inf. Technol. Intell. Syst., pp. 427–432, 2016. https://doi.org/10.1109/ISRITI.2018.8864269
L. Vinet and A. Zhedanov, “A ‘missing’ family of classical orthogonal polynomials,” J. Phys. A Math. Theor., vol. 44, no. 8, pp. 329–334, 2011. https://doi.org/10.1088/1751-8113/44/8/085201
J. Thanaki, Python Natural Language Processing. 2017.
P. Prameswari, I. Surjandari, and E. Laoh, “Opinion mining from online reviews in Bali tourist area,” Proceeding - 2017 3rd Int. Conf. Sci. Inf. Technol. Theory Appl. IT Educ. Ind. Soc. Big Data Era, ICSITech 2017, vol. 2018-Janua, pp. 226–230, 2017. https://doi.org/10.1109/ICSITech.2017.8257115
S. Dolnicar and T. Otter, “Which Hotel Attributes Matter? A review of previous and a framework for future research,” Preterm Birth Prev. Manag., pp. 270–273, 2010.
A. F. Hidayatullah, S. Cahyaningtyas, and R. D. Pamungkas, “Attention-based CNN-BiLSTM for Dialect Identification on Javanese Text,” Kinet. Game Technol. Inf. Syst. Comput. Network, Comput. Electron. Control, vol. 4, pp. 317–324, 2020. https://doi.org/10.22219/kinetik.v5i4.1121
SimilarWeb, “Top Apps Ranking.”
H. Jangid, S. Singhal, R. R. Shah, and R. Zimmermann, “Aspect-Based Financial Sentiment Analysis using Deep Learning,” pp. 1961–1966, 2018. https://doi.org/10.1145/3184558.3191827
A. Yadav and D. K. Vishwakarma, “Sentiment analysis using deep learning architectures: a review,” Artif. Intell. Rev., vol. 53, no. 6, pp. 4335–4385, 2020. https://doi.org/10.1007/s10462-019-09794-5
A. Mittal, “Understanding RNN and LSTM,” towards data science, 2019.
S. Kostadinov, “Understanding GRU Networks,” towards data science, 2017.
G. Liu and J. Guo, “Bidirectional LSTM with attention mechanism and convolutional layer for text classification,” Neurocomputing, vol. 337, pp. 325–338, 2019. https://doi.org/10.1016/j.neucom.2019.01.078
P. Chen, Z. Sun, L. Bing, and W. Yang, “Recurrent attention network on memory for aspect sentiment analysis,” EMNLP 2017 - Conf. Empir. Methods Nat. Lang. Process. Proc., pp. 452–461, 2017. http://dx.doi.org/10.18653/v1/D17-1047
M. T. Luong, H. Pham, and C. D. Manning, “Effective approaches to attention-based neural machine translation,” Conf. Proc. - EMNLP 2015 Conf. Empir. Methods Nat. Lang. Process., pp. 1412–1421, 2015. http://dx.doi.org/10.18653/v1/D15-1166
D. Bahdanau, K. H. Cho, and Y. Bengio, “Neural machine translation by jointly learning to align and translate,” 3rd Int. Conf. Learn. Represent. ICLR 2015 - Conf. Track Proc., pp. 1–15, 2015.
J. Xie, B. Chen, X. Gu, F. Liang, and X. Xu, “Self-Attention-Based BiLSTM Model for Short Text Fine-Grained Sentiment Classification,” IEEE Access, vol. 7, pp. 180558–180570, 2019. https://doi.org/10.1109/ACCESS.2019.2957510
P. Zhou et al., “Attention-based bidirectional long short-term memory networks for relation classification,” 54th Annu. Meet. Assoc. Comput. Linguist. ACL 2016 - Short Pap., pp. 207–212, 2016. http://dx.doi.org/10.18653/v1/P16-2034
Y. Kim, “Convolutional neural networks for sentence classification,” EMNLP 2014 - 2014 Conf. Empir. Methods Nat. Lang. Process. Proc. Conf., pp. 1746–1751, 2014. http://dx.doi.org/10.3115/v1/D14-1181
C. L. S.-T. M. Networks, “CNN Long Short-Term Memory Networks,” Machine Learning Mastery, 2019.
M. Abdullah, M. Hadzikadicy, and S. Shaikhz, “SEDAT: Sentiment and Emotion Detection in Arabic Text Using CNN-LSTM Deep Learning,” Proc. - 17th IEEE Int. Conf. Mach. Learn. Appl. ICMLA 2018, pp. 835–840, 2019. https://doi.org/10.1109/ICMLA.2018.00134
Z. Rajabi, O. Uzuner, and A. Shehu, “A Multi-channel BiLSTM-CNN model for multilabel emotion classification of informal text,” Proc. - 14th IEEE Int. Conf. Semant. Comput. ICSC 2020, pp. 303–306, 2020. https://doi.org/10.1109/ICSC.2020.00060
M. Rhanoui, M. Mikram, S. Yousfi, and S. Barzali, “A CNN-BiLSTM Model for Document-Level Sentiment Analysis,” Mach. Learn. Knowl. Extr., vol. 1, no. 3, pp. 832–847, 2019. https://doi.org/10.3390/make1030048