Cover Image

HARC-New Hybrid Method with Hierarchical Attention Based Bidirectional Recurrent Neural Network with Dilated Convolutional Neural Network to Recognize Multilabel Emotions from Text

Md Shofiqul Islam, Mst Sunjida Sultana, Mr Uttam Kumar, Jubayer Al Mahmud, SM Jahidul Islam

Abstract


We present a modern hybrid paradigm for managing tacit semantic awareness and qualitative meaning in short texts. The main goals of this proposed technique are to use deep learning approaches to identify multilevel textual sentiment with far less time and more accurate and simple network structure training for better performance. In this analysis, the proposed new hybrid deep learning HARC model architecture for the recognition of multilevel textual sentiment that combines hierarchical attention with Convolutional Neural Network (CNN), Bidirectional Gated Recurrent Unit (BiGRU), and Bidirectional Long Short-Term Memory (BiLSTM) outperforms other compared approaches. BiGRU and BiLSTM were used in this model to eliminate individual context functions and to adequately manage long-range features. Dilated CNN was used to replicate the retrieved feature by forwarding vector instances for better support in the hierarchical attention layer, and it was used to eliminate better text information using higher coupling correlations. Our method handles the most important features to recover the limitations of handling context and semantics sufficiently. On a variety of datasets, our proposed HARC algorithm solution outperformed traditional machine learning approaches as well as comparable deep learning models by a margin of 1%. The accuracy of the proposed HARC method was 82.50 percent IMDB, 98.00 percent for toxic data, 92.31 percent for Cornflower, and 94.60 percent for Emotion recognition data. Our method works better than other basic and CNN and RNN based hybrid models. In the future, we will work for more levels of text emotions from long and more complex text.

Keywords


BiGRU; BiLSTM; Convolutional Neural Network; Sentiment Analysis; Text Classification; Natural Language Processing (NLP)

Full Text:

PDF

References


B. Liu, "Sentiment analysis and opinion mining," Synthesis lectures on human language technologies, vol. 5, no. 1, pp. 1-167, 2012. https://doi.org/10.2200/S00416ED1V01Y201204HLT016

M. S. I. Shofiqul, N. A. Ghani, and M. M. Ahmed, "A review on recent advances in Deep learning for Sentiment Analysis: Performances, Challenges and Limitations," COMPUSOFT: An International Journal of Advanced Computer Technology, vol. 9, no. 7, pp. 3768-3776, 2020.

M. S. Islam, S. Sultana, U. K. Roy, J. A. Mahmud, "A review on Video Classification with Methods, Findings, Performance, Challenges, Limitations and Future Work," Jurnal Ilmiah Teknik Elektro Komputer dan Informatika, Vol 6, No 2, pp. 47-57, 2020. https://doi.org/10.26555/jiteki.v6i2.18978

B. T. Atmaja and M. Akagi, "Deep Multilayer Perceptrons for Dimensional Speech Emotion Recognition," 2020 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), 2020, pp. 325-331.

S. Lai, L. Xu, K. Liu, J. Zhao, "Recurrent convolutional neural networks for text classification," in Twenty-ninth AAAI conference on artificial intelligence. Vol. 29 No. 1, 2015.

G. Algan, I. Ulusoy, "Image classification with deep learning in the presence of noisy labels: A survey," Knowledge-Based Systems, vol. 215, 106771, 2021. https://doi.org/10.1016/j.knosys.2021.106771

M. Schuster and K. K. Paliwal, "Bidirectional recurrent neural networks," in IEEE Transactions on Signal Processing, vol. 45, no. 11, pp. 2673-2681, Nov. 1997. https://doi.org/10.1109/78.650093

Y. Du, X. Zhao, M. He and W. Guo, "A Novel Capsule Based Hybrid Neural Network for Sentiment Classification," in IEEE Access, vol. 7, pp. 39321-39328, 2019. https://doi.org/10.1109/ACCESS.2019.2906398

J. Xu, D. Chen, X. Qiu, X. Huang, "Cached long short-term memory neural networks for document-level sentiment classification," Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 2016. https://doi.org/10.18653/v1/D16-1172

L. LI, A. Zhou, Y. Liu, S. Qian, H. Geng, "Aspect-based sentiment analysis based on dynamic attention GRU," Scientia Sinica Informationis, vol. 49, no. 8, pp. 1019-1030, 2019. https://doi.org/10.1360/N112018-00280

Y. Kim, "Convolutional neural networks for sentence classification," Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2014. https://doi.org/10.3115/v1/D14-1181

X. Wang, W. Jiang, and Z. Luo, "Combination of convolutional and recurrent neural network for sentiment analysis of short texts," in Proceedings of COLING 2016, the 26th international conference on computational linguistics: Technical papers, 2016.

K. Cho, B. van Merriënboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, Y. Bengio, "Learning phrase representations using RNN encoder-decoder for statistical machine translation," Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2014. https://doi.org/10.3115/v1/D14-1179

Y. Han, M. Liu and W. Jing, "Aspect-Level Drug Reviews Sentiment Analysis Based on Double BiGRU and Knowledge Transfer," in IEEE Access, vol. 8, pp. 21314-21325, 2020. https://doi.org/10.1109/ACCESS.2020.2969473

A. Yadav and D. K. Vishwakarma, "Sentiment analysis using deep learning architectures: a review," Artificial Intelligence Review, vol. 53, pp. 4335-4385, 2020. https://doi.org/10.1007/s10462-019-09794-5

J. Pennington, R. Socher, and C. D. Manning, "Glove: Global vectors for word representation," in Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), 2014. https://doi.org/10.3115/v1/D14-1162

F. Yu and V. Koltun, "Multi-scale context aggregation by dilated convolutions," arXiv preprint, arXiv:1511.07122, 2015. https://arxiv.org/abs/1511.07122

N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, R. Slakhutdinov "Dropout: a simple way to prevent neural networks from overfitting," The journal of machine learning research, vol. 15, no. 1, pp. 1929-1958, 2014.

B. Pang, and L. Lee, "Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales," in Proceedings of the 43rd annual meeting on association for computational linguistics, Association for Computational Linguistics, P05-1015, p. 115–124, 2005. https://doi.org/10.3115/1219840.1219855

A. L. Maas, R. Daly, P. T. Pham, D. Huang, A. Y. Ng, C. Potts, "Learning word vectors for sentiment analysis," in Proceedings of the 49th annual meeting of the association for computational linguistics, pp. 142-150, 2011.




DOI: http://dx.doi.org/10.26555/jiteki.v7i1.20550

Refbacks

  • There are currently no refbacks.


Copyright (c) 2021 Md Shofiqul Islam, Mst Sunjida Sultana, Mr Uttam Kumar, Jubayer Al Mahmud, Md Jahidul Islam

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.


 
About the JournalJournal PoliciesAuthor Information
 


Jurnal Ilmiah Teknik Elektro Komputer dan Informatika
ISSN 2338-3070 (print) | 2338-3062 (online)
Organized by Electrical Engineering Department - Universitas Ahmad Dahlan
Published by Universitas Ahmad Dahlan
Website: http://journal.uad.ac.id/index.php/jiteki
Email 1: jiteki@ee.uad.ac.id
Email 2: alfianmaarif@ee.uad.ac.id
Office Address: Kantor Program Studi Teknik Elektro, Lantai 6 Sayap Barat, Kampus 4 UAD, Jl. Ringroad Selatan, Tamanan, Kec. Banguntapan, Bantul, Daerah Istimewa Yogyakarta 55191, Indonesia