HARC-New Hybrid Method with Hierarchical Attention Based Bidirectional Recurrent Neural Network with Dilated Convolutional Neural Network to Recognize Multilabel Emotions from Text
DOI:
https://doi.org/10.26555/jiteki.v7i1.20550Keywords:
BiGRU, BiLSTM, Convolutional Neural Network, Sentiment Analysis, Text Classification, Natural Language Processing (NLP)Abstract
We present a modern hybrid paradigm for managing tacit semantic awareness and qualitative meaning in short texts. The main goals of this proposed technique are to use deep learning approaches to identify multilevel textual sentiment with far less time and more accurate and simple network structure training for better performance. In this analysis, the proposed new hybrid deep learning HARC model architecture for the recognition of multilevel textual sentiment that combines hierarchical attention with Convolutional Neural Network (CNN), Bidirectional Gated Recurrent Unit (BiGRU), and Bidirectional Long Short-Term Memory (BiLSTM) outperforms other compared approaches. BiGRU and BiLSTM were used in this model to eliminate individual context functions and to adequately manage long-range features. Dilated CNN was used to replicate the retrieved feature by forwarding vector instances for better support in the hierarchical attention layer, and it was used to eliminate better text information using higher coupling correlations. Our method handles the most important features to recover the limitations of handling context and semantics sufficiently. On a variety of datasets, our proposed HARC algorithm solution outperformed traditional machine learning approaches as well as comparable deep learning models by a margin of 1%. The accuracy of the proposed HARC method was 82.50 percent IMDB, 98.00 percent for toxic data, 92.31 percent for Cornflower, and 94.60 percent for Emotion recognition data. Our method works better than other basic and CNN and RNN based hybrid models. In the future, we will work for more levels of text emotions from long and more complex text.References
B. Liu, "Sentiment analysis and opinion mining," Synthesis lectures on human language technologies, vol. 5, no. 1, pp. 1-167, 2012. https://doi.org/10.2200/S00416ED1V01Y201204HLT016
M. S. I. Shofiqul, N. A. Ghani, and M. M. Ahmed, "A review on recent advances in Deep learning for Sentiment Analysis: Performances, Challenges and Limitations," COMPUSOFT: An International Journal of Advanced Computer Technology, vol. 9, no. 7, pp. 3768-3776, 2020.
M. S. Islam, S. Sultana, U. K. Roy, J. A. Mahmud, "A review on Video Classification with Methods, Findings, Performance, Challenges, Limitations and Future Work," Jurnal Ilmiah Teknik Elektro Komputer dan Informatika, Vol 6, No 2, pp. 47-57, 2020. https://doi.org/10.26555/jiteki.v6i2.18978
B. T. Atmaja and M. Akagi, "Deep Multilayer Perceptrons for Dimensional Speech Emotion Recognition," 2020 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), 2020, pp. 325-331.
S. Lai, L. Xu, K. Liu, J. Zhao, "Recurrent convolutional neural networks for text classification," in Twenty-ninth AAAI conference on artificial intelligence. Vol. 29 No. 1, 2015.
G. Algan, I. Ulusoy, "Image classification with deep learning in the presence of noisy labels: A survey," Knowledge-Based Systems, vol. 215, 106771, 2021. https://doi.org/10.1016/j.knosys.2021.106771
M. Schuster and K. K. Paliwal, "Bidirectional recurrent neural networks," in IEEE Transactions on Signal Processing, vol. 45, no. 11, pp. 2673-2681, Nov. 1997. https://doi.org/10.1109/78.650093
Y. Du, X. Zhao, M. He and W. Guo, "A Novel Capsule Based Hybrid Neural Network for Sentiment Classification," in IEEE Access, vol. 7, pp. 39321-39328, 2019. https://doi.org/10.1109/ACCESS.2019.2906398
J. Xu, D. Chen, X. Qiu, X. Huang, "Cached long short-term memory neural networks for document-level sentiment classification," Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 2016. https://doi.org/10.18653/v1/D16-1172
L. LI, A. Zhou, Y. Liu, S. Qian, H. Geng, "Aspect-based sentiment analysis based on dynamic attention GRU," Scientia Sinica Informationis, vol. 49, no. 8, pp. 1019-1030, 2019. https://doi.org/10.1360/N112018-00280
Y. Kim, "Convolutional neural networks for sentence classification," Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2014. https://doi.org/10.3115/v1/D14-1181
X. Wang, W. Jiang, and Z. Luo, "Combination of convolutional and recurrent neural network for sentiment analysis of short texts," in Proceedings of COLING 2016, the 26th international conference on computational linguistics: Technical papers, 2016.
K. Cho, B. van Merriënboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, Y. Bengio, "Learning phrase representations using RNN encoder-decoder for statistical machine translation," Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2014. https://doi.org/10.3115/v1/D14-1179
Y. Han, M. Liu and W. Jing, "Aspect-Level Drug Reviews Sentiment Analysis Based on Double BiGRU and Knowledge Transfer," in IEEE Access, vol. 8, pp. 21314-21325, 2020. https://doi.org/10.1109/ACCESS.2020.2969473
A. Yadav and D. K. Vishwakarma, "Sentiment analysis using deep learning architectures: a review," Artificial Intelligence Review, vol. 53, pp. 4335-4385, 2020. https://doi.org/10.1007/s10462-019-09794-5
J. Pennington, R. Socher, and C. D. Manning, "Glove: Global vectors for word representation," in Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), 2014. https://doi.org/10.3115/v1/D14-1162
F. Yu and V. Koltun, "Multi-scale context aggregation by dilated convolutions," arXiv preprint, arXiv:1511.07122, 2015. https://arxiv.org/abs/1511.07122
N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, R. Slakhutdinov "Dropout: a simple way to prevent neural networks from overfitting," The journal of machine learning research, vol. 15, no. 1, pp. 1929-1958, 2014.
B. Pang, and L. Lee, "Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales," in Proceedings of the 43rd annual meeting on association for computational linguistics, Association for Computational Linguistics, P05-1015, p. 115–124, 2005. https://doi.org/10.3115/1219840.1219855
A. L. Maas, R. Daly, P. T. Pham, D. Huang, A. Y. Ng, C. Potts, "Learning word vectors for sentiment analysis," in Proceedings of the 49th annual meeting of the association for computational linguistics, pp. 142-150, 2011.
Downloads
Published
How to Cite
Issue
Section
License
Authors who publish with JITEKI agree to the following terms:
- Authors retain copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License (CC BY-SA 4.0) that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.
This work is licensed under a Creative Commons Attribution 4.0 International License
