HARC-New Hybrid Method with Hierarchical Attention Based Bidirectional Recurrent Neural Network with Dilated Convolutional Neural Network to Recognize Multilabel Emotions from Text

Authors

  • Md Shofiqul Islam Faculty of Soft computing, FSKKP, UMP, Pahang, Malaysia.
  • Mst Sunjida Sultana Islamic University, Kushtia-7600, Bangladesh
  • Mr Uttam Kumar Assistant Programmer at Bangladesh Bank-The Central Bank of Bangladesh.
  • Jubayer Al Mahmud Senior Software Engineer at Charja Solutions Limited, Dhaka.
  • SM Jahidul Islam Senior Software Engineer at Software firm: Oscillo Soft Private Limited, Dhaka-1205.

DOI:

https://doi.org/10.26555/jiteki.v7i1.20550

Keywords:

BiGRU, BiLSTM, Convolutional Neural Network, Sentiment Analysis, Text Classification, Natural Language Processing (NLP)

Abstract

We present a modern hybrid paradigm for managing tacit semantic awareness and qualitative meaning in short texts. The main goals of this proposed technique are to use deep learning approaches to identify multilevel textual sentiment with far less time and more accurate and simple network structure training for better performance. In this analysis, the proposed new hybrid deep learning HARC model architecture for the recognition of multilevel textual sentiment that combines hierarchical attention with Convolutional Neural Network (CNN), Bidirectional Gated Recurrent Unit (BiGRU), and Bidirectional Long Short-Term Memory (BiLSTM) outperforms other compared approaches. BiGRU and BiLSTM were used in this model to eliminate individual context functions and to adequately manage long-range features. Dilated CNN was used to replicate the retrieved feature by forwarding vector instances for better support in the hierarchical attention layer, and it was used to eliminate better text information using higher coupling correlations. Our method handles the most important features to recover the limitations of handling context and semantics sufficiently. On a variety of datasets, our proposed HARC algorithm solution outperformed traditional machine learning approaches as well as comparable deep learning models by a margin of 1%. The accuracy of the proposed HARC method was 82.50 percent IMDB, 98.00 percent for toxic data, 92.31 percent for Cornflower, and 94.60 percent for Emotion recognition data. Our method works better than other basic and CNN and RNN based hybrid models. In the future, we will work for more levels of text emotions from long and more complex text.

Author Biography

Md Shofiqul Islam, Faculty of Soft computing, FSKKP, UMP, Pahang, Malaysia.

I am Md Shofiqul Islam, I have complete my B.Sc from Islamic University,Kushtia,Bangladesh. Now i ma a research assistant at University Malaysia Pahang(UMP), I am a teacher at ADUST university ,Dhaka. I am in teaching profession since 2015. My research field are: Deep learning, Machine learning, Natural Language Processing, Image Processing. I have published a lot fo papers in my field.

References

B. Liu, "Sentiment analysis and opinion mining," Synthesis lectures on human language technologies, vol. 5, no. 1, pp. 1-167, 2012. https://doi.org/10.2200/S00416ED1V01Y201204HLT016

M. S. I. Shofiqul, N. A. Ghani, and M. M. Ahmed, "A review on recent advances in Deep learning for Sentiment Analysis: Performances, Challenges and Limitations," COMPUSOFT: An International Journal of Advanced Computer Technology, vol. 9, no. 7, pp. 3768-3776, 2020.

M. S. Islam, S. Sultana, U. K. Roy, J. A. Mahmud, "A review on Video Classification with Methods, Findings, Performance, Challenges, Limitations and Future Work," Jurnal Ilmiah Teknik Elektro Komputer dan Informatika, Vol 6, No 2, pp. 47-57, 2020. https://doi.org/10.26555/jiteki.v6i2.18978

B. T. Atmaja and M. Akagi, "Deep Multilayer Perceptrons for Dimensional Speech Emotion Recognition," 2020 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), 2020, pp. 325-331.

S. Lai, L. Xu, K. Liu, J. Zhao, "Recurrent convolutional neural networks for text classification," in Twenty-ninth AAAI conference on artificial intelligence. Vol. 29 No. 1, 2015.

G. Algan, I. Ulusoy, "Image classification with deep learning in the presence of noisy labels: A survey," Knowledge-Based Systems, vol. 215, 106771, 2021. https://doi.org/10.1016/j.knosys.2021.106771

M. Schuster and K. K. Paliwal, "Bidirectional recurrent neural networks," in IEEE Transactions on Signal Processing, vol. 45, no. 11, pp. 2673-2681, Nov. 1997. https://doi.org/10.1109/78.650093

Y. Du, X. Zhao, M. He and W. Guo, "A Novel Capsule Based Hybrid Neural Network for Sentiment Classification," in IEEE Access, vol. 7, pp. 39321-39328, 2019. https://doi.org/10.1109/ACCESS.2019.2906398

J. Xu, D. Chen, X. Qiu, X. Huang, "Cached long short-term memory neural networks for document-level sentiment classification," Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 2016. https://doi.org/10.18653/v1/D16-1172

L. LI, A. Zhou, Y. Liu, S. Qian, H. Geng, "Aspect-based sentiment analysis based on dynamic attention GRU," Scientia Sinica Informationis, vol. 49, no. 8, pp. 1019-1030, 2019. https://doi.org/10.1360/N112018-00280

Y. Kim, "Convolutional neural networks for sentence classification," Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2014. https://doi.org/10.3115/v1/D14-1181

X. Wang, W. Jiang, and Z. Luo, "Combination of convolutional and recurrent neural network for sentiment analysis of short texts," in Proceedings of COLING 2016, the 26th international conference on computational linguistics: Technical papers, 2016.

K. Cho, B. van Merriënboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, Y. Bengio, "Learning phrase representations using RNN encoder-decoder for statistical machine translation," Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2014. https://doi.org/10.3115/v1/D14-1179

Y. Han, M. Liu and W. Jing, "Aspect-Level Drug Reviews Sentiment Analysis Based on Double BiGRU and Knowledge Transfer," in IEEE Access, vol. 8, pp. 21314-21325, 2020. https://doi.org/10.1109/ACCESS.2020.2969473

A. Yadav and D. K. Vishwakarma, "Sentiment analysis using deep learning architectures: a review," Artificial Intelligence Review, vol. 53, pp. 4335-4385, 2020. https://doi.org/10.1007/s10462-019-09794-5

J. Pennington, R. Socher, and C. D. Manning, "Glove: Global vectors for word representation," in Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), 2014. https://doi.org/10.3115/v1/D14-1162

F. Yu and V. Koltun, "Multi-scale context aggregation by dilated convolutions," arXiv preprint, arXiv:1511.07122, 2015. https://arxiv.org/abs/1511.07122

N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, R. Slakhutdinov "Dropout: a simple way to prevent neural networks from overfitting," The journal of machine learning research, vol. 15, no. 1, pp. 1929-1958, 2014.

B. Pang, and L. Lee, "Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales," in Proceedings of the 43rd annual meeting on association for computational linguistics, Association for Computational Linguistics, P05-1015, p. 115–124, 2005. https://doi.org/10.3115/1219840.1219855

A. L. Maas, R. Daly, P. T. Pham, D. Huang, A. Y. Ng, C. Potts, "Learning word vectors for sentiment analysis," in Proceedings of the 49th annual meeting of the association for computational linguistics, pp. 142-150, 2011.

Downloads

Published

2021-04-24

How to Cite

[1]
M. S. Islam, M. S. Sultana, M. U. Kumar, J. A. Mahmud, and S. J. Islam, “HARC-New Hybrid Method with Hierarchical Attention Based Bidirectional Recurrent Neural Network with Dilated Convolutional Neural Network to Recognize Multilabel Emotions from Text”, J. Ilm. Tek. Elektro Komput. Dan Inform, vol. 7, no. 1, pp. 142–153, Apr. 2021.

Issue

Section

Articles

Similar Articles

1 2 > >> 

You may also start an advanced similarity search for this article.