Unidirectional-bidirectional recurrent networks for cardiac disorders classification

The deep learning approach of supervised recurrent network classifiers model, i.e., recurrent neural networks (RNNs), long short-term memory (LSTM), and gated recurrent units (GRUs) are used in this study. The unidirectional and bidirectional for each cardiac disorder (CDs) class is also compared. Comparing both phases is needed to figure out the optimum phase and the best model performance for ECG using the Physionet dataset to classify five classes of CDs with 15 leads ECG signals. The result shows that the bidirectional RNNs method produces better results than the unidirectional method. In contrast to RNNs, the unidirectional LSTM and GRU outperformed the bidirectional phase. The best recurrent network classifier performance is unidirectional GRU with average accuracy, sensitivity, specificity, precision, and F1-score of 98.50%, 95.54%, 98.42%, 89.93%, 92.31%, respectively. Overall, deep learning is a promising improved method for ECG classification.

2) comparing unidirectional and bidirectional-based learning algorithms, 3) proposing the best model for the application. All the phases of the proposed method are presented in Figure 1.

ECG raw data
The ECG data used in this study was collected from PhysioNet: the PTB diagnostic ECG database [30]. This study's algorithm tests the database because it provides a total of 549 records: 80 records with healthy control and the remaining records of nine cardiac disorders. The disorders can be seen in Table 1. Each record includes the conventional 12 leads, and the 3 Frank lead ECGs, with 15 leads, are used in this study. This study aims to classify healthy control patients and four CDs, i.e., 1) myocardial infarction, 2) cardiomyopathy, 3) bundle branch block, and 4) dysrhythmia. The other cardiac disorders in this database were discarded because they belonged to classes not considered in the study. Based on our previous work for binary classification [20], a fixed window size of 4 seconds was determined for each sequence for ECG pre-processing. The total sequence data for five classes was 13.610 sequences.

Recurrent network classifiers
The sequence model consists of sequences of ordered elements, recorded with or without a concrete notion of time. The recurrent process in the neural network operates on sequences of data. The recurrent network takes each element of a sequence, multiplies the element by a matrix, and the previous output is summed from the network. There are two directions for the learning phase in neural networks: unidirectional and bidirectional [31]. The unidirectional preserves the information of the past and runs the inputs only in forward (left-to-right) passes. The bidirectional phase runs the inputs in the forward (left-to-right) and backward (right-to-left) passes and preserves the information from both past and future, as presented in Figure 2.
Recurrent neural networks (RNNs) capture relationships among sequential data types. Feedback loops at hidden layers of RNNs are unidirectional. Unidirectional means the process from left-to-right, in which the flow of the information is only in the forward direction [29]. A unidirectional model can still access inputs by windowing, looking-ahead, or delaying output with a reasonable increase in the number of parameters. Bengio et al. [32] showed that capturing long-term dependencies using a simple RNNs are difficult because gradients tend to either vanish or explode with long sequences. Two techniques have been proposed to solve the gradient problems: long short-term memory (LSTM) and gated recurrent unit (GRU). Schuster presented new concepts of sequence learning in which the information flow is in forward and backward feedback [26]. The connections in the forward direction help us learn from previous representations, and those going backward help us to learn from future representations. Both connections are called "bidirectional RNN (BiRNNs)", which enables the network to predict outputs using inputs of the entire sequence. BiRNNs can be learned using all available input data for a specific timeframe in the past and future [33]. BiRNNs are trained with the same algorithm as a regular unidirectional RNNs, because there are no interactions between the two types of state neurons. Graves et al. proposed an RNNs that uses LSTM cells and computes both forward and backward hidden sequences [23]. It is called bidirectional LSTM (BiLSTM). BiLSTM is the LSTM version of the BiRNNs architecture and can expand LSTM performance in classification procedures [28]. In contrast to the regular LSTM structure, two dissimilar LSTM networks are trained for sequential inputs in the BiLSTM architecture [28]. The neuron in a forward state of BiLSTM acts as a unidirectional LSTM structure, but bidirectional networks are still much more effective than unidirectional networks [23]. The current hidden state depends on two hidden states: the forward and the backward pass of LSTM. The BiLSTM equations in the forward and backward passes are given below [31]:

TELKOMNIKA Telecommun Comput El Control
From (10) and (11), the output of BiLSTM layer at a time t : 1 0 where the output depends on ; 0 h is initialized as a zero vector. Cho et al. implemented the GRU to allow each recurrent unit to adaptively capture the dependency of different time scales [34]. Similarly, to the BiLSTM, GRU contains a forward U GR  which reads the signal Although the GRU is still relatively new and is not commonly used compare to LSTM, some previous literature on GRUs has been explored for the ECG classification task [35]. The phase of BiGRU is simpler than BiLSTM due to the reduction in the gates and the combination of the forget and input gates into the update gate. The  (8) and (9). Then, the hidden states can be seen below; where W and U are the additional parameters. The notation of • is a Hadamard product.

RESULTS AND DISCUSSION
This study presents the one-per-class-coding, which is an extensible algorithm for the binary case. For this study, the total number of classes in this study was five, which represents HC, MI, C, BBB, and D. The extensible algorithms from binary can use a one-per-class-coding concept. For instance, if a five-class case is considered, the output codes for HC, MI, C, BBB, and D were 10000, 01000, 00100, 00010, and 00001, respectively. This technique fixes a problem that is encountered with label encoding when working with categorical data. Each class's performance is based on five metrics: accuracy, sensitivity, specificity, precision, and F1-score. Similar to the previous study [20], the unidirectional and bidirectional multiclass classification process in this paper was divided as 90% and 10% for training and testing, respectively. For computing platforms, this study used a GeForce RTX 2080 graphics processing unit (GPU). The operating system was Windows 10 64-bit. The hyperparameters were the batch size of 512 samples, softmax function in the output layer, and the total epochs were 100.
For the unidirectional multiclass classification process, the class of MI produces good sensitivity, precision, and F1-score, higher than those of the other classes in unidirectional RNNs. The results are presented in Table 2. This might be due to the quantity of HC, C, BBB, and D data available, which was less than the MI of the total data. The imbalanced distribution causes the majority class to achieve higher performance. From a total of five classes, unidirectional RNNs have an average of sensitivity, precision, and F1-score of 90.25%, 85%, and 87.27%, respectively. To prove the gradient problems in standard RNNs architecture, the fine-tuning of unidirectional LSTM and GRU architecture was proposed. In LSTM computation, the average of sensitivity, precision, and F1-score increased to 94.17%, 89.58%, and 91.72%, respectively. In another GRU computation, the performance showed greater increases, with an average of 95.54%, 89.93%, and 92.31%, respectively. In a bidirectional case, the LSTM and GRU still outperformed RNNs and achieved outstanding results, as presented in Table 3, even though the different outcomes were not significant. The bidirectional LSTM result, presented in Table 3, obtained averages for sensitivity, precision, and F1-score of 94.16%, 87.21%, and 90.33%, respectively. Additionally, bidirectional GRU showed averages of sensitivity, precision, and F1-score as 93.71%, 88.78%, and 90.96%, respectively. First, for RNNs in the unidirectional sequence model, the average values of accuracy and specificity in five classes of CDs were higher than those for the bidirectional pass, as presented in Tables 2 and 3. The averages of accuracy and specificity for unidirectional and bidirectional were 97.81% and 97.77%, and 97.33% and 96.66%, respectively. The averages of accuracy and TELKOMNIKA Telecommun Comput El Control  Unidirectional-bidirectional recurrent networks for cardiac disorders classification (Annisa Darmawahyuni) 907 specificity, respectively, were 97.81% and 97.77% for the unidirectional and 97.33% and 96.66% for the bidirectional RNNs. In contrast, the accuracy and specificity for the average of sensitivity, precision, and F1-score, the bidirectional RNNs obtained better performance than the unidirectional RNNs. The averages of sensitivity, precision, and F1-score for bidirectional RNNs are 90.26%, 88.21%, and 89.07%. Second, different from RNNs, the unidirectional LSTM achieved better overall performances than a bidirectional model. The average accuracy, sensitivity, specificity, precision, and F1-score achieved was 98.39%, 94.17%, 98.49%, 89.58%, and 91.72%, respectively. How about GRU as the last model? The unidirectional GRU shows better performance than the bidirectional GRU, similar to LSTM. Compared with the previous work [20] in terms of MI classification in the binary case, unidirectional LSTM performed better than GRU. In contrast, both unidirectional and bidirectional GRU was better than LSTM overall regarding the multiclass classification case. GRU is computationally more efficient than LSTM, due to having only two gates for computing. In time computation, the training process of GRU was faster than that of LSTM and did not have much computational power. Notably, in this study, unidirectional LSTM and GRU do not have different time computations for each epoch's training. However, for bidirectional LSTM and GRU, the time equals 5 and 4 seconds, respectively. Besides, some deep learning techniques have been implemented. However, it is still limited to specific ECG leads with fewer classes and only applicable to the common unidirectional model. According to the comparison between unidirectional and bidirectional GRU, this study proposes the best model, i.e., unidirectional GRU. The unidirectional GRU performance obtained an average accuracy, sensitivity, specificity, precision, and F1-score of 98.50%, 95.54%, 98.42%, 89.93%, and 92.31%, respectively, for 15-leads of ECG.
The proposed method's performance is compared with the previous literature in recent years, as illustrated in Table 4. Tripathy et al. [36] proposed the least square SVM with an accuracy of 89.93%, 93.95%, 93.03%, 90.09%, and 85.29% for HC, MI, C, D, and hypertrophy. The average accuracy is 90.34% in all classes. In another literature, Acharya et al. [37] identified three classes of Normal, MI, and other CDs, based on discrete cosine transform (DCT). The DCT outperformed discrete wavelet transform (DWT) and empirical mode decomposition (EMD). The accuracy, sensitivity, and specificity were 98.50%, 99.72%, and 98.46%, respectively. In the same year, Acharya et al. [38] explored the four CDs condition classes, then proposed Contourlet transformations. Lui et al. [15] proposed combining the CNNs and LSTM, in which the recurrent layer increased the sensitivity to classification by 28% relative to the CNNs itself. In the following year, Strodthoff et al. [22] only proposed CNNs architecture for classifying anterior myocardial infarction (aMI), inferior myocardial infarction, (iMI), and healthy control. Using 10-fold cross-validation, CNNs reach 93.3% and 89.7%, for sensitivity and specificity, respectively. As mentioned earlier, from the benchmark study with the same dataset implementation, our proposed unidirectional GRU architecture can improve 98.5% accuracy. According to the comparison between unidirectional and bidirectional, this study proposes the best model, unidirectional GRU. The performance of unidirectional GRU obtained an average ACC, SEN, SPE, PRE, and F1-score of 98.50%, 95.54%, 98.42%, 89.93%, and 92.31%, respectively, for 15-leads of ECG. In contrast, hitting low-point in PRE and F1-score of unidirectional RNN led to a performance of 85% and 87.27%, respectively.

CONCLUSION
ECG classification process has the greatest role in the clinical diagnosis of CDs. A high number of ECG signal classes may have different slopes of signal, timing, and amplitude, which change the ECG waveforms. ECG multiclass classification is not as simple as binary case problems. Several conventional machine learning techniques have been proposed in previous literature for specific multiclass cases. However, the main drawback of machine learning is still about feature definition and representation. This paper successfully designs a supervised deep learning model by comparing unidirectional and bidirectional recurrent network algorithms for multiclass cases to figure out the best model performance. From this study, the unidirectional and bidirectional phase are compared with deep learning for the CDs multiclass classification in ECG 15-leads. Despite a large increase in the number of parameters, the future inputs of unidirectional can still be accessed by windowing, looking ahead, or delaying output. Yet, in some cases, the bidirectional networks outperform unidirectional ones, much faster and more accurate than standard recurrent networks and time-windowed multilayer perceptrons (MLPs). Overall, both unidirectional and bidirectional GRU is better than LSTM in multiclass cases. Besides, unidirectional GRU shows the best performance model and achieves an average accuracy, sensitivity, specificity, precision, and F1-score of 98.50%, 95.54%, 98.42%, 89.93%, and 92.31%, respectively.