The accuracy of Hopkins Symptom Checklist – 25 (HSCL-25) depression subscales (Indonesian version) on adolescents

Adolescence is one of the lifetime periods when a person is susceptible to many mental health problems and disorders (Merikangas et al., 2010). One of the most widespread mental health problems is depression (Goyal, Srivastava, & Bansal, 2009; Jayanthi & Thirunavukarasu, 2015; Kementerian Kesehatan, 2013; The National Institute of Mental Health, 2017). Studies showed that around 23% of adolescents had experienced symptoms of depression in their lifetime, with as many as .2% to 9.5% of community-based adolescents suffering from severe and extreme depression (Nair, Paul, & John, 2004; Sund, Larsson, & Wichstrøm, 2011). Another study showed that the prevalence of both major and minor depression in adolescents is larger than 11% (Kessler & Walters, 1998; Merikangas et al., 2010). Adolescents show a slightly different symptom of depression, particularly with regards to the prolonged sad mood. Sadness is often expressed in terms of unstable and irritable mood, long and persistent boredom, low energy, and awkwardness in a social ARTICLE INFO AB STRACT


Introduction
Adolescence is one of the lifetime periods when a person is susceptible to many mental health problems and disorders (Merikangas et al., 2010). One of the most widespread mental health problems is depression (Goyal, Srivastava, & Bansal, 2009;Jayanthi & Thirunavukarasu, 2015;Kementerian Kesehatan, 2013;The National Institute of Mental Health, 2017).
Studies showed that around 23% of adolescents had experienced symptoms of depression in their lifetime, with as many as .2% to 9.5% of community-based adolescents suffering from severe and extreme depression (Nair, Paul, & John, 2004;Sund, Larsson, & Wichstrøm, 2011). Another study showed that the prevalence of both major and minor depression in adolescents is larger than 11% (Kessler & Walters, 1998;Merikangas et al., 2010).
Adolescents show a slightly different symptom of depression, particularly with regards to the prolonged sad mood. Sadness is often expressed in terms of unstable and irritable mood, long and persistent boredom, low energy, and awkwardness in a social diagnostic interview guide composing of SCID and few adolescents' depression theories so it would be more accurate as the gold standard of adolescents' depression.
The accuracy testing of HSCL-25 has been replicated in many studies across nations and cultures. These studies mainly showed good accuracy with moderate to high-level sensitivity and specificity in comparison to various gold standards (Fröjdh, Håkansson, & Karlsson, 2004;Hinton et al., 1994;Lee, Kaaya, Mbwambo, Smith-Fawzi, & Leshabari, 2008;Ventevogel et al., 2007). The HSCL-25 Indonesian adapted version has undergone cultural validation (Turnip & Hauff, 2007) but the diagnostic testing, particularly accuracy testing of this version has never been done before.
This study aimed to test the accuracy (sensitivity and specificity) of HSCL-25 Indonesian adapted version for depression subscales. This study also aimed to test the appropriateness of the current cut-off score and later explore the optimum cut-off that could be used to differentiate between adolescents with depressive tendencies and those who are non-depressive. This study is part of a longitudinal study assessing the mental health status among adolescents in Indonesia. We hope that this study could increase the validity of the Indonesian version of HSCL-25 depression subscales, improving the preventive measures of depressive in adolescents.

Method
This research was a non-experimental study that compares two types of data, quantitative (HSCL-25 depression subscale score obtained from the preliminary study) and qualitative (depression diagnostic interview obtained from this study). These data collected to obtain the accuracy of HSCL-25 depression subscales, which are sensitivity, specificity, likelihood ratios, & predictive values.
This study was part of a longitudinal mental health study of Indonesian adolescents. Thus, the current participants were sampled from the preliminary research. The preliminary study was conducted on 623 participants from five randomly chosen samples of high schools that represent five Jakarta administrative regions (West, East, South, North, and Central Jakarta). Participants were freshmen students (grade X) aged between 14 to 18 years old. During the preliminary study, participants were asked to complete various mental health instruments, including HSCL-25 to assess their anxiety and depression.
Once the results were obtained, we used the HSCL-25 depression subscale scores to divide the participants into two categories based on the cut-off score (1.75). This HSCL-25 cut-off score is widely used across the cultures to determine whether someone is screened in the high or low-risk group in anxiety and depression (Dewayani et al., 2011;Hollander, Ekblad, Mukhamadiev, & Muminova, 2007;Sandanger et al., 1998;Veijola et al., 2003;Winokur, Winokur, Rickels, & Cox, 1984;Wulanari, 2013). The high depression group consists of participants who scored more than 1.75, while the low depression group comprised of participants whose depression subscale scored less than 1.75. These groups (excluding participants with comorbidity) were used as the population of the present study.
Next, we randomly selected 20 samples from each depression groups (of the preliminary study) to be the participants in this study. The researcher team (author was excluded) conducted the selection of the participants to avoid any expectation biases when collecting the data (double-blind study). There was a total amount of 40 random samples from three randomly chosen sample schools that participated in this study.
This study was a cross-sectional study conducted around November 2017, when the participants were in the middle grade of high school (Grade XI). We used HSCL-25 depression subscale as our primary instrument and a semi-structured interview guide as the gold standard. HSCL-25 is a brief self-report measure to detect the severity of anxiety and depression symptoms (Derogatis et al., 1974;Najarian & Davoodi, 2001). It is a short version of HSCL, containing 25 subscales that assess anxiety (10 items) and depression (15 items) (Derogatis et al., 1974). HSCL-25 measures the estimation of symptoms severity through a form of Likert scale, ranging from one (not at all) to four (extremely). The final score was obtained by dividing the total score with the number of items. The mean score used as the standard for high mental health disorder is 1.75, which was then used as the cutoff point for the preliminary study (Najarian & Davoodi, 2001). This entire instrument showed excellent internal consistency, while the depression subscale showed a Cronbach's α of .85 (good) (Tay, Jayasuriya, Jayasuriya, & Silove, 2017). The Indonesian version of HSCL-25 was adapted by Turnip & Hauff (2007) and had been through cultural validity testing and resulted in good validity (Turnip & Hauff, 2007).
The gold standard we used in this study is a semi-structured diagnostic interview guide and clinical observation. This method was chosen because it is primarily used in health-related studies, specifically in assessing patients' medical problems, becoming the basis for diagnosis and proper treatment plan (Rudolph, 2005). We constructed the interview guide based on many works of literature on adolescent depression (Hammen & Watkins, 2008;Keena, 2005) and the Structured Clinical Interview for DSM IV-TR (SCID) guide. We also used clinical observation to check the symptoms that they reported. SCID is a structured-interview guide for establishing the clinical diagnosis based on DSM IV-TR which is divided into several modules for assessing various mental health disorders. The depression symptoms belong to module A (First & Spitzer, 2002). SCID has good reliability in diagnosing Axis-I disorders in the DSM IV-TR (Lobbestael, Leurgans, & Arntz, 2011). The decision to add the adolescents' depression theory on top of SCID, was done to increase the usefulness of the gold standard in capturing the depression phenomena among adolescents.
We measured the sensitivity and specificity of HSCL-25 through descriptive and crosstab analysis. Additionally, we also used the Receiver Operating Characteristic (ROC) curve analysis (conducted using IBM SPSS version 22.0) to examine whether the current cut-off score is appropriate for this population as well as explore the optimum cut-off point.
This study was obtained ethical clearance from the Ethical Committee of the Faculty of Psychology, University of Indonesia. Once permission was granted, we conducted individual interviews for all participants who have agreed to participate in the study in either the counseling or student council room. The interview was roughly 30-60 minutes per student, containing the introduction, explanation about the study, demographic and clinical data collection, and debriefing procedure. Several students rejected to participate after we asked the final permission. After we conducted another randomization, a total of 21 participants from high depression group and 19 from the low depression group were selected to participate in the interview process. The high and low depression groups were selected based on the HSCL-25 score in the preliminary study. Table 1 presents the demographic information of the study participants. Table 1 The Distribution of Study Participants (n = 40)

Results
In the preliminary study, 520 out of 623 samples (83.5%) endorsed the high depression characteristics, based on the depression subscale of HSCL-25 (mean = 2.69) (see Table 2). This prevalence is very high compared to other mental health issues obtained in the preliminary study. Meanwhile, the diagnostic interview results in 14 out of 40 samples (35%) being endorsed into the high depression group (mean = 2.60, SD = .50), with the remaining rest 26 (65%) classified as either in no or low depression group (mean = 1.80, SD = .80). Table 2 The Prevalence of Depression in the Preliminary Study Table 3 depicts the score comparison between HSCL-25 depression subscale and a diagnostic interview. The crosstab analysis shows that out of 14 students classified as having high depression symptoms by the diagnostic interview, 12 of them (85.7%) were also categorized as high depression by the HSCL-25 depression subscales (true positive group) (Spiers et al., 2013). The remaining two participants were categorized as low depression group by the HSCL-25 depression subscales, namely the false negative group (Carlson, 2012). Next, our analysis also resulted in 17 participants being classified into the true negative group (65.4%) (Spiers et al., 2013). Meanwhile, the remaining nine participants were categorized as high depression by HSCL-25 depression subscales. This group is considered as the false positive group (Carlson, 2012). The large false negative group could be caused by misinterpretation of the subscales as either participants' current emotion or everyday life experiences, or just that they did not report their full consent in the interview. Based on these findings, then the sensitivity of HSCL-25 depression subscales was 85.7%, while the specificity was 65.4% (Spiers et al., 2013).  Next, the result shows that 12 out of 21 participants were categorized as high depression group by both HSCL-25 depression subscales and the diagnostic interview, resulting in a positive predictive value of .57 (57%) for HSCL-25 (Spiers et al., 2013). Meanwhile, 17 out of 19 participants were categorized as low depression group by both HSCL-25 depression subscales and the diagnostic interview, indicating that the negative predictive value of HSCL-25 depression subscales is .90 (90%) (Hulley et al., 2007). The positive likelihood ratio of HSCL-25 depression subscales is 2.50, and the negative likelihood ratio is .20 (Wiese, 2006). Table 4 presents the predictive value and likelihood ratios.

Table 4 Predictive Values and Likelihood Ratios Results
The area under the curve (AUC) of ROC (Figure 1) shows that the current sensitivity and specificity scores .75 (the confidence interval is between .59 and .91). The curve also formed in the middle to left of the entire area and the area under the ROC curve (AUC). The AUC and optimum cut-off points were presented in Table 5.  Table 5 Area

Under the Curve (AUC) of ROC and The Optimum Cut-Off Points
Next, by the distribution of symptoms analysis, the depression symptoms showed in the depression subscales of HSCL-25 is similar to the diagnostic interview. Table 6 shows the item distribution.

Discussion
Our findings show that the HSCL-25 has a sensitivity percentage score of 85.7%, while the specificity is 65.4%. These findings show that the Indonesian adapted version of HSCL-25 is considered to be an excellent instrument to screen people with symptoms of depression (Zhu, Zeng, & Wang, 2010). This is indicated by the higher numerical value of the sensitivity and specificity. This instrument is also suitable for adolescents. It is an excellent tool for preventive measures due to its' high sensitivity (Wassertheil-Smoller & Smoller, 2015). These findings answer the research problem on whether the depression subscales of HSCL-25 have better sensitivity and specificity for detecting depression in adolescents, compared to the gold standard. The above results are also supported by the fact that the +LR was considered as fair because it is not bigger than one. Meanwhile, the -LR was considered as good because it is lesser than one (Wiese, 2006). Thus, it show that HSCL-25 depressions subscales are considered as a fair to a good instrument. Similarly, the AUC analysis and ROC curve also indicate that the HSCL-25 depression subscale is a good instrument (Kumar & Indrayan, 2011).
However, we suggest the cut-off score to be increased to 2.50 if the instrument is intended to be used as a diagnostic tool for people with high depression characteristics. This is because HSCL-25 can be used easily by many non-health professionals (Ventevogel et al., 2007). Thus, it would be more practical for them to use a single for all mental health assessments. This cut-off score would be good for HSCL-25 depression subscales as a diagnostic instrument because it still has a good compromise of sensitivity and specificity (sensitivity: .714, 1specificity: .269).
Next, based on the interview, mood change was a factor that many participants experienced (10 out of 14). This finding supports the existing literature that state that two of the most typical symptoms of depression found in adolescents were fluctuating mood and irritability (Keena, 2005). For that reason, we suggest adding items on mood changes to measure symptoms of depression in adolescents. By adding the mood change item to the HSCL-25 depression subscales, we hope that this instrument will better describe adolescents' depression.
Our overall findings theoretically contribute to expanding the knowledge repertoire about the validity of HSCL-25, particularly the depression subscales. It also contributes to becoming the first study to describe the accuracy of HSCL-25 depression subscales on adolescents in Indonesia. Practically, these results could aid in diagnosing adolescents with mental health condition more accurately, allowing for a proper treatment plan to be constructed at an early stage.
However, several limitations exist in this study. First, the study participants were typically very homogenous in terms of age (16-18 years), familial, economic, residential, school, and cultural backgrounds. In other words, the result is limited in the matter of generalization. We suggest future studies to involve participants from a more heterogenic demographic background to improve the generalizability and representativeness of the result.
Moreover, there were also issues regarding the relatively large number of false negatives in this study that were caused by misinterpretation of the items. We recommend prospective studies to use the diagnostic interview with longer duration or rapid sessions with a more in-depth approach. This suggestion is made to ensure that the researcher and participants have built the necessary rapport for participants to talk more openly about their mental health conditions. Additionally, it will also aid researchers in understanding the stories of each participant more deeply, clarifying the process of understanding the symptoms of depression.

Conclusion
According to the findings of this study, HSCL-25 depression subscales (Indonesian version) has the sensitivity percentage score of 85.7% and specificity of 64.3%, implying that it is a fair to good screening tool for depression in adolescents. These results are also supported by the predictive values, likelihood ratios, and ROC curve analysis results. The current cut-off for Indonesian adolescents (1.75) is an optimum cut-off score. Thus, it can still be used continuously for preventive measures of adolescents' depression. However, the cut-off score should be changed to 2.50 if used for diagnosis purposes. From the symptom distribution analysis, we also suggest adding the mood changes item to complete this instrument for use on adolescents.