Measuring the conformity level of the evaluation instruments in the BSE for indonesian language subject with the 2013 curriculum

Article’s information ABSTRAK History: Submitted Revised Published : 4 November 2020 : 18 April 2021 : 30 April 2021 Tujuan penulisan artikel ini adalah untuk mendeskripsikan tentang hasil penelitian mengenai kesesuaian instrument evaluasi yang terdapat dalam BSE SMP Kelas IX tahun 2019/2020 dengan kompetensi dasar yang terdapat dalam Kurikulum 2013. Metode penelitian yang digunakan adalah analisis isi, yaitu dengan cara memadukan setiap instrumen evaluasi yang terdapat dalam BSE Bahasa Indonesia SMP Kelas IX tahun 2019/2020 dengan kritria kompetensi dasar Kurikulum 2013. Data yang dianalis adalah sebanyak 77 instrumen evaluasi yang berkaitan dengan empat materi ajar, yaitu teks eksemplum, teks tanggapan kritis, teks tantangan, dan teks rekaman percobaan. Hasil penelitian menunjukkan, bahwa rata-rata tingkat kesesuaian instrumen evaluasi dalam BSE Bahasa Indonesia SMP Kelas IX 2019/2020 adalah 89,70%. Rata -rata hasil penelitian tersebut menunjukkan, bahwa tingkat kesesuaian instrumen evaluasi yang terdapat dalam BSE Bahasa Indonesia dengan Kurikulum 2013 sangat tinggi. Hasil penelitian ini dapat dijadikan sebagai salah indicator penilaian BSE yang berkualitas. Kata kunci: Instrument evaluasi1 BSE 2 Kurikulum 2013 3


Introduction
Textbooks have an important role in learning activities at school (Asri 2017). School book is one of the main media used to carry out learning activities. Freire (2007), as quoted by Wijayanti (2020), stated that books can be a means of communication between teachers and students in carrying out classroom learning activities. In fact, textbooks are very useful in stimulating, advancing, and educating students (Efendi 2009). Quality textbooks can support the school curriculum (Rahmawati 2015) and can deliver social and cultural values (Suwandi, S., & Yunus 2016). In addition, textbooks also contribute to building student independence in learning (Anisah 2017).
Textbooks, which are now available in the form of electronic school books (BSE), are a form of effort made by the government to provide affordable textbooks that are easy to access, both by the public and students (Wahyu 2017). Therefore, BSE can be an alternative to support learning activities (Safi'i, I., & Yanti 2020). Thus, it is no doubt that BSE is currently one of the textbooks that are widely used in various schools (Indhaka, W. A., Supraptono, E., & Sugiarti 2016).
As one of the supporting factors in learning, BSE contains at least two basic elements, namely teaching materials and evaluation instruments. The evaluation instruments included in the BSE can function as exercise and enrichment materials to test the students' mastery at the materials that have been studied. Therefore, teaching materials and evaluation instruments included in the textbook must be structured in a quality manner to be used as effective media to achieve learning objectives (Safi'i, I., Witdianti, Y., Tarmini, W., & Yanti 2020).
Quality teaching materials and evaluation instruments must conform to the current curriculum. It is because the curriculum contains basic competency targets that must be met by students. It is in line with Hidayati's (2015) statement that a curriculum is designed to develop students' spiritual, social, intellectual and psychomotor skills. Therefore, the teaching materials and evaluation instruments contained in a textbook must always be directed towards achieving or mastering these student competences. Good teaching materials and evaluation instruments in a textbook can also be used to equip and train students with a set of competencies and personal values (Lasmawan 2015).
Coming from the urgency to have a quality textbook with evaluation instruments that are based on the current curriculum, a study on the conformity of evaluation instruments in BSE with the basic competencies in the current curriculum is highly needed and interesting to do. The study can be used as an effort to measure the quality of evaluation instruments in the textbook. In addition, it can also be used as one of the bases to find out the quality of a textbook as well as a framework for developing a quality textbook and evaluation instrument.
Several previous studies on this topic have been carried out by a number of researchers. Some researchers (Safi'i, I., Listeini, F. Y., & Tarmini 2020) reviewed the quality of evaluation instruments in BSE for Indonesian subject in terms of materials, language and construction. In their study, Safi'I, et al. focused their study on evaluation instruments related to one of the teaching materials in Junior High School BSE, namely Membuat Laporan Hasil Pengamatan (Making Observation Result Reports). From the research results, it was found that the evaluation instrument related to Making Observation Result Reports did not meet the ideal criteria, especially in terms of the materials and construction. In terms of the materials, 75% of the evaluation instruments were not yet relevant; while for the construction aspect, 32% of the evaluation instruments were not constructively correct.
Furthermore, Pratiwiningtyas, B. N., Susilaningsih, E., & Sudana (2017) also conducted a similar study on the readiness of reading evaluation instruments in Indonesian subject for primary school students. Pratiwiningtyas et al. stated that at the time of the research there was no standardized evaluation instrument to measure students' reading ability. Therefore, in their research, they emphasized the importance of developing a standardized reading evaluation instrument. They developed an evaluation instrument referring to the PIRLS model.
As scientific works, the two studies previously mentioned need to be positively appreciated. Both of the studies attempted to bring up some facts related to the readiness of the evaluation instruments in the electronic textbook for Indonesian language subject. However, a more comprehensive study on the quality of a textbook needs to be carried out. A more comprehensive explanation on the quality of evaluation instruments in a textbook can serve as a stronger basis for making efforts to improve the evaluation instruments in textbooks. It is expected that the textbook will gradually meet the objectives of Indonesian language learning.
Starting from the two points above, this article will present the results of a more comprehensive study on the evaluation instruments in the BSE for Indonesian language subject. This article will describe the research results on the conformity of the evaluation instruments in the BSE for Indonesian language subject for Junior High School with the teaching materials included in the 2013 Curriculum. Some of the teaching materials in the curriculum are in the forms of exemplum texts, critical response texts, refutation texts, and experimental record texts.

Method
The method used in this research was content analysis, i. e. the procedures proposed by Mayring (2010), consisting of 1) establishing research questions, 2) establishing research criteria, 3) conducting analysis, 4) revising criteria if needed 5) conducting further analysis, 6) determining the category of findings, 7) examining the results of the study 8) interpreting the results of the study.
The data source used in this study was the BSE for Indonesian language subject for grade VIII students in the academic year of 2019/2020. The data obtained in this study was the conformity level of the evaluation instruments for the materials of exemplum, critical response, refutation, and experimental record texts. The measurement of the conformity level of the evaluation instruments with the 2013 curriculum was carried out by presenting the conformity and non-conformity findings with the total number of instruments. The more evaluation instruments that are relevant to basic competencies in the 2013 Curriculum, the better the evaluation instruments in the textbook are. Conversely, the fewer evaluation instruments that are relevant to the basic competencies in the 2013 Curriculum, the worse the evaluation instruments in the textbook are. The findings on the conformity level of the evaluation instruments for each teaching material are categorized based on the following table.

Result and Discussion
Based on the study, relevant data have been obtained regarding the conformity level of the evaluation instruments in the BSE for Indonesian language subject for Grade IX of Junior High School in the academic year of 2019/2020 with the basic competencies in the 2013 Curriculum. The following are the findings of the research data related to the evaluation instruments for the materials of exemplum, critical response, refutation, and experimental record texts in the BSE for Indonesian language subject for grade IX students in the academic year of 2019/2020. Exemplum Text Exemplum text can be interpreted as a text that tells the behavior or experiences of a character. Through an exemplum text, students are expected to gain an imaginative experience that can enrich their attitudes and knowledge; and develop their creativity. As stated by Khalimah, E., Martha, I. N., & Nurjaya (2016), exemplum texts aim to deliver a moral message. The moral message received by students through exemplum texts can be related to social, spiritual, and intellectual dimensions. Thus, through the habituation of reading, comprehending, understanding, and analyzing exemplum texts, students' maturity will also be established.
Based on the analysis of the evaluation instruments in the BSE, the number of evaluation instruments related to exemplum text teaching materials was 21. The number of instruments that conformed to the 2013 Curriculum was 19 or 85.71%, while the number of evaluation instruments that did not conform to the 2013 Curriculum was 3 or 14.3%. Thus, in general, the conformity level of the evaluation instruments related to exemplum text teaching materials in the BSE for Indonesian language subject for grade IX students with the basic competencies in the 2013 Curriculum is considered to be high.
The conformity level indicates that most of the evaluation instruments related to exemplum text teaching materials are categorized as 'high'. It means that the evaluation instrument has measured what needs to be measured (Sugiharni 2017). Evaluation instruments that have a high validity will have a smaller chance for measurement errors (Hidayati, K., & Listyani 2010). Conversely, an invalid evaluation instrument will be likely to have measurement errors. An invalid evaluation instrument is not suitable to measure the competence needs to be measured. In other words, a number of evaluation instruments related to exemplum text teaching materials have been made well enough to support students' competencies in comprehending, analyzing, and understanding the texts.
Some evaluation instruments that are categorized as not conforming to the basic competencies in the 2013 Curriculum are related to Basic Competency 3.4, which is identifying an exemplum text, both oral and written.
Lexical identification means determining or establishing the identity of an object so that it can be fully recognized. Through the basic competency of identifying an exemplum text, both oral and written, students are expected to be able to determine or establish the identity of an exemplum text in order to get a good understanding of the text characteristics. The identification of an exemplum text can be made from the content, structure, and various other characteristics related to the text. Therefore, the evaluation instruments given to students should explore students' abilities in identifying the text. Thus, students will finally get a complete understanding of the differences between an exemplum text and other texts. In fact, a further competency required from students is to produce an exemplum text.
The following are several evaluation instruments which are declared to be incompatible with Basic Competency 3.4, which is identifying an exemplum text, both oral and written. 1) In order to better understand the use of the four adverbial groups, students are asked to identify the words that are included in the adverbs of place, time, purpose and method in the text titled "Putri Tangguk"! (page: 25) 2) After understanding the intra-and inter-sentence conjunctions, students are asked to identify the conjunctions and their meanings in the text titled "Putri Tangguk"! (page: 28) 3) To practice your understanding of compound sentences, take a closer look at the exemplum text of "Putri Tangguk" above! Identify the compound and complex sentences, and the conjunctions found in the following text! (pages: 30-31) The three examples of evaluation instruments above do not measure students' competence in identifying exemplum texts, but rather to measure students' competence in identifying linguistic aspects. Therefore, the evaluation instruments do not measure what needs to be measured, i. e. competencies about exemplum text as stated in the 2013 Curriculum. The instruments that do not measure what competencies need to be measured certainly cannot enhance student competencies as emphasized in the 2013 curriculum. In addition, the evaluation instruments in the textbook that are not relevant to the curriculum will also affect the quality of the textbook. In other words, the instruments cannot be used to achieve maximum student competence. A good textbook, in this case BSE, must be a learning resource that can support the achievement of student competencies (Sari, E. A., Sugiyanto, R., & Tjahjono 2012).

Critical Response Text
Critical response text is a text that contains the delivery of responses in the form of expressions of agreement, disagreement with an in-depth analysis and argumentation. Critical response text is one of the texts used to improve student competency in critical thinking. Critical thinking can train students to get an understanding of a more complex and deeper meaning. In addition, through the habituation of responding critically to problems through the text in the evaluation instruments, students will be able to establish their competency in better decision making and problem solving (Najeeb 2013).
The number of critical response text evaluation instruments in the BSE for Indonesian language subject for Grade IX in the academic year of 2019/2020 was 18. The number of evaluation instruments that was in accordance with the 2013 Curriculum was 16 or 83.33%, while the number of evaluation instruments that was not in accordance with the 2013 Curriculum was 2 or 16.66%. The number of evaluation instruments for the critical response text teaching materials that conformed to the 2013 Curriculum is categorized as 'high'.
The type of evaluation instruments on critical response texts in the BSE for Indonesian language subject for Grade IX that did not conform to the basic competency 4.1 which is comprehending the meaning of critical response texts, both oral and written, are as follows.
1) The critical response text of "Remaja dan Game Online" above has several words whose meanings are difficult to understand, students are asked to use the Great Dictionary of the Indonesian Language (KBBI) to describe their meanings. Do this assignment according to the provided format! (page: 102-105) 2) The critical response text titled "Homeschooling" above has several words whose meanings are difficult to understand, students are asked to use KBBI to describe the meaning. Do this assignment according to the provided format! (page: 116-119) 3) Define the main idea within the part of the structure of "Homeschooling" critical response text above. Do the following assignments according to the provided format! (page: 119-120) The three evaluation instruments above do not measure students' competence in understanding or capturing the meaning of a critical response text. Instruments 1 and 2 only measure the students' ability in mastering vocabulary, especially the difficult words found in the text. This type of measurement is definitely partial, because the understanding of a text is comprehensive, meaning that the series of vocabulary is interpreted as a whole based on the context of the text. Therefore, it is possible that students will be able to understand the text and give critical responses to the contents of the text even though there are some vocabulary items that they do not understand.
Furthermore, the third evaluation instrument is used to measure students' ability in determining the main sentence in each paragraph of the critical response text. The student's ability to determine the main sentence of the paragraph is not always directly proportional to the student's ability to understand the content of the text as a whole. Therefore, the student ability to determine the main sentence also cannot be used as a basis for measuring students' ability to identify various forms of critical responses in the text. Thus, the third evaluation instrument did not conform to the basic competencies in the 2013 Curriculum.

Refutation Text
Refutation text is a text that contains responses in the form of rebuttals or rejection of policies or general problems with supporting evidence and data (Triningsih 2018). The structure of a refutation text consists of three parts, namely an introduction, argument, and conclusion. The introduction of the refutation text contains the issue or problem. The arguments are statements that contain contradictions or rebuttals, while the conclusion contains reaffirmations and suggestions of what has been refuted.
Through refutation texts, students are expected to improve their critical thinking, argumentative, and problem-solving skills. It means that, in the refutation text, students are not only accustomed to questioning the true or untrue information, but also providing strong reasons for their opinions. Students must also provide the best solution to the critical problem. This kind of thinking ability is in line with what has been expressed by Carter, Creedy, and Sidebotham (2017) regarding the ability to think in a directed manner, which is a way of thinking that includes efforts to be careful in making a decision and appropriate action for a problem. This kind of critical and analytical thinking will improve students' competence in making logical decisions (Heijltjes et al. 2014).
The number of evaluation instruments in the BSE for Indonesian language for Junior High School in the academic year of 2019/2020 that were related to refutation text was 19. A total of 16 evaluation instruments (84.21) were declared to conform to the basic competencies in the 2013 Curriculum, while 4 evaluation instruments (15.79%) did not conform to the basic competencies of the 2013 Curriculum. Based on the number of instruments that were in accordance with the curriculum, it can be concluded that the evaluation instruments related to refutation text in the BSE for Indonesian language subject have a high level of conformity.
Several types of evaluation instruments regarding refutation text in the BSE that did not conform to the 2013 Curriculum Basic Competency 3.2, which is distinguishing both oral and written refutation texts and Basic Competency 3.3, which is classifying refutation texts, both oral and written, are as follows.
1) Students are asked to read and analyze the text titled "Pengurangan Subsidi Bahan Bakar Minyak" carefully, and then determine the structure of the text and its linguistic characteristics! (pages: [136][137][138][139] 2) Students are asked to read and analyze the text titled "Perlukah Pengurangan Subsidi BBM?" carefully, and then determine the structure of the text and its linguistic characteristics! (pages: 139-142) 3) Students are asked to read and analyze the text titled "Subsidi BBM Perlu Dikurangi" carefully, and then determine the structure of the text and its linguistic characteristics! (pages: 143-145) 4) Students are asked to classify words or phrases in the refutation text titled "Kebijakan Mobil Murah" that are considered difficult, then define the words or phrases. In writing the definitions, students can use  Evaluation instruments 1, 2, and 3 were intended to measure students' competence in distinguishing refutation texts. When we take a look at the instructions, the three evaluation instruments are not relevant to Basic Competency 3.2 because the evaluation instruments required students to identify the structure of a refutation text. The students' ability to identify the structure of a refutation text is certainly different from the ability to distinguish refutation texts. Student competence in distinguishing refutation texts requires students to compare or identify various differences in a refutation text with other texts. Students can also be asked to compare one refutation text with another refutation text. This activity can simultaneously measure the students' ability to understand the content and differences of each text and parts of the refutation text.
Furthermore, evaluation instrument number 4 instructs students to classify the words or phrases in the refutation text. The instrument emphasizes students' understanding of linguistic aspects. This competency is very partial and certainly not relevant to the basic competency of the 2013 Curriculum, namely classifying refutation texts, both oral and written. The basic competency emphasizes that students understand and are able to classify refutation texts, especially from the structure, including the problem, arguments, and conclusion. With the ability to identify the text, students also get a glimpse of the modeling for refutation texts so that they can compile a refutation text later on. Through modeling and guidance from the teacher, student competence will be improved. This is in line with what Chang & Hwang (2018) stated as quoted by Safi'i, I., Tarmini, W., & Ilyas (2020), that teacher guidance not only benefits students in improving their performance, but also increases their learning motivation, critical thinking tendencies, and group self-efficacy.

Experimental Record Text
Experimental record text can be interpreted as a text that contains a description of the report on an experiment or research results regarding a certain object or event that is presented systematically, logically, empirically and can be justified (Kayati 2019). Through the evaluation instruments for experimental record texts, students are expected to be able to understand the structure and linguistic rules of the text. Students can also get a model in composing an experimental record text properly. Finally, students are also expected to be able to reconstruct or construct the experiment record text in a good manner. These competencies are included in Basic Competencies 3.1, 3.2, 3.3, 4.1, 4.2, 4.3, and 4.4.
The number of evaluation instruments related to experimental record text was 19. All evaluation instruments were in accordance with the basic competencies of the 2013 Curriculum, meaning that the conformity level of the instruments is categorized as 'very high'. The conformity level of the evaluation instruments for the experimental record text shows that the formulation of the evaluation instruments as a part of material enrichment has been well prepared. Good evaluation instruments in the textbook certainly enhance the quality of the textbook or BSE.
Some of the evaluation instruments proposed are as follows.
1) Students are asked to understand the text titled "Percobaan Membuat Teleskop Sederhana" and answer the questions related to the text! (pages: [187][188][189][190] 2) Before determining the structure and type of the text titled "Membuat Tinta Tidak Terlihat", students need to answer the following questions ! (pages: 195-198) 3) Before determining the structure and type of text titled "Kenaikan Tarif Tol Tidak Mempertimbangkan Hak Konsumen", students need to answer the following questions! (pages: 198-200) 4) Students are asked to classify words or phrases in the experimental record text titled "Jebakan Tikus Sederhana" which they consider difficult. Define the words by using KBBI! (pages: 203-205) 5) Formulate an experimental record text in groups (Task 2: formulating an experimental record text (pages: 220-221) The five examples of evaluation instruments in the BSE were in accordance with the basic competencies in the 2013 Curriculum. Through these five evaluation instruments, students' competencies in understanding, analyzing, and constructing experimental record texts are expected to be improved and more refined. In terms of readiness, these instruments are very suitable and ready to be used to support quality textbooks. In line with this, Mudzakir (2010) also stated that the quality of a textbook can be seen from the provision of ready-to-use assignments or evaluation instruments that are relevant to the curriculum competencies. With that quality, a textbook can be a support for teachers and students in achieving learning goals. As emphasized by Efendi (2009), textbooks are very important tools to achieve learning objectives.

Conclusion
Based on the results of the research and discussion above, it can be concluded that, in general, the evaluation instruments related to the Exemplum Text, Critical Response Text, Refutation Text, and Experimental Record Text teaching materials in the BSE for Indonesian language subject for Grade IX of Junior High School in the academic year of 2019/2020 have a very high conformity level with the basic competencies of the 2013 Curriculum, with a percentage of 88.31%. The conformity level of the evaluation instruments with the basic competencies of the 2013 Curriculum also shows that, in general, the evaluation instruments in the BSE are very suitable to be used as evaluation and enrichment materials to enhance student competencies. The feasibility of the evaluation instrument can also be an indicator of a quality book.
This study has not examined the evaluation instruments related to all teaching materials in the BSE for Indonesian language subject for junior high school students. Therefore, a more comprehensive similar study on the evaluation instruments in the BSE for Indonesian language subject needs to be carried out. As a result, more comprehensive data regarding the quality of the evaluation instruments in the BSE for Indonesian language subject can be obtained.