Correlation of Isomorphic, Open-Ended, and Conventional Score on the Ability to Solve Kinematics Graph Questions

This study explores how difficulties are perceived in terms of three assessment styles and how isomorphic and conventional to open-ended can reflect students' learning difficulties on kinematic's graphs. This study used a quantitative descriptive form of survey research. The respondents were twenty-nine senior high school students from Semarang. The instruments were adopted from the Test of Understanding Graphs-Kinematic (TUG-K) by Robert J.Beichner (1994). There were nine questions used from the TUG-K with open-ended questions that have been validated. The results show that students have difficulties learning about kinematic's graphs, and there is no relation between open-ended scores and the other two scores. Nonetheless, judging from the correlation coefficient, the correlation between open-ended and conventional tasks is 0.075, which means low correlation, while for isomorphic score and openended score in the amount of 0.109 is higher than the correlation of traditional and open-ended. This correlation is insignificant. However, the closest correlation direction to the open-ended test is isomorphic compared to the conventional one. Therefore, it is concluded that isomorphic assessment is more useful than traditional review in kinematic graphics.


INTRODUCTION
Information about student learning outcomes and difficulties is needed as a reference for learning improvement (States et al. 2018). Meanwhile, teachers' initial information about learning difficulties can be used to develop targeted learning designs on learning topics that are not yet understood by students (Crisp and Ward 2008;Kallesta and Erfan 2018;Oyelami Benjamin Oyediran 2010). Information about learning difficulties, in this case, is needed to identify which students have learning difficulties to be of more concern by teachers to solve the questions they face (Haqiqi and Sa'adah 2018). In addition, information on learning difficulties also provides an overview of students' abilities on the learning topics being discussed (Kola 2017). In physics education, the use of information on student learning difficulties due to concepts in physics is often misunderstood by students, especially for basic concepts such as kinematics (Muliyati, Septiningrum, Ambarwulan, and Astra 2020).
In understanding kinematics concepts, it does not always use verbal language and calculations in formulas. However, it can also be explained through graphs (Kilic et al. 2012). Graphs in kinematics themselves make it easier for students to understand kinematics even though it often causes students confusion because students themselves are unable to read it. On the other hand, the ability to understand graphics is a discipline that must be mastered by kinematics students (Phage et al. 2017). The primary role of pictures is to summarize data and facilitate the delivery of information to be interpreted (Kali 2005). Therefore, a fair assessment is needed to provide input to the kinematic neural learning topic's difficulty (Zavala et al. 2017).
Several methods have been developed to obtain information about learning difficulties in physics, still have limitations. Interviews require a long time, need practice, interviewer bias, doubtful findings, and complex data (Adadan and Savasci 2012). Open-ended test, language problems make identification of students' misconceptions difficult because students are generally less eager to write their answers in full sentences (Gurmu 2016). A conventional test is one level multiple-choice test in which the students' choice in the correct answer still does not certainly give the right reason. Two-tier multiplechoice tests, the limitations of this test are in distinguishing misconceptions, errors, or scientific knowledge. Three-level multiple-choice test, students are asked to know their confidence in the choice in the first two levels, which allows underestimating the proportion of students 'lack of knowledge and students' grades that are too high. Four-tier multiple-choice tests, requiring longer testing time, are not recommended for use in attaining goals, and the choice of students' answers at the first level can influence their choice of response at the level of reasoning. (Gurel, Eryilmaz, & McDermott 2015).
Based on the lack of existing assessment instruments, new assessment instruments are needed in identifying student learning difficulties. This study offers a further isomorphic assessment analysis in determining students' on problems kinematics graph concept. The isomorphic test assists students in learning certain material with different contexts (Rahmawati et al. 2020). Compared to other tests, the isomorphic test has a better reliability level and relatively simple . The isomorphic test consists of three items of multiple choices with different contexts (Rahmawati et al. 2020). The form of isomorphic test is presented in printed media to avoid students' dishonesty in doing the test during the research (Kusairi 2012;Universitas Airlangga, Indonesia et al. 2018). However, for the implementation of the learning, the isomorphic test used a too recommended computer. The isomorphic test uses a computer that can give quick and precise feedback (Nadhiif et al. 2015). Isomorphic tests have previously been used with high accuracy results in measuring student understanding (Diyanahesa et al. 2017). However, it has never been seen how to compare isomorphically and conventionally to open-ended tests. Thus, isomorphic tests can be a practical choice compared to multiple options in assessing students' learning difficulties in physics.

METHODS
This research is included in survey research with quantitative descriptive form. The subjects in this study were class X students in one private school in the city of Semarang. Students at this school are selected students from the Central Java area. They have gone through rigorous selection in academics before entering the school. Twenty-nine participants attended this study. Subjects were taken based on the consideration that they had received kinematics material before. Meanwhile, the condition of this school is very good with complete facilities and infrastructure.
Besides, the teaching teacher also received professional training from the local government. The research instrument contains questions that discuss reading graphs in kinematics. The instrument was developed in the form of isomorphic problems. Three items include the same material but with different editors. The questions are placed randomly, so they are not recognized. The question was adopted from the Robert J. Beichner (1994) Test of Understanding Graphs-Kinematic (TUG-K). The problem is modified by shifting the language and arrange the question form so that it is easy for students to understand. The question modification is conducted to make students understand the question well and avoid interpretation mistakes (Cunha and Sasaki 2020). Modification is done through expert assistance, namely the Physics lecturer of Universitas Negeri Malang. Three sub-materies were taken from the test: determining the speed, determining the acceleration, and determining the displacement. There are nine items, as shown in  Providing one open-ended question with the same material can check students' abilities in more detail. Students are asked to translate a graph, synthesize a new graph, and interpret the graph they made in open-ended questions. With this open-ended question, it is hoped that students will have the opportunity to explore their knowledge in depth.
Several experts have validated the instrument, so it is suitable to be used as a measurement tool. The validated instrument was sent to the physics teacher at the school to be tested. Students are not given a time limit for spelling out questions to think according to their abilities. Even so, students remain under the supervision of the teacher is doing it. The average time needs by students to do the test is one hour. Researchers did not conduct research directly but entrust the teacher concerned. The time required for data retrieval is two weeks.
Answer sheets that have been received are scored conventionally, isomorphically, and open-ended. Open-ended questions are scored based on rubrics that have been made by previous researchers. The conventional maximum score is 9, the isomorphic score is 6, and the score for open-ended questions is 9. The answer to the open-ended question has been corrected by two researchers to be made as a comparison to minimize subjective judgment. The rating rubric for open-ended questions can be seen in TABLE 2 below: Researchers processed the three scores (conventional, isomorphic, and open-ended) to find the correlation. The researchers conducted a descriptive analysis to find out the mean, median, and standard deviation. Researchers also presented data on the acquisition of scores in the form of pie and histogram to determine the material parts that students have difficulty. Then the score was tested for normality to determine the correct correlation analysis. Researchers used Spearman's Rank correlation analysis because the data were not normally distributed. Students' maximum score is not too high, 5 on conventional scores, 2 on isomorphic scores, and 3 on open-ended scores. The display of learning difficulties for each kinematics graph material in FIGURE 1 below. Prerequisite tests in this study include tests of normality, homogeneity, and linearity. In the normality test, the significance value of conventional scores> 0.05 indicates that the data is normally distributed. Instead of isomorphic and open-ended scores, the significance value is <0.05. This shows that the data is not normally distributed. For homogeneity tests, because the data is taken from the same person, it can be homogeneous. For the linearity test the three scores are stated linear. To test the correlation between conventional, isomorphic, and open-ended assessments was done by using the Sperman's Rank's Assistance SPSS 16.0.
Correlative analysis test results show that conventional scores and isomorphic scores have a very close relationship marked by asterisks and significance values <0.05. The correlation between these two scores is included in the strong correlation with the correlation number 0.768. This result also shows that the open-ended score does not correlate with the other two scores, but we can see the  There are significant differences in the use of multiple-choice, isomorphic, and open-ended assessment instruments in measuring students' learning difficulties in the kinematics graph concept.

Discussion
Comparison of the average score of conventional and isomorphic assessment is different. The average score using the test's traditional assessment is 2.6, while the isomorphic test average is 0.448. These mean differences indicate that conventional and isomorphic assessments have a significant score difference in measuring student learning difficulties on the concept of kinematics graphs.
Isomorphic is a form of test that presents various shapes and structures of questions but with different material (Merriam-Webster, 2019) This average difference occurs because an isomorphic assessment is done by giving students a score by considering each item's answers. The consideration is that if there are three correct students' answers, it means that they understood the concept and given a score of 2, the right answers to 2 questions are stated that they quite understood the idea and given a score of 1. In contrast, if there are no correct answers or only 1 question that has the right answer, then it is stated that they did not understand the concept so that given a score of 0. Those who do not understand the idea here is interpreted as having learning difficulties. There is an almost equal difference between the isomorphic and open-ended where the isomorphic average is 0.48, and the openended is 0.44.
We took a sample of each child to explore isomorphic and conventional assessments of open-ended, and it was found that there were inconsistent student scores on each test score. For example, the data in TABLE 5 follows. Inconsistent scores in the table above show that conventional assessments differ significantly compared to isomorphic and open-ended judgments. Using multiple-choice questions will not read students' minds on other answers (Attali et al., 2016;Mitnik et al., 2009). Therefore, it needs to be interpreted that the use of learning difficulties using isomorphic evaluation approaches the open-ended assessment. Assessment with multiple choice differs considerably, so that it would be better if used isomorphic review in measuring student learning difficulties on kinematics graphs.
Overall, student results are not right where the maximum score obtained is still very low. Judging from any assessment, the results show that students have difficulty in solving kinematics graph problems. Graphs are the primary key to understanding physics. Drawing and interpreting graphs will increase students' understanding of physics (Mitnik et al., 2009). The graph in kinematics is a physics representation that requires thinking highly to understand it (Bektasli & White 2012). Other research e-Jurnal: http://doi.org/10.21009/1 also shows that students have difficulty learning to interpret graphs (Bednárová et al., 2012). The problem in interpreting these displacement, speed, and acceleration graphs are not only demonstrated by students but also by people of different ages (Mitnik et al., 2009).
Students' most difficult item is in determining the acceleration where the percentage results obtained from each item is at least 23%. The most difficult items for students to answer are the second item in the form of questions. In this item (FIGURE 2.), only 3 out of 29 students were able to answer correctly. After exploring, it turns out that these three subjects are not consistent in the assessment, as explained above. Therefore we conclude that the material that many students have difficulty in is determining the acceleration through graphs.
The obtained student results indicate that the assessment score is not much different between isomorphic and open-ended. Isomorphic and conventional tests, when linked to open-ended tests, were not significantly related. However, the direction of the correlation closest to the open-ended test is isomorphic compared to traditional. Analysis of open-ended questions can be a better type of assessment than multiple choices, although sometimes it is difficult to correct it. The open-ended research produces meaningful conclusions to provide good feedback (Sychev et al. 2020).
The correlation between open-ended and isomorphic analysis indicates that isomorphic tests are more effective than multiple-choice tests in measuring the difficulty of learning kinematics charts.
The weakness of this study is the limited number of participants. It is recommended to research with larger participants so that the results obtained are significant.
The previous expectation of the researchers was that the isomorphic test was significantly related to the open-ended test, but after being analyzed statistically, it turns out that isomorphic is not significantly related to open-ended. Therefore the number of subjects must be considered as well as possible.

CONCLUSION
Based on the study results, the researchers concluded that there were still many students who had difficulty learning kinematics charts. This can be seen from the overall results, which are still low. When comparing conventional and isomorphic assessments of open-ended, it was found that the closest relationship with open-ended is isomorphic. Therefore it can be said that isomorphic review is more effectively used than conventional assessment. Isomorphic becomes a new assessment recommendation for measuring student learning difficulties. The teacher is expected to provide a more appropriate explanation to form the initial knowledge of students about the graph (Eshach, 2014). Future studies are expected to lead to the development of isomorphic diagnostic tests on every physical material.