Effect Size of General English Modules Development through MPI + R Model on the Lecturer Professionalism

Abstract


INTRODUCTION
Lecturer professionalism is a crucial issue in the education system (Liu & Zhang, 2024).Hattie (2008) in Roslina (2023) conducted more than 800 studies on educational topics and found that the educator is the main element in determining learning success.Thus, if there are problems in a lesson, the teacher should be the one to carry out a self-evaluation first (Weilage & Stumpfegger, 2022) and (Roslina, Hanna, Lio, & Nasrul, 2023a).
Referring to the previous explanation, as the educators is the main element of the learning process, the solution for the GE courses' problems should be focused on the lecturers' problems.Because increasing lecturers' professionalism can solve lecturers' teaching problems (Bao, Chen, & Jiang, 2024) as well as (Dijk, Tartwijk, Schaaf, & Kluijtmans, 2020).From several studies on GE, lecturers' main problems faced are the using methods, strategies, techniques, or teaching materials.Then, lecturers should use authentic material in language teaching (Gilmore & Abingdon, 2019), but many lecturers also need more competence in authentic material.They prefer to use downloaded materials or textbooks for convenience reason The existence of teaching materials can shape the lecturer's pedagogical framework (Jiang, 2022).These conditions' consequences impact teaching and lead to students' final competencies, as examples of research results by Huang and Tsai (2024).Students with good basic English skills are likely to experience increased competence, while students with low skills are unlikely to experience increased competence.One of the main reasons is that lecturers provide material only based on their knowledge and experience (Vereijken & Rijst, 2022) rather than based on need analysis.Consequently, the material used is sometimes acceptable for limited students only.
Starting from the initial research motivation, namely increasing the lecturer professionalism and their teaching material problems, this research encourages to increasing the lecturer professionalism through instructional design (ID) namely General English Module Development.It is carried out because sometimes the lecturer has understood their weaknesses, but in the learning process, they still need a guidance.Meanwhile, lecturers have sought several pedagogic stages through GE module development by the MPI + R ID model (Roslina, Hanna, & Lio, 2023).These stages can serve as guidance for improving the professionalism of lecturers.
Researchers emphasize that the ID positively affects lecturer professionalism.It was just that currently, previous research was only concerned with discussing the effectiveness of ID products in improving student performance, like Nagelli, Mohammed, Nayak, and George (2023) and Kuhlmann et al. (2023).Even though several studies have linked it to teacher competences (Tasir & Pin, 2012), it still focused on product effectiveness.There had been no studies that specifically investigated the effectiveness of the ID process on teacher performance.
So it seems essential for carrying a research that may report the effectiveness of ID, in this case, not only from the perspective of the product effectiveness but also from the perspective of the ID process effectiveness.Moreover, a review from Nassaji (2021) suggests that in the world of education, especially in language education, there is a need to grow a new research methods.He expressly implied the importance of quantitative education research, which is not only focuses on exploring how significant the effect is (significant based the significance effect level) but as well as reports the measurement of effect sizes (which refers to the magnitude of an effect or relationship) when presenting the results of significance tests.So, that the effect size research can provide an evidence for the strength of the emerging themes.
The state of the art of this research can be explained that this research is the follow-up to the author's previous publication (Roslina, Hanna, Lio, & Nasrul, 2023b), which showed limited only that the GE module as a product of ID through MPI + R had significant effectiveness for students.Meanwhile, this research explores further the effectiveness of the process of GE module development as an ID model MPI +R in increasing lecturer professionalism.So, the novelty of this research is not limited only looking at the effectiveness of the MPI + R product but also showing the effect size of the MPI + R process on increasing lecturer professionalism.
In terms of increasing lecturer professionalism, the state of the art from this research can be compared with two previous studies.They are Zulfikar, Emawati, Dahliana, Akmal, and Hidayat (2022) and Prasetyo, Ilham, and Asvio (2022).Both of believed lecturer professionalism can be done through training, seminars or other formal education.These activities are, of course limited to certain participants.Meanwhile, this which is efficient and can be carried out by anyone.So, this research shows the effectiveness of ID as an alternative activity to increase lecturer professionalism.
This research aims explicitly to answer a question "How is the effect size of General English (GE) Module through Instructional Design (ID) Model Instruksional Desain (MPI) + Revalidation (R) on lecturer professionalism?".This was done as an initialle research to propose MPI + R as an alternative strategy for lecturer professionalism development.It is hoped that it can provide a basis for further experimental research.Which can carry out a significance tests and/or statistical power tests on the effectiveness of ID model MPI + R in increasing lecturer professionalism.So, in the end, an academic conclusion can be drawn not only based effect size statistical tests parameters but also based on significance and statistical power test parameters.

METHODS
In general, this research was carried out in a quantitative approach.Since this research aims to measure the effect size of the ID Model Model Pengembangan Instruksional (MPI) + Revalidation (R) in increasing lecturer professionalism.As Aydin and Tanious (2022) stated that most effect size research is carried out in the form of a single case experimental design, this research was carried out using the one group pretest-posttest quasi experimental design.This design was chosen with the consideration that one way to minimize problems related to having no control or comparison group is to measure the same dependent variable in one group before and after a treatment (Anonimous, 2019).
By the principle of one group pretest-posttest design (Sugiyono, 2019) this research uses three main procedures: (1) The pretest assessment was carried out on September 24 2021 where the head of the study program and two senior lecturers assessed the lecturers' performance when teaching General English without ID; (2).The treatment stage is through the MPI + R model ID intervention from December 2021 to April 2022.The MPI + R model combines the nine stages of MPI (Suparman, 2018) with followed by Revalidation stages (Roslina, Hanna, et al., 2023b); and (3) The post-test assessment done on June 6 2022 The illustration of this research procedure can be seen in Figure 1.Meanwhile, the intervention from MPI + R was carried out through nine MPI stages plus a revalidation stage.MPI was used because this model results from Suparman's efforts to combine several previous development models to improve the instructional development model (Suparman, 2018).Meanwhile, the addition of the revalidation stage is carried out with the consideration that the module has undergone several improvements (revision after expert validation, revision after small group testing, and revision after extensive group testing), so it was considered necessary to carry out the revalidation from experts in the field of English education and taking into account reviews from lecturer which were involved.
In addition, this research involved three lecturers from the English education study program who taught General English subjects.The three samples were chosen, considering they represented the heterogeneity of lecturers at USN Kolaka.In this case, one lecturer represents a certified lecturer; one lecturer still needs to be certified but has taught general English for more than ten years, and one junior lecturer still needs to be certified.
Data was collected by using a lecturer performance assessment sheet, which is an adaptation of the Danielson teaching performance assessment framework (FFT), considering that FFT has been used globally since 2013 (Mathews, Stark, Jones, Brownell, & Bell, 2020) and FFT has become a teacher assessment instrument which has been offered in the In-Service Professional Teacher Education (PPG) Pedagogy course module since 2019.This means that FFT is valid in Indonesia (Pujiriyanto, 2019).
However, the instrument was first validated by two experts before use.Based on expert input, revisions were made for improvements.The assessment sheet contains four elements of the assessment domain: the planning domain, the classroom environment domain, the learning management domain, and the selfreflection domain.
In addition, other data was collected through open interviews to obtain lecturers' perceptions of developing their professionalism through ID activities.Like the first instrument, the interview guideline is first reviewed and revised.In order to support the data findings based on performance assessments given by superiors, the interview grid contains questions regarding the sample's impressions of the four domains of FFT.
After testing normality using the Kolmogorov Smirnov test, it was found that the data was normally distributed.However, the significance value could not be concluded because only three samples existed.So, the significance test is carried out using a nonparametric test.By the type of data, namely ordinal data, the T-Test referential test is used.
Then, to measure the value of the influence, the effect size test was carried out using Cohen's D analysis (Cohen, Manion, & Morrison, 2007) in Roslina (2023).Cohen D was chosen because this statistical test is intended to measure the effect of implementing an Impact of experimental design on the effect size model.In this case, this test was carried out to measure the effect of MPI + R on the average of lecturer performance assessments by FFT.The formula used is: The data in this study was analyzed using an online calculator from the Good Calculator website.Then, the data obtained is described.
Meanwhile, measuring the effect size of module use on lecturer professionalism can be seen in Table 1 below.
Great effect

RESULTS & DISCUSSION
The results of the professionalism assessment of the three lecturers before using the module can be seen in Table 2  Table 2 above showed that the overall average performance of lecturers is 2.66, which means they are in a poor classification.Then, from the three lecturers who were assessed, it was seen that there was a difference in comparing the scores between the three, which could have been more striking.D1 represents permanent lecturers who are certified and have taught MKDU English for more than 10 years, D2 is not yet certified but has taught MKDU English for more than ten years and D3 is a lecturer who has not been certified and taught MKDU English for less than five years.
Referring to Danielson's (2013) teaching framework evaluation standards, the professionalism standards for the three lecturers are at the second level, namely the Basic level.In this case, to achieve the highest professional standards, the MKDU English lecturer needs to increase to two more levels, namely from the Basic to the Proficient and Distingueshed levels.
The assessment results of the three lecturers before using the module can be seen in Table 3 below.
Table 3. Table 3 above shows that the average performance of lecturers is 3.90, which means it is in the perfect classification.Referring to Danielson's (2013) teaching framework evaluation instrument, the professionalism standards of the three lecturers are at the highest level, namely the proficient level.
The hypothesis test for the effectiveness of MPR + I on lecturer professionalism was analyzed through inferential analysis by first carrying out a normality test using the Kolmogorov-Smirnov test with a significance level of 5%.Moreover, based on decision-making, the residual value is normally distributed if the significance value is <0.05.Data from the normality test results can be seen in Table 4 below.The table shows that the data is normally distributed, but the the significance value cannot be calculated because the number of cases or samples is only three.So, the inferential test for the effectiveness of MPI + R is carried out using a non-parametric inferential test.In this case, the inferential test used is the t-test paired two samples for means.Based on the value above, it can be concluded that the sig value, (2 tailed) < 0.05, then there is a significant difference in lecturer professionalism before and after implementing MPI + R. The calculation results can be seen in Table 5.
Tabel From the table above, the two-tail P(T<=t) value is 0.0026.This means the sign value (2-tailed) < 0.005.Thus, there is a significant difference in the professionalism of lecturers before and after implementing MPI + R.
Next, to measure the magnitude of the effect size of the use of the module on lecturer professionalism, a Cohen-d test was carried out.The statistical calculation formula from Cohen's d is as follows: D Cohen = (mean difference) / (standard deviation) d = 26 : 2,64 = 9,82 From the calculation above, the d value is 9.82.In this case, 9.2 > 1.30, the effect size of the influence of MPI + R is in the great effect category.Based on the results of the effect size analysis, it can be concluded that MPI + R can increase lecturer professionalism.
In this case, 10.32 > 1.30, so the effect size of the use of the module on the professionalism of MKDU English lecturers has a great effect.
Table 2 and Table 3 show that through MPI + R, there is an increase in the average value of lecturers' teaching performance in the four domains.Candias (2022) stated that ID is closely related to lifelong learning, where life-long learning implies continuity of learning for a person to become a professional in their field.
This statement is supported by the findings of Bajracharya (2019), who stated that ID is a systematic and reflective process that seeks to translate learning principles into planning instructional materials, activities, information sources or evaluation material.This statement implies that in ID process, there is planning stages for learning activities and as well as reflection stages.
Other research that aligns with this research is Thohir, Sukarelawan, Jumadi, Warsono, and Citrasukmawati (2021).Their research explored the effects of instructional design development on teaching competency.They found that instructional design can improve teacher competence in student analysis, curriculum organization, instructional strategy selection, technological knowledge, evaluation design, and content knowledge.Meanwhile, this research found that MPI + R as an ID model has increase lecturers teaching professionalism.So, it can be said that this research confirms ID has a positive effect on lecturers' pedagogical competence.
The increasing of performance in these four domains occurred due to the ID stage of the MPI + R model covering the overall implementation of lecturer teaching performance indicators.An overview of the rationalization of the MPI + R stages with indicators from the four domains of teaching performance and their output can be seen in Table 6.

Reflection on learning Final Post-Revision Expert Validation
The initial design of the module

Eligibility assessment from experts Module Improvements
For the planning domain, there has been a change in the positive behaviour of lecturers in planning, namely from the Basic level to the Proficient level.This happens because, in the MPI + R process, there are stages of formulating the General Instructional Objectives, mapping students' competencies and subcompetencies, formulating Specific Instructional Objectives, topics and teaching materials, application in the classroom, and assessing the suitability of the material.Through this involvement, the lecturers have prepared well enough and received confirmation of reinforcement from each other.
This can be seen from the recognition of the lecturers involved.Two lecturer statements regarding the development of the GE model are: "It makes it easier to plan various ways, for example, in achieving learning objectives, designing learning activities and assessment systems."Also, other lecturers admitted that" I am facilitated in the whole teaching-learning process."The two statements above showed that lecturers recognize the MPI + R ID model can improve their performance in the planning domain.
There also an increase in classroom environmental management from the Basic level to the Proficient level.This finding is in line with Hoadley and Campos (2022), who stated that as a bridge, ID bridges the need for material development and human concerns and their relationship to the learning environment.This statement implies that an instructional design developer not only tries to solve the problems of the study topic but also serves the environmental conditions of their clients.Indirectly, this statement showed that a developer must organize theirs learning environment.This statement in line with the interview result, the lecturer said: "By being involved in this ID, I, as a permanent lecturer, can provide a conducive learning environment where students are free to express themselves according to the learning material.I also challenged the students to make conversation videos based on the material."From this statement, the MPI + R ID model can increase lecturer teaching creativity.So the atmosphere in the class becomes more lively.
This statement is supported by the findings of Weng and Chiu (2023) that developers are creative in creating learning environments, instructional materials, and learning experiences in carrying out instructional design.In the classroom environment creation stage, student assignments in pairs support increasing lecturer professionalism.This makes it possible to create positive and supportive relationships between fellow students.This positive relationship can be seen from the lecturer's attentive attitude towards students in explaining instructions, which sometimes must be repeated in Indonesian.Mutual respect between students was seen during the implementation of language games.Students allow each other to ask questions to find answers to their respective assignments.
Lecturers can also maintain a learning culture throughout the lesson because the activities in the module are more varied in terms of material or focus on just one language skill.Learning activities include activities to improve language skills and mastery of grammar.Each activity requires students to play an active role from the beginning to the end of learning.
For classroom management procedures, the construction activities facilitate the increase of lecturer professionalism.The learning activities are organized using the same pattern.Then, the learning steps are detailed and described in the lecturer's manual.It is easier for lecturers to organize students for doing assignments.It also allows lecturers to create an effective class routine for each meeting.Not much different, Trif-Boia (2022) stated that ID specifically supports novice teachers' performance in getting an overview of the teaching they will do.
Meanwhile, for experienced teachers, ID supports teachers' abilities to update teaching materials and integrate technology into their learning.These findings imply two keywords: learning activities and integrating all learning resources in the classroom.This means that through ID, teachers can easily manage learning activities.
This research shows that each assignment and exercise activity in the module also allows lecturers to manage student behaviour.In this case, each activity is accompanied by implementation instructions so the lecturer can easily direct students to complete the assignment.Apart from that, these instructions can guide lecturers in measuring whether student behaviour is correct in the learning process.
Furthermore, there was an increase in the learning management domain from the Basic level to the Distinguished level.After all, lecturers carry out class management through ID (Rayanto & Supriyo, 2021).As in this research, since the beginning of the process, lecturers and students have been actively involved in two-way communication through questions and answers or discussions.This creates a learning environment that is friendly and respectful.
The design of exercises and assignments also allows lecturers to involve all students using active speaking, for example, practising vocabulary pronunciation or language expression together.The various types of exercises and assignments allow lecturers to assess students fairly.Several lecturers have opinions regarding the influence of the MPI + R ID model on learning management performance.One of the lecturers answered, "Has the development of the GE module changed your teaching?"L3 answered: "A self-development process so that both lecturers and students can play an active role."The MPI + R ID model can increase classroom activity from these comments.
Meanwhile, for the question, "What aspects of module development contribute to increasing your professionalism?".L2 answered: "Almost all aspects significantly contribute to developing my professionalism as a Gelecturer, but what contributes most is the existence of various exercises and language games in each unit, which can increase my creativity as a lecturer."From these answers, formulation practice assignments and language games contribute positively to learning management.
Meanwhile, there was an increase in this condition in the reflection domain, meaning that there has been an increase in lecturer professionalism from the Basic level to the Proficient level.This finding is in line with the findings of Sutherland and Markauskaite (2012) in Radović, Hummel, and Vermeulen (2021) that ID encourages development teams to carry out experimental learning.Meanwhile, the most reported professional benefits of experiential learning are placed in two subgroups: Better understanding of the profession and practice and Becoming more thoughtful, reflective and critical.
The improvement in learning reflection performance occurred because, through the use of the GE module, the teaching lecturers exchanged ideas and strategies during preparation while indirectly reflecting on their strengths and weaknesses.It indirectly encourages providing opportunities for teaching lecturers to grow and develop professionally.This is in line with the findings that through the development of ID, educators can contribute each other Wang et al. (2022); and share mutual knowledge Yang, Pei-Yu Cheng, and Huang (2021).As well as growing their trustfulness in teaching (Kolleck, Schuster, Hartmann, & Gräsel, 2021).
Then, for the learning reflection domain, several positive statements were obtained from the lecturer.For example, the answer to the question, "Did the development of the module bring changes to your way of teaching?".L3 answered:"It is beneficial in the self-development process so that both lecturers and students can play an active role."The statement above illustrates that the existence of the MPI + R ID model can support the progress of lecturers' selfdevelopment.
Next to the question, "Referring to the development of the GE module, what factors are the most important in solving your challenging problems in increasing your professionalism as a lecturer?".L2 answered: "The most important thing is classroom management."Meanwhile, L3 answered, "Internal factors, in this case, are the classroom environment."Considering that most students do not have the basics of English, lecturers must be smart in managing classes and making breakthroughs that can attract students' attention in learning English."From L3's statement, MPI + R can improve the performance of managing the classroom environment.Overall, the comments from the three lecturers existence significantly influenced their professionalism as GE lecturers.
By looking at the contribution of MPI + R to increasing the professionalism of lecturers, the practical implication that can be offered from the research is that each lecturer should be able to develop their instructional materials, especially using MPI + R, moreover, by paying attention to the principles of the Merdeka curriculum which requires the use of teaching materials with attention student characteristics (Damiati, Junaedi, & Asbari, 2024) and (Ritonga, Harahap, & Adawiyah, 2023).So, the lecturer likes it or not, they are obliged to develop instructional materials.Meanwhile, in MPI + R there is an instructional analysis stage which aims to match student characteristics with the provided material or activities level.
Another practical implication offered is that increasing professionalism can basically be done independently without having to wait for opportunities to participate in training or other formal education.Apart from not being timebound, developing instructional materials is also more efficient in terms of financing.
Meanwhile, the implications of this research contribute to ID theory in terms of improving the stages of modern instructional design.Where in general the ID stage ends with a final revision without any revalidation.Meanwhile, this research offers additional stages of revision as a refinement so that lecturers as administrators can fulfil the entire domain of teaching performance by going through the process of each stage of MPI +R.Where in revalidation lecturers are involved in assessing the suitability of ID products through reflection on their learning.
The limitations of this research can be seen from the research design used.In this case, the researcher used a quasi experimental design, a one-shot case study type.Meanwhile, for more clearly comparison between the professionalism of lecturers with ID, an interactive group comparison should be carried out.
Then, based on data processing techniques, this research is limited to descriptive data processing.Meanwhile, to further explore the influence of ID on increasing lecturer professionalism, associative testing or research on the influence between these two variables can be carried out.

CONCLUSION
This research shows that apart from a significant influence between GE module development through MPI + R, the magnitude of an effect or relationship between the two is also measured in the great effect category.In other words, this research shows that ID educators can fulfill all the teaching indicators required to fulfill their professionalism through activities at each stage.The MPI + R ID model can be proposed as a scientific ID model that will not be lost over time.This research is limited to studying the effect size of ID on lecturer professionalism.This research examines lecturers' teaching performance in detail from four domains, namely the domain of planning, managing the classroom environment, managing the learning process, and reflection.However, the variable that is influenced is lecturer performance, which is discussed in general, namely the average lecturer performance.So, further research is needed to test the influence between variables.Further research is also needed to examine the effectiveness of ID on the lecturer professionalism by a larger scale.

Table 2 .
below.Lecturer Assessment Results Pre MPI + R Table of Lecturer Assessment Results Post MPI + R 5. t-Test: Paired Two Sample for Means