Factors affecting students’ continuance intention to use teaching performance assessment application from technology continuance theory

ABSTRACT


INTRODUCTION
Universities have intensively applied service assessment policies to improve their education quality. One aspect that has been frequently evaluated regularly is teaching quality assessment. The teaching performance assessment (TPA) is a strategy to find out teaching quality which is crucial to improve meaningful and sustainable learning in a highly competitive educational industry. The outcome of teaching performance evaluation is usually used by decision-makers of university stakeholders such as rectors, deans, and the quality assurance (QA) department for teaching staff development and accreditation purposes.
Mobile apps for education quality evaluation have been implemented in some universities [1], [2]. The apps are used to evaluate the quality of education, such as teaching performance. The teaching performance evaluation is an educational institution's activity to assess their teaching staff performance for better improvement [3]. In higher education institutions, the objective of TPA is to improve the quality of teaching and education as demanded by stakeholders. As such, lecturers are required to improve their teaching performance through external critics. In this regard, the critics come from their students who receive teaching material and activities. The evaluation of the teaching process can be carried out qualitatively [4], [5] or it can also involve quantitative measuring in the assessment process [6], [7]. The evaluation results of the teaching staff performance have long been a reference for advanced universities to take further actions toward teaching staff improvement across countries around the globe [8]. The acts might include providing training, peer teaching critics, and pursuing further education level.
In Indonesia, universities are imposed on practicing teaching performance evaluation when a university proposes accreditation based on the directorate for higher education regulation Number 12 of 2012. The results of the teaching quality assessment reflect stakeholders' satisfaction, such as students, parents, and graduate users, and it also becomes an indicator to determine the quality of teaching in the university. The indicator can be used for accreditation and university quality improvement. Indonesian government regulation number 14 of 2005 mandates all teaching staff to be accredited for professionalism before they are liable for certification allowances. Their professionals are usually measured by their capability in teaching.
In response to professional issues, universities implement various TPA mechanisms. For example, some universities use conventional TPA methods through paper-based surveys by distributing hard-copy of questionnaires to students Dandalt and Brutus [9], and Buchanan et al. [10] or using a mobile-based application [11]. However, instead of rampant implementation of mobile apps to evaluate teaching performance, more research needs to be conducted to determine factors influencing students' continuance intention to use mobile apps for teaching performance evaluation. Therefore, the study's objectives are to find out factors that affect students' motivation to continue using an android-based application TPA and how the significance of each factor influences their continuance motivation in using the system. The results of this research may shed light on academia and practitioners regarding factors that affect the continuance intention to use the android-based application in TPA and use the finding to improve the mobile application evaluation systems.

LITERATURE REVIEW 2.1. Previous studies
Several researchers within various educational contexts have studied teaching staff performance. For example, some researchers have studied teaching performance evaluation within higher education institutions [12], [13] and middle school contexts [14]. Most research on teaching evaluation performance was conducted using paper-based questionnaires, which might hinder maximum evaluators' involvement. For example, a paper-based teaching performance evaluation conducted by Guolla [13], who distributed printed questionnaires to 184 undergraduate and postgraduate students, found that fewer students were willing to take part in the study. However, with limited participation, the study found the strengths and weaknesses of teachers' teaching performance. The study has a weakness because most instruments were not returned, which proved that the manual teacher performance assessment restricted broader students' participation.
Teaching performance evaluation using information technology has intensively been applied since various cutting edges mobile technologies emerged in the education sector. Early studies in teaching performance evaluation used information technology-based, and researchers used desktop-based and web-based information systems. Hussein [15], for example, applied a teaching management information system called 'JUSUR' to assess educators' teaching quality at King Saud University. The study recruited all students as participants, and they were asked to complete a 5-point Likert scale questionnaire on desktop-based applications in the campus computer networks. The study results provide information regarding the quality of teaching within all the university departments [15]. The weakness of the system is that students must use campus computers connected to campus networks. In addition, many students need help accessing the computer network.
Meanwhile, another research conducted by Sher [16] and Felton et al. [17] uses a web-based system to evaluate teaching quality based on criteria of quality, easiness, and sexiness. Students from all faculties were selected purposively, and they were asked to access the university website. However, many students were reluctant to access websites made for the evaluation process. In addition, a slow internet connection caused the lack of access to the web-based valuation system. The problem with the internet connection reflects the weakness of a web-based teaching quality evaluation system.
A new approach to assessing teaching staff has been developed. Noguera et al. [18] developed 3D mobile application to evaluate the teaching quality delivered to physiotherapy students. The application helps practice assessment where personal computers were not possible to use. Syaifudin et al. [19] developed a web-based application to help students evaluate learning activities online. The application was accessed and smoothly utilized by the students. However, both studies did not conduct further research to understand how

Theoretical constructs and hypotheses
Previous research concentrated on the early phase of adoption of an android application, while limited studies have been conducted on long-period use of the information systems [20]. The theories to understand the early adoption stage have been adapted from social and psychological areas. However, such theories were unable to explain the continuance of information systems. This research used the technology continuance theory (TCT) adapted from Liao et al. [21] to understand students' motivation to continue using an android-based teaching assessment application. The TCT theory argues that users' continuance intention to use an application is because they perceive confirmation and benefits, and it is easy to use. However, the perceived confirmation, benefits, and ease to use are encouraged by factors such as the information system quality, relevancy, productivity, performance, and mobility [22]. Therefore, we adopted the theory concept, but some variables were developed and integrated into the theory to study the continuance intention usage of android-based TPA to fit our study context. The variables were identified and developed as the constructs of our study, as depicted in Table 1.

System productivity
The concept of "Productivity" refers to an application that can increase the effectiveness of users in carrying out their daily routine tasks [41]. Within the educational context, productivity has become the main concern in carrying out teaching tasks such as class management, lesson planning, teaching evaluation, student performance evaluation, and all other related teaching tasks [42]. Meanwhile, from students' perspectives, an application is considered highly productive when the application supports them in evaluating their teachers' performances effectively and efficiently. In other words, quality teaching assessment applications help students to handle their teachers' evaluations as efficiently as possible.
At the same time, management and instructors can also obtain their TPA results quickly [43]. By using such an application, students can evaluate their teachers' teaching performance as often as possible, every time required. As such, the willingness to use the application can be maintained whenever the teaching performance evaluation is required to produce detailed teaching performance reports [44]. An android-based teaching evaluation performance can help students to assess a number of their teacher performances in a relatively short time [45]. Therefore, when an android-based application can reduce the resources and time required for students to perform their teachers' teaching assessment, their intention to use the application is high [46]. Our hypothesis is as: H1: The productivity of android-based teaching performance evaluation (TPE) positively influences users' continuance intention usage

System performance (SP)
Information system performance is understood as the ability of the system to perform given tasks [47]. Excellent information system performance resulted in user satisfaction, defined as the "sum of one's feelings regarding an information system" [48]. The performance of an information system is determined by its efficiency, such as the system can improve the communication process, decision-making, and organizational operations. Information system performance is also causing users' confirmation of expectation, perceived usefulness, and perceived ease of use of the information systems, leading to users' continuance intention to use the information systems. Bhattacherjee [38] posits that information systems performance expectations become an important factor in users' continuance intention to use an information system. Users build positive behavior towards the information systems when the performance fulfill their expectation. The high performance of an information system indicates a high level of task-technology fit and satisfaction with the information system [49]. As such, we hypothesize as:

System relevancy
The information system literature views relevance as an essential criterion of satisfaction [50]. Several studies view relevance as a contextual and situational-based concept as users resolve their information needs. In this study, we argue that information system relevance is critical to students' intentions to use an information system [51], [52]. The dimension "relevant" explains the extent of the benefits of an application for teaching staff in all classes and disciplines. Teaching quality assessment applications must be able to support all teaching staff in getting information about their teaching abilities. When assessing the relevance of the application, evaluators (students) need to see the relevance of the tasks that must be completed by the lecturer in the classroom and the courses taught in his paper by preparing, giving, assessing, and analyzing the teaching material given to students [44]. Existing facilities in the application must also be relevant to aspects that will be assessed in lecturer learning activities. If aspects are not relevant to be assessed, it will cause the results that appear in the application cannot to be used for decision-making by their superiors. In other words, the more relevant an information system, the more likely users will use it [53]. The relevancy is often associated with the compatibility and usefulness of the information system as the most determinant of continuance intention to use it [54]. The hypothesis is that: H3: The relevancy of android-based teaching performance evaluation (TPE) positively influences users' continuance intention usage

System quality
The quality of an Information system is achieved as the result of an IS development and maintenance process [55]. The information system has quality features to support ease of use and maximum operation, and users can operate it conveniently. DeLone and McLean [25] argue that a high-quality information system is reliable to use, has excellent response time, is easy to use, useful, easy to access, and flexible. The information produced by the system is reliable and trusted to be used for decision-making and solving problems. Myers et al. [24] consider a high-quality information system as a system with a quick response time, and it also increases efficiency in tasks performed [35]. The systems can reasonably respond to users' needs when data is required to support decision-making.
A high-quality application can support users to interact smoothly and efficiently in completing their tasks [44]. Therefore, users who have a positive experience when using an application tend to use it maximum and will continue to use it. Furthermore, users also find that all application features support its functionality to increase their intention to use it. Moreover, the application's functionality includes some dimensions that support collaboration, communication, and the ability to save work in progress [44]. In other words, an application has good functionality when designed to align with task requirements [56]. Therefore, we hypothesize as: H4: The quality of android-based teaching performance evaluation (TPE) positively influences users' continuance intention usage

Perceived mobility
One essential feature that distinguishes android-based TPA from traditional teaching evaluation systems is mobility. Perceived information system mobility is understood as "the extent of user awareness of the mobility value of mobile services and systems" [57]. Users, who perceive the value of application mobility, appreciate the ubiquity of mobile teaching performance evaluation and have a strong intention to use the application. Previous studies show that perceived application mobility significantly impacts the perceived use of the application anytime and anywhere and enhances users' acceptance of mobile services within teaching assessment contexts [36].
In the android-based TPA literature, a mobile application tends to refer to the individuals' ability to make evaluations independent of time and place [58]. Students can be located in more than one place physically with current conditions. More clearly, students can evaluate their instructors' quality of teaching from home or when traveling. Students can also practice all teaching assessments through android-based applications [59], [60]. A mobile information system is considered productive when the system can reduce time in production, control products, improve responsiveness, change management, and improve product quality [61]. Therefore, we hypothesize as: H5: Mobility of android-based teaching performance assessment positively influences users' continuance intention usage

TPA continuance usage
Information system continuance intention usage is mostly influenced by users' expectations and perceived performance of the information systems [38]. The expectation of the information systems (IS) performance is related to factors such as the quality and functionality of the IS [59], [60], relevancy of the IS with tasks fulfillment [61], [62], information system productivity [24], [63], information system mobility [36], [64]. Those factors enhance users' satisfaction, resulting in the intention to use the systems for a long time.
The continuance intention to use a mobile application that is developed to assess teaching performance has also been found to be affected by the factors discussed above [65], [66]. Most users view a mobile application as a system with characteristics that reinforce users' intention to continue using the application [39]. When a mobile application user's perceived usefulness of a mobile application characteristics to the degree to which the user's expectations about the application are confirmed, the intention to continue using the application becomes a habit [21]. The habits, then, familiarize users tend to continue using the mobile application [67]. The concept posited that continuance intention was influenced by users' habits, satisfaction, and post-acceptance of the mobile application characteristics of ease and usefulness perceptions [40].

METHOD
This study used the quantitative method, and it was conducted at the State Islamic University of Datokarama Palu. Data was collected using an online structured questionnaire approach. There were 240 respondents recruited from four faculties, with 132 men and 108 women using a quota random sampling approach. Structured questionnaires were distributed to the respondents. The questionnaires were available online for one month. The questionnaires consisted of six variables with twenty-eight questions relating to their perception of the continuance use of an android-based TPA. The respondents' characteristics are depicted in Table 2. Our study conducted an online structured survey as an instrument for data gathering strategy. Perception measures in the form statements were applied for measuring each factor corresponding to five Likert scales ranging from strongly agree (5), agree (4), neutral (3), do not agree (2), and strongly do not agree (1). However, before we distributed the questionnaire, we conducted a pre-tested with relevant research experts and prospective respondents and then followed by a pilot test with 20 university students. The pilot test results showed that the Cronbach-Alpha value for all statements in the survey was higher than 0.7, which means the survey was reliable. The data were analyzed using analysis of moment structure (AMOS) version 22. First, we conducted an explanatory factor analysis (EFA) with main components extraction to explore all constructs used in this study. The constructs were five variables, X, and one variable, Y. Then, we conducted confirmatory factors analysis (CFA) to measure factor loading, reliability, convergent, and discriminate validity. The flow of the TPA application process can be seen in Figure 1.
The android-based TPA application was built by using PHP hypertext pre-processor. In addition, we used MySQL database and sublime text three as well as the android studio. The TPA android-based system was provided in the Google App store. We also adopted the framework from Alkhafaji and Sriram [68] in building the system data flow process. Our android application for TPA provided the primary system and sub-system for students and the university quality assurance unit (LPM). At the same time, the administrator of the system and the university management are sub-system users. The students used the application by entering their lecturers' teaching assessment data. Then the quality assurance unit accessed the android application database to analyze the data and then made reports on the lecturers' teaching performance quality at the university. The android application administrator managed users' IDs, and passwords by maintaining the application via the internet. The university management only received the lecturers TPA reports for their future decision-making in teaching staff management.

Variables influence continuance intention to use TPA
The constructs were used to determine the factors that influenced the students' continuance intention usage of the TPA. Table 3 shows the results of the measurement. The results below show that all items fit their variables. All the loading items from scores 0.585 to 0.875 shows higher than the threshold of 0.50. The coefficient of Cronbach's alpha for all factors variables spanned between 0.721 to 0.847, which means they are higher than the 0.7 value. Furthermore, the composite reliability values (CR) were higher than 0.8 (ranging from values 0.84 to 0.918), while the average extracted variances (AEV) were higher than the recommended 0.5 value, meaning that the relationship between all variables and the motivation to continue using of TPA has shown internal consistency reliability or the variables consistency with the scale [69]. Table 3 also indicates that the more factors' loading different among items within similar variables, the bigger the gap between the score of CR and Cronbach's Alpha. Our study also compared the square root of the average variance extracted from each variable and the correlation between variables in examining the discriminant validity [70]. The discriminant validity indicated whether or not the variables in the research model are significantly related among them.
The square roots of the average extracted variance of all variables were found to be larger than the relationship estimated with the other variables. In other words, the computation technique exceeds adequate  Table 4 shows the correlation matrix and discriminant validity.
Data for our research hypothesis testing is depicted in Table 5. The model testing results include the standardized regression coefficient, the critical ratio (t-value), and the probability (P-value). The model tested in this study shows the significant influence of all five variables on the continuance intention of the students to use the android-based TPA. As shown in Table 5, all variables were found significantly influence the student's continuance intention to use TPA with a determined significant level was 5% (p<0.05).  ItoCU2: I have got good knowledge and skills to continue the use of the TPA ItoCU3: I prefer to use TPA than other applications for teaching performance assessments ItoCU4: I will suggest my friends use the TPA application for teaching performance assessment ItoCU5: I will keep using the TPA as frequently as possible ItoCU6: I will enjoy using the TPA in the future ISSN: 2088-8708  Factors affecting students' continuance intention to use teaching performance … (Nurdin Nurdin)

5349
In our research model proposed in the theoretical construct section, we hypothesized that all factors, system productivity, system performance, system relevancy, system quality, and system mobility, positively influence the users' continuance intention to use the teaching performance evaluation application. Based on our statistical calculation results, the standardized regression coefficient indicated the significance of all factors to the users' continuance intention to the TPA application ranging from 0.228 to 0.366. The factor of system performance has the strongest influence on the users' continuance intention to use the TPE application. Meanwhile, the factor of system relevancy played the weakest influence on the users' continuance intention to use the system.

Discussion
Our study shows that our research model proposed earlier and hypothesis are proven significantly and able to describe students' continuance intention to use the android-based TPA. Tested constructs (factors): application productivity, performance, relevancy, quality, and mobility were all important factors that affect the student's continuance intention to use the TPA. The two modified constructs, system quality (β=0.228, P<0.04) and mobility (β=0.313, P<0.04), were important for all students' responses.
Our results support earlier studies. For example, variable SPr influenced users' continuance motivation to use the TPA. The finding indicated that users expect a mobile application to assess teaching performance should support them to assess more lectures in a certain time, and it can be performed in a short time [71]. Such expectations reflect teaching activities sometimes are performed more than twice a day in the Indonesian university context. This requires a teaching evaluation system that can support assessment productivity.
Our study also found that the performance of a mobile application has significantly influenced users' continuance intention to use it [72], [73]. Our findings indicated that users with high-performance expectancy tend to use the android-based teaching performance evaluation continually. Furthermore, users usually consider the performance of an information system to determine their satisfaction with using the information system within an organization's work practice because the information system can support their learning effectiveness [74]. In this study, better TPA performance has supported the teaching quality improvement to satisfy the students because the university lecturers got input from students' assessment process.
The variable SR also significantly influenced the students' continuance intention to use the TPA in which the application features support all students to evaluate all aspects of lectures' teaching performance. Relevance information systems features have been found to influence users' continuance intention to use the system [75]. The android-based TPA system's relevance with the need of the students to evaluate lecture skills in performing their teaching has enhanced the students' continuance intention to use the application in future teaching and learning activities.
The SQ of the TPA was found to be significantly affecting the students' continuance intention to use the TPA. Early related research also showed that the quality of an information system influenced users' intention to use it [76], [77]. In this study, the quality is referred to as easy to access, quick to download, easy to select the features, and easy to find a lecture name. Furthermore, the students did not experience difficulties in using the application every time they used it. As such, those indicators positively affected the students' continuance intention to use the android-based system, as found by Nikon and Economides [78] in their studies. Variable SM was found to have an important effect on the students' continuance intention to use the TPA. The finding suggested that an application that can be accessed and used everywhere can maintain users' continuance intention to use it [79], [80]. An application with higher mobility enhances users' continuance intentions toward mobile applications [81]. In this study, the students can access and use the TPA everywhere they want. They did not have to access the application in the classroom where a lecturer noticed them. As such, the flexibility to access and use the system can increase users' conformability in using the system, which might lead to continuance usage.

CONCLUSION
Our study contributes to both the theory and practice world. From a theoretical perspective, our study shed light on the understanding of the predictors of continuance intention to use android-based TPA, and this study might become a reference for both academia and practitioners in higher education institutions. For the researchers, the model used in the study can be tested within a broader context involving more populations from other universities. Meanwhile, for education institutions, our study provides important insights into supporting teaching quality assessment to maintain the high quality of teaching as a university's core business. The limitation of this study is that the students may assess a lecturer who is not teaching in their class due to the freedom to select on the database. Further research needs to limit each lecture based on a specific course to be assessed by a particular student's class.

BIOGRAPHIES OF AUTHORS
Nurdin Nurdin is a Professor in Information Systems at the Department of Economics and Business, Universitas Islam Negeri Datokarama Palu. He has been a faculty member since 1999. He earned his Master's degree in Information Management at the Faculty of Economic and Commerce, University of Western Australia (UWA), and a Ph.D. degree in Information and Communication Technology from Swinburne University of Technology, Australia. He has been a teaching staff at UIN Datokarama Palu since 1997. His research interests lie in the areas of e-government, information systems, social media, and knowledge management. He has published in a number of international journals and proceedings. He can be contacted at email: nurdin@uindatokarama.ac.id.

Sagaf S. Pettalongi
is a Professor in Education Management at the Faculty of Tarbiyah and Teacher Training Universitas Islam Negeri Datokarama Palu. He has been a faculty member since 1991. He obtained his Doctor degree in education management from the State University of Malang. His research interest lies in education management, education administration, education theory, higher education, and multiculturalism in education. He can be contacted at email: sagafspettalongi@uindatokarama.ac.id.

Muhammad Nur Ahsan
is a doctoral student at the Faculty of Social Science, University of Gadjah Mada Yogyakarta. He is also a lecturer at the Faculty of Philosophy and Islamic communication Universitas Islam Negeri Datokarama Palu. His research areas are social sciences, history, and philosophy. He can be contacted at: mnurahsan@uindatokarama.ac.id.

Vindy Febrianti
is a postgraduate student at the department of Islamic Education Universitas Islam Negeri Datokarama Palu. Her research interest includes education management, Islamic education, and education at the middle level. She can be contacted at email: vindy.febrianti@student.uindatokarama.ac.id.