Intelligent solution for automatic online exam monitoring

ABSTRACT


INTRODUCTION
Generally significant learning outcomes can be categorized into three learning domains: cognitive, affective, and psychomotor [1]. All are important to student learning. The cognitive domain involves the recognition of knowledge and the development of intellectual skills. It is composed of six levels classified from the simplest to the most complex: knowledge, comprehension, application, analysis, synthesis and evaluation. The highest level is evaluation. It consists of examining learning outcomes and can be used for both measurement and improvement purposes. On the one hand, assessment measures the performance of students and whether they have achieved the intended learning outcomes. On the other hand, it is about monitoring students' progress through the learning process and providing guidance and feedback to help them move forward. However, the exam has a significant influence on students' learning behaviors. Since most students care about their grades, they cheat for different psychological or social reasons [2]. Such as fear of failure, pressure from parents, lack of time, low probability of being caught, feeling incompetent, desire to get a better grade. Furthermore, fraud is common in all countries regardless of their level of development. It represents a problem of educational inefficiency in all levels from primary to higher education [3]. In this sense, a recent study [4] proposes an academic dishonesty mitigation plan that encompasses strategies from prevention and detection approaches for effective security and integrity of online assessments. Generally, students can be assessed in many ways depending on the type of exam. Thus, exams can be divided into two types: the traditional exam (paper-based) and the online exam. Nevertheless, online tests raise additional concerns [5]. First, they take place on online learning platforms without students and instructors being present in the same location [6]. In addition, in an online assessment students can use computer technology to cheat. This is called  [7]. Therefore, strong and continuous security is needed to eliminate cheating [8]. In addition, it is difficult for the proctor to keep an eye on all students at once. Thus, this type of assessment presents a great challenge to the teacher [9]. On the one hand, the most difficult aspect is to verify the authenticity of the participants. On the other hand, how can we prove that the students in an online exam are who they say they are? In addition, how to detect, limit and prevent fraud in an online exam?
In this study, we address these issues by proposing a continuous authentication system that can verify the identity of candidates in real time during an online exam. Our strategy is based on machine learning algorithms using closed-circuit television (CCTV) cameras in the assessment room in addition to cameras installed on the candidates' personal computers (PCs). Throughout the online assessment process, our intelligent rule-based inference system continuously detects any inappropriate behavior. After this introduction, the rest of the paper is structured as follows: section 2 presents the literature review. Section 3 describes the proposed approach. The results and implementation are presented in section 4. Finally, the manuscript is concluded in section 5.

LITERATURE REVIEW
In the age of dematerialization, continuous authentication is applied in many fields. In relation to the academic world, during an online exam, educational institutions use continuous authentication systems to continuously verify the identity of students. In this sense, several academic researches [10] have been conducted in which the authors propose models using different identification methods and techniques, described below.
Many research studies have used a variety of methods based on biological characteristics [11]. For example, in their work, Shdaifat et al. [12] proposed a model using the iris biometric recognition technique in mobile examinations. Similarly, in [13] authors used fingerprint reader and eye tribe tracker to propose a system for monitoring online reviews. In their study [14], authors proposed a method to improve the robustness to pose and lighting variations in facial recognition. Hu et al. [15] proposed a new model to monitor students' abnormal behavior in an online exam, which determines the relationship between the candidate's head and mouth through a webcam.
In other studies, multiple biometric data are combined, such as facial recognition with fingerprint [16] or fingerprints and mouse patterns [17]. In [18] it was proposed a system combining different biometric features based on the type of interaction and the type of device used at a specific time. Indeed, the system captures raw data on facial features, voice, touch behavior, mouse dynamics and typing patterns during the use of the e-learning platform. Ryu et al. [19] proposed a continuous multi-biometric authentication system for student identification in an online exam using two modalities, facial recognition and keyboarding.
In addition, the study [20] from Michigan State University (MSU) in the United States proposed an automated monitoring solution for cheating prevention in online exams based on several biometric features with real-time tracking. About user behaviors, Morales and Fierrez [21] proposed a continuous authentication system based on keystroke dynamics. Other authors [13] have measured students' emotions during assessment and developed a multimodal emotion model for online exams. Another study suggested by Ullah et al. [22] used a series of personal and academic questions as challenge questions. They developed a dynamic model of student authentication in online exams based on the profile (PBAF). Kossingou et al. [23] proposed a platform to organize online exams while minimizing the possibility of cheating by students of the Central School of Free Software and Telecommunications of Dakar using various tools.
Other authors have developed monitoring solutions using machine learning and artificial intelligence techniques [24]. Radwan et al. [25] presented an intelligent approach using deep learning to efficiently detect suspicious behavior in real-time examination. Similarly, Malhotra and Chhabra [26] suggested a model for exam invigilation using deep learning and computer vision. Cheating detection was done based on the students' neck and head movement. In addition, the study implemented by Garg et al. [27] proposed a system to track students' faces in the exam based on deep learning techniques. Tiong and Lee [28] addressed the concerns of online cheating and proposes mechanisms for monitoring and reducing dishonest academic behavior using AI technology. Ashwinkumar et al. [29] developed an algorithm to facilitate online exam monitoring using deep learning. Duhaim et al. [30] suggested a model to reduce fraud in online assessments during the coronavirus disease 2019 (COVID-19) pandemic by extracting a set of reliable features from the Moodle platform while using data mining techniques.

METHOD
This section describes our continuous authentication model for verifying the identity of candidates in an online exam in real time. We start with a presenting the security policy. Then we detail the architecture of the proposed system. Finally, we list the monitoring rules and the risk classification.

Security policy
Our security policy is based on strong three-factor authentication. First, the student needs to authenticate himself with his multiservice smart card [31] and insert the personal identification number (PIN) code (first authentication phase) to request a session. Since the identification of a person from facial images is essential for security applications [32], the second phase is identity verification using facial recognition. After that, continuous identity verification throughout the examination is ensured. Following the same principle of functional process safety for risk reduction, based on the "separate monitoring from control" approach, our proposed model consists of two physically separate systems [2]. An online exam management system (EMS), operative part, and an automatic monitoring system (AMS) command part. The online exam server starts an exam session only after receiving an agreement from the automatic monitoring server. In the same way, in case of fraud, the monitoring system gives a session closure order to the examination platform. This separation has the advantage of ensuring availability, reliability and security. Thus, we have defined a set of techniques and rules to eliminate fraud cases. Figure 1 illustrates the different components of this policy.

Proposed system architecture
When conducting an online exam, educational institutions face the following challenges: i) verify student identity online at all times during the assessment period, ii) ensure that students pass exams without cheating, iii) report cases of fraud, and iv) evaluate students' skills appropriately. To address these challenges, two questions naturally emerge. First, on what basis does an automatic monitoring system define fraud. Then, how can fraud be prevented. To answer these questions, the main objective of our model is to make a decision about student action without disrupting their concentration based on a set of rules. These rules differ according to three main modules of the proposed system as shown in Figure 2. The three modules include video input processing, object detection and active window capture. The latter allows for automatic detection of all processes running in the system during the exam. With the video input, it is possible to monitor the student's face throughout the exam. While the student is taking the exam, it is possible that his or her head moves frequently or that he keeps his head turned for some time. All these elements are recorded. Thus, object detection is very useful in case the candidate tries to use the phone or search for text in a book, document, notepad.

Video analysis
Video input processing is the first module to be analyzed. Indeed, a webcam is used to capture the student's facial movements during the exam. These videos are the inputs to the automatic monitoring system. Thus, face detection as shown in Figure 3 allows to discover: several detected faces Figure 4 detected face looking forward; detected face looking to the right; detected face looking to the left or no detected face. After detection, face tracking ensures that the student is physically present during the evaluation. In addition, tracking the student's movements is very important. As in traditional surveillance, abnormal head direction can be an indicator of possible fraud. For example, if the student looks around a few times or when his eyes are not on the screen for an extended period of time.

System usage analysis
With the help of system usage analysis, window changes in the computer used by the student are detected. In case the candidate opens a web browser to search for answers or connects a device to the system or uses another software, this usage capture is detected. In fact, during an evaluation the output of the active window capture is a log with the start and end time of the processes [33]. For example, when the student tries to connect devices to the PC via the universal serial bus (USB) port, it is detected by the capture process.

Object detection
The third module is object detection. The student may make many mistakes that can be captured by object detection. For example, the presence of a cell phone next to the student can be a clue to potential fraud. This is also checked for text in an exam with unauthorized materials; reading from a text is a form of cheating. Lastly, these outputs (video input, system usage and object detection) form the inputs to the rulebased intelligent inference system. Then, the system classifies these rules to detect the possibilities of errors produced. In the following section, we present the different monitoring rules and their classification.

Monitoring rules: risk classification
The proposed automatic monitoring system consists of evaluating a set of important rules to ensure security and eliminate fraud. By analyzing the students' actions and movements, a classification module determines the risk level of the detected fault. Indeed, we have defined three levels of risk classification: low, medium and high. Each rule is assigned to a risk level Figure 5. Consequently, the inference system makes decisions following the result of the risk classification module. Obviously, when the system detects an attempt to cheat (non-compliance with the rules) an alert is generated and the student receives a warning indicating the fault committed.

Low risk fault
We define a low-risk fault when the authenticated student is always present in the camera's field of view. In addition, his movements do not pose a security risk to the electronic evaluation system. After three such alerts, the fault is reclassified as medium risk.

Medium risk fault
An error is classified as medium risk, if the user tries to cheat by opening another system window for example (he will be blocked by the system). Also, if the system detects the same login from two different places. If the student repeats the same error three times, the error is reclassified as high risk.

High risk fault
Detecting the faces of students being examined throughout the assessment is very important. It ensures the face tracking process. Thus, a fault is defined as high risk when the student's presence becomes suspicious. For example, when the student goes out of the camera's field of view or when the system detects several faces in the same frame. After three high-risk faults, the session will be automatically closed.

RESULTS AND DISCUSSION
In this section we first introduce the techniques used. Then we discuss the results obtained. Finally, we present the possible performance tests of the proposed model.

Techniques used
To implement our approach, we used Python as the programming language due to its cross-platform aspect and rich library in the artificial intelligence domain. We used the relational database management system SQLlite as the database server to store the data. For object detection we used the yolo v3 library. In addition, we used the deep face library for the emotion algorithms. Finally, we used the open source computer vision library (OpenCV) library [34] library and the face recognition library in the process of detecting and recognizing students faces for continuous authentication. Thus, to meet the challenges of biometric face authentication, we set up two new techniques to handle face tracking. The first one is based on a motion detection algorithm that calculates the difference between two consecutive video frames. Based on a previously defined threshold, it determines the movements in the video stream. The second one is based on the use of OpenCV face detection algorithms with face recognition association. After the initial detection, a pattern matching is performed to detect the face pattern in the new video frames. To control any fraud attempt we have developed several features. Certainly, several e-learning applications exist [35], [36] but to implement our automatic monitoring system, we used the Moodle 3.9 learning management system (LMS). The purpose of such a choice is twofold; First, Moodle is the e-learning platform used by the Hassan II University of Casablanca via the digital workspace. In addition, as of this version, Moodle is fully compatible with the safe exam browser (SEB) which locks the Moodle platform and ensures that no student accesses an other program. As a result, this improves the security of online exams.

Authentication phase
Authentication is the first step in the process by which users can access university services [37]. First, the student accesses the examination room with his or her multi-service smart card [38]. Then, he/she must authenticate with the same card to open a new session by inserting it in the card reader connected to the computer and typing his/her PIN code. After that, our program verifies the identity of the student using facial recognition Figure 6 and displays a summary of the personal data. Finally, by clicking on 'start' the examination platform is automatically launched and the student can start the evaluation.

Monitoring (back-end)
During the exam, the student only has access to the Moodle platform and specifically to the exam. However, our automatic monitoring system runs on the back-end. The fraud attempt counter is initially set to 0 Figure 7. Our program processes, calculates, and reports all fraud attempts Figure 8. For example, if the student uses a cell phone during the exam Figure 9. In addition, if the student reaches three warnings, the session will be closed automatically. In addition, the teacher or administrator can see the candidate's activity from the school's workstations via the local network. In this regard, we developed a script sending the live video images over the network socket. We used the python language and OpenCV for the image processing. Thus, this session is recorded from start to finish in case of need for a later review. A video will be recorded with the student's national code.

Real-time performance testing
The automatic surveillance system proposed in this research requires real-time processing, including face detection, facial recognition, decision making, alert reporting, live streaming, and end-to-end recording as illustrated in Figure 10. The specification of these modules appears in the processing and response time constraints. In this regard, it is necessary to verify that the operating system is capable of operating under realtime conditions. For this purpose, we used the real-time (RT) Linux system to run our proposed model. Finally, to test the performance of the operating system on which our solution runs in real time, we put the resources under stress in order to load the system, both the processor and the memory. Thanks to the cyclictest tool we have deduced that the system meets the requirements even when overloaded. Figure 11  with PREEMPT_RT patch). While Figure 12 shows the execution of the same program 'final.py' on the PREEMPT_RT patched Kernel 5.4.43-rt25. Figure 11. Test on the unmodified linux kernel Figure 12. Kernel 5.4.43-rt25 patched PREEMPT_RT In the first case, Figure 11 allows us to conclude that for an unpatched Linux system, the average 2-thread delay is distributed at 306-316 (us) deviation from the desired trigger. Thus, the right column represents the most important result, i.e., the worst-case latency where the maximum delay is distributed at 43831-44159 (us). In the second case Figure 12, 2 threads with the same priority running in the system with a PREEMPT_RT patched core, the average is very slightly degraded to 232-239 (us). However, the main observation is that the maximum deviation is distributed at 5062-5909 (us) which is significantly better than before. We can conclude for a two central processing unit (CPU) core system running one thread (SCHED_FIFO) per core at priority 80 and which is also under high load due to stress execution in a separate terminal: the maximum detected processing latency, in the real-time system is much lower compared to an unpatched RT system. Therefore, an rt-patched kernel has a more deterministic behavior with respect to latency jitter.

CONCLUSION
In this study, an automatic online and continuous monitoring system has been proposed. Our specific purpose is to detect, limit and prevent fraud during an online exam based on automatic face recognition technology using artificial intelligence. The proposed model is divided into several modules. First, to take an exam, the student must authenticate himself. The first one requires multi-service smart card authentication while the second one relies on facial recognition. After confirming the student's identity, the control module takes over to continuously authenticate the student throughout the online assessment process.
The monitoring module is used to detect incorrect behavior and attempted fraud based on a set of rules. It includes system usage analysis, CCTV and computer video streams. The last module provides a logging system, in which exam sessions can be viewed by administrators or teachers in real time. Thus, these sessions are recorded from end to end in case they need to be reviewed later. Next, we tested the performance of the operating system on which our solution runs in real time. Using the stress' tool preparing a stressed environment for cyclic test we concluded from the experimental results that the maximum latencies are much lower in a real-time kernel compared to non-RT Linux.
The proposed model will be implemented and enhanced using the recent evolutionary operators and hybrid recommender systems. This article opens perspectives for improvement and refinement of existing work such as the enhancement of the automated proctoring system to include remote online exams. Thus, students residing in different geographical areas and cities can take the exam from any location. Therefore, continuous automatic proctoring will eliminate the constraints of the institution's location in this case.