NLP-based personal learning assistant for school education

ABSTRACT


INTRODUCTION
Computers plays an important part in our lives.Computers are now widely used for educational purposes like lesson tutoring, language testing, diagnosing errors and also for documents archival.This has led to many researchers think about designing an intelligent tutoring system (ITS).These intelligent tutoring systems imitate one-to-one tutoring to deliver the content in the learning material better than using a class room environment.Artificial intelligence (AI) plays a major role in understanding and in interpreting the human language in order to develop learning application.
AI has been adopted by many educational institutions in various forms from a web based online education system to a web based chatbots [1].It determines the student's level of competence and adapt the computer responses to student input.The intelligent tutoring system (ITS) uses NLP technique to process and access students text input [2].The intelligent tutoring system provides the students with an interactive learning environment.An ITS helps to identify the gaps in student knowledge and also helps to provide personalized instruction based on student knowledge level [3].It is very important for the system to provide timely feedback which would enhance the learning experience [4].NLP is about building an intelligent machine with computational technologies and linguistics.The availability of large amount of educationallyrelevant text have increased the use of NLP to resolve the challenges faced by students and teachers [5].A Int J Elec & Comp Eng ISSN: 2088-8708  NLP-based personal learning assistant for school education (Ann Neethu Mathew) 4523 question-answering (QA) system finds precise answer or find the precise portion of the text from a given collection of documents, when the user poses a query.QA system answers to the questions asked by the user in human language.The answer is retrieved from either a database or a collection of documents using NLP techniques [6].
Chatbots are an artificial intelligence (AI) conversational user interface that allow users to communicate with services using a messaging application.Eliza, Alice, Jabberwacky [7] are few early chatbots that could simulate the human interaction and was able to have conversation with users.The chatbots can be used in formal education where most frequent questions of the students can be answered by the assistant chatbots thus useful in e-learning classrooms [8].The bots bring a product or service into any messaging products such as Slack, Facebook via conversation and can simulate an interaction with a user in natural language through messaging application.Chatbots represent the evolution of question answering (QA) systems using NLP and it responds to users queries in natural language.When a chatbot is trained with lot of data, more satisfaction and precision is reached [9].
With this increased adoption of technology in education, we see a need to also ensure that quality of learning is not compromised on.Technology is best used as a supporting tool that improves the overall productivity and effectiveness of the teachers and provides a better way for education coverage to scale up and grow.In this paper, an attempt is made at exploring the feasibility of using AI and machine language (ML) in a teaching assistant context to create a near human interaction experience and also provide a means for teachers to get insights about each student's learning progress.An understanding of past work in this space was developed by doing a review of existing literature.A summary of the literature review is provided in section 2. Section 3 explains the learning assistant chatbot design and associated prototype implementation work.Section 4 covers the results from the chatbot trial run experiments.Section 5 explains the conclusion of the paper.

Intelligent tutoring system in education
The intelligent tutoring system provides the students with an interactive learning environment.The intelligent tutoring system uses NLP technique to process and access students text input [2].ITS has a very significant role in one-to-one tutoring and by incorporating NLP in ITLSs made it more capable to understand the learners input text and also provided pedagogical feedback.
Emotional intelligence is incorporated in intelligent tutoring systems [10] that use different styles to teach the learning material to students.An emotionally intelligent tutoring system (EITS) finds the emotional disposition of the learner.The learner's facial expression, mouse movements, clicks and speed are some of the inputs to the tutoring system to analyse the emotional state of the student.A virtual assistant is used to motivate the students for solving algebra problems using a dialog flow tool [11].It gives an alert about the mistakes made when solving algebra related question.

Student modelling for personalization
A teaching-learning process helps to identify the gap in student's knowledge [12].An ITS is an effective system that fills the gap by providing personalized instruction to the students based on their knowledge and learning capacity.The confidence level of the student in applying these grammar rules are measured on the scale very good-good-average-weak-poor.The student's model captures the students' knowledge level after every exercise and generates new exercise if any weak areas are identified.
Deep tutor, a dialogue based ITS is developed to help the students with the science topics [13].Students interact with the system to solve physics problems.Students are given hints to help them find the solutions themselves and using natural language assessment methods, the solutions are evaluated.The Tutoring system gives positive or negative feedback.Thus, there is a significant learning gain in the system when compared to reading worked out solutions.
A conversational intelligent tutoring system (CITS) is used to enhance the learning in autistic children by adopting a visual audio kinaesthetic learning style [5].The conversational intelligent system is adopted to the learning style of these children.Short text similarity (STS) algorithm and text based pattern matching are used to get the response from a domain for a user utterance.The CITS was built on three main components: Component 1 consists of knowledge base and the log files, component 2 consists of the conversational agent and component 3 is the ITS.

Multimodal interaction on educational systems
The framework of multiple modal interaction system for education (MMISE) implements a client server architecture [14].The client deals with various input signals from the student like keyboard typing, mouse click, pad touch, speech recognition and face recognition.These signals are stored in the database for analysis depending on the pedagogical requirement.The output generated from the system is text, voice or any avatar animation to engage the students in the learning process.The survey conducted on teachers' experiences in conducting multimodal teaching and learning show the use of technology as a medium for imparting information, collaborative learning and also to design learning activities for improving the learning experience [15].The communications and the relationship between the teachers and students have changed and learners cope more efficiently with computer and other new modes of information presentation [16].

Application of NLP in pedagogy
Implementing natural language in the learning process not only brings an effective improvement in the learning process but also helps in developing the language process [17].NLP is an effective tool for bringing an improvement in the education sector.Many educational contexts such as e-learning, research and evaluation integrate NLP techniques to bring a positive outcome in the education system.When NLP is implemented in education, there is an increase in the student's learning output.Sometimes, student fails to understand the context due to the obstacle of language.NLP can be a desired approach for improving the learning ability of students.NLP and Multimedia technology is combined to develop a computer assisted teaching system [18].An educational application area of language assessment uses NLP to access the students typed answers [19].Syntactic analysis detects any writing errors while semantic analysis accesses the meaning of student's responses.The discourse processing is used to find the connection between sentences NLP tools are leveraged for measuring or accessing students' content in a project based learning activities [20].A linguistic inquiry and word count 2015 (LIWC2015) is used to analyse the content in the student's blog for the assessment work given by the teacher.The LIWC captures the analytical thinking, emotional tone, clout and authenticity.These categories provide details about a student's engagement while learning in an ITS.The knowledge evaluation of the student's assessment is done through natural language processing.NLP is used for framing the questions, checking the answers given by students and also for giving motivational feedback [21].

Information retrieval via a conversation interface
Question answering (QA) model extracts the exact and precise answer from the dataset [22].The QA model has four different modules namely question processing module, document processing module, document extraction module and document processing module.The answer extraction algorithm extracts the keyword from the question and match the relevant paragraph having the keyword in the dataset.The answer extraction algorithm extracts the keyword from the question and goes through the paragraph extractor and sentence extractor according to the type of questions.
For implementing a question answering system pre-processing operation like stemming, stop-word removal, are performed on the text file of corpus [23].The extracted keywords are stored in Index term dictionary.POS tagging, stemming is done on user query and the extracted keywords are matched with the term in the Index dictionary.POS tagging is applied on the extracted documents to match the same sense as the query sense.A simple question parsing algorithm and question semantic similarity algorithm is used to parse a simple question [24].The experiments results show the precision and recall as high indicated that question answering system using natural language is an effective method for students and teachers to exchange question and answers on network teaching.The new generation bots use NLP to provide human like interaction [25].The bots are classified as conversational chatbots and service oriented chatbots.The paper proposes a concept of 'botplications' for providing services and data through chatbots for an efficient experience.

Knowledge evaluation through conversation
Chatbot developed is used to introduce the concepts of variables and condition and also advanced concepts of finite state automata [26].The chatbot uses pattern matching, lemmatization technique and finite state automata to provide these assessments to the students.The authors did two observational studies to analyse the effect of chatbot on students.Different indicators like task completion, participation, self-reported interest and willingness to learn are measured.It is found that the task completion rate is more for students who used the chatbot.And describe that the chatbot can be used in educational background.
Quiz chatbot responds to student's frequently asked questions to reduce educator's workload [9].The Quiz chatbot asks the students to respond to multiple choice questions and also asks for justification for the answer they had selected.The chatbot also provides personalized feedback, thus allowing the students to identify misconceptions or errors.This also helps the educators to identify the common areas where students struggle.These two applications show that chatbots can be used within education and hence chatbots can improve the students learning experience.An ITS is developed to assist the high school students for learning general knowledge [27].The tutoring system, a web-based application is accessed by a large number of learners worldwide.It uses NLP Techniques to find the answers for queries given by students.This ITS system is trained on a knowledge base of general knowledge questions and answers.The paper discusses about different cloud-based chat-bot platforms like Dialogflow.com (API.ai),Wit.ai, Luis.ai, and Pandorabots.com.The survey conducted shows that majority of students use chatbot to clear their doubts related to their education and students prefer them as one of the quickest modes of communication [28].
A chatbot is developed to measure the memory retention and learning outcome of students [29].A chatbot of an object-oriented programming language is developed and is deployed.It evaluates the students learning methodology.The results of obtaining answers from google and results from chatbot system developed are compared.The results show that the students who use chatbot system show a higher level of memory retention and better learning outcome than the students using the conventional search engine.

METHOD
The approach decided for the research was to pick a suitable subject using which a learning assistant prototype could be built and analysed.The subject must also be efficiently captured within an electronic knowledge base which can be used in an NLP context for information extraction.For evaluating the learning assistant, sufficient quantity of real user queries across various subject topics regarding the key concepts need to be collected from students.The user queries will mimic or simulate the interaction of the chatbot with individual students.

Choosing the subject for the knowledge base
One of the critical underlying imperatives for school education is to shape the way students think.Over the past decade, schools across the world have been turning to Scratch.It is by far, the most popular tool for computer programming education in primary schools by millions of school children.It is a very visual and block-based, drag-and-drop system that enables a highly creative learning experience for all children, irrespective of their abilities and levels of proficiency.Using Scratch, children enjoy making presentations, simple games, cartoon animations, conversation simulations, and puzzles.This provides the freedom for them to explore their boundless creativity.However, since the learning is very experiential, the maximum learning happens while the students try out various projects, learning how to explore and apply the various computational concepts, such as sequencing, iteration, and variables, and computational practices, such as debugging and abstraction.However, the traditional pedagogical models followed in schools for most students are not very suited to subjects like Scratch which are very "hands-on".Based on a survey of students, teachers and parents across a sample of popular schools, we see that during the first year of introduction to Scratch, around 25-30% of the students are seeking help from their parents to clarify their understanding, while doing their Scratch assignments and projects.What if we could let technology don this "learning assistant" role, in a fun and engaging way?Hence this research is focused on exploring the application of NLP for implementing a learning assistant chatbot, which can be of aid in enabling students' better and faster learning of Scratch basics.

Chatbot prototype implementation-rasa natural language processing (NLU) integration
The Rasa stack was used for the development of this conversational chatbot.Rasa NLU and Rasa Core are two popular open-source Python libraries for developing a chatbot.They can provide the required natural language understanding, machine learning and dialogue management capability.The developed bot called "SCRATCHAI" is integrated with Slack for a User Interface, selected based on popular choice.The picture shown in the Figure 1 previous illustrates how a chatbot built using Rasa responds to a message from the user. A question is received from the student and is passed on to an interpreter (Rasa NLU) to extract the intent and entities. The Tracker is used to maintain the conversation status.It gets a notification that a new message has come in  The policy gets the current state of the tracker. The policy decides what action to be taken next. The action that is chosen is logged on to Tracker. The action is executed and answer is sent to the student.

Training the chat-bot
Extracting the intent and the entities is an important part of the conversation management.Rasa NLU is used to derive intent and entities from user's natural language.Intent-Intent is defined as what the user wants to perform.It is expressed in users' messages.The user's message is "Hi", the intent is extracted as "Greetings".If it is "What is a Sprite", the intent is "ask".Entity is the useful information extracted from the user input is defined as entity.The message "What is a Sprite," the entity is "Sprite".
Any NLP Bot should be taught to understand the messages first.In order to teach the model, we must run training sequences through the NLU model which will take the user input in simple text for, based on which it would extract the intents.This intent extraction helps the bot to understand and classify the message from the student.The visual studio project "SCRATCH AI" is the base project directory.For files, a "data" directory under "SCRATCH AI" with a training file is used.The user messages that the bot should understand or the intents are defined in this.The NLU and the core components that the model use will be defined in a configuration file and pipeline "pretrained embeddings spacy" is also added.The NLU model is trained and is passed to the NLU Interpreter to parse the sample intents to find if NLU is able to classify the intent and extract the entities correctly.

Deploying the bot on slack
The messaging interface chosen is Slack 3.9.Slack is a widely used messaging platform which is available on the mobile and desktop.The slack API provides a variety of actions that bot can do on the platform.The bot can send messages into Slack and it can receive messages from the user.The content can include rich text and images and more.The user can also use Slack as an identity provider by signing-up with a Slack account.The Scratch AI bot is integrated onto Slack using ngrok, a lightweight utility that establishes a secure tunnel between slack and the SCRATCH AI program.A bot user OAuth access token is generated from Slack and the details of this token are used in the 'credentials.yml' file of SCRATCH AI project to authorize the connection from SCRACHAI bot to Slack, through the tunnel created using ngrok, on port number 5004.The ngrok tunnel URL generated is also mapped as an Event Subscription Request URL within Slack. Figure 2 shows the creation of the tunnel URL using ngrok

Chatbot evaluation and analysis
To evaluate the performance of the chatbot and to improve it, we have used a 2-pass basic test sample approach, as recommended for any AI/ML model.As part of this, the test stimuli would be first divided randomly into 2 separate halves, to ensure there is no sampling bias introduced at this stage.The first Int J Elec & Comp Eng ISSN: 2088-8708  set of conversation samples would be run on the chatbot and observations made on the quality and accuracy of responses.Based on analysis of the results of the first round of sample testing, the various model configurations, knowledge base and ontology definitions would be tweaked or corrected as the need be.
In the second run, the second sample would be run on the chatbot and same measurements noted.This process can be continued in the same manner until the required level of accuracy and quality is reached, subject to having enough test stimuli for conducting the repeated testing.The test stimulus for this exercise was collected from 60 Grade-3 students in a nearby school of high repute.The students were asked to provide a list of information-seeking questions that they have, for the Scratch module in their 3rd grade computer science syllabus.A total of ~310 questions were received, which were all first combined and then divided into two random sets of 150-160 each.

Analysis results-first iteration
The first iteration of testing the Scratch chatbot for its performance in information retrieval was conducted using the first set of 159 query text samples.It was observed that the chatbot was successful in capturing and matching the intent and entity for only ~35% of the test stimuli.The failure to capture and match the intent and entity for ~65% of the stimuli indicates that the knowledge database that has been built up for the chatbot evaluation needs to be enhanced to cover the missing topics.The confidence levels for intent and entity extraction and matching are shown in Figures 3 and 4. Of the stimuli for which intent and entity extraction was successful, 70% had intent confidence levels of at least 80%, and 96% of them had entity confidence levels of at least 80%.
The distribution of the various extracted entity values is shown in Figure 5.In a real-life situation, teachers and administrators can make use of such entity insights that can be inferred from the chatbot usage by the students.For example, as seen Figure 4, the top 2 topics that were observed, were regarding the "pen block" and "stage" concepts within Scratch.The teachers can then allocate more time and activities towards such most-queried topics.This is a clear illustration of potential direct and indirect benefits that such a system can create for school education.

Analysis results-second iteration
The second iteration of testing the chatbot for its performance was conducted using the second set of 166 query text samples.It was observed that the chatbot was successful in capturing and matching the intent and entity for ~72% of the test stimuli.The confidence levels observed for the intent and entity extraction and matching are shown in Figures 6 and 7.In this second iteration, it was observed that of the stimuli for which intent and entity extraction was successful, though a significantly better 97.4% had intent confidence levels of at least 90%, only a lower 87% of them had entity confidence levels of at least 80%.

CONCLUSION
The work presented in this paper explored the expansion of chatbots into the realm of school education.We demonstrated the application of NLU/NER/NLP techniques to a practical chat bot implementation using computer science school subject as the knowledge base topic.A detailed review of related literature that touch upon various design methodologies and NLP use cases is also presented, which helped in understanding some of the current approaches and their potential drawbacks.
The field of artificial intelligence has grown manifold over the recent years.Development of chatbots or conversational agents and their applications have seen stupendous success in many different business domains.This gives us enormous confidence that with further research and development, such conversational agents can be put into work in a completely unsupervised manner, in a school education context.By a collaborative and ecosystem-based approach, governments and start-ups can drive the required process and technology standardization, to enable quicker adoption and maximum return of investment for the respective educational institutions and content providers.
Such a learning chatbot has the potential to provide coverage across the whole subject if the knowledge base components and language models are enhanced further.This can be achieved by detailed model training and enhancing the knowledge base.A learning assistant chatbot can go another step further if it could perform learning assessments through periodic and on-demand evaluation of the students and provide educators and teachers the access to view and analyse the summary of evaluation results of all the students.Such a system would enable students to get feedback on their learning progress, faster and more frequently.