Augmented reality objects design in augmented story book mobile application for better engagement

ABSTRACT


INTRODUCTION
The folktale is often defined as stories such as fairy tales and wonder tales which consist of magical elements [1]. Older people generally would tell the folktales to children so that the presence of the folktales will remain to exist as time passes. There are some benefits of the folktales when they are presented to children. Children who frequently hear stories can easily come up with the right and wrong concept as well as build up positive attitudes [1]. When the folktales are delivered by storytelling, it might build up the sense of cultural identity amongst children [2].
However, people these days start to disregard those concepts and benefits of the folktale. The parents are no longer give their children bedtime stories about folktale. Smartphone seems to take place in providing children some entertainments so that they are usually seen using mobile devices instead of conventional books.
Smartphones' capabilities currently have gone through some enhancements, making augmented reality books, known as augmented books, which combines the use of conventional book and smartphone can be developed. Augmented Reality (AR) itself is as a system created by computer graphics that consist of virtual worlds combined with virtual reality [3]. Various fields have been applying this sophisticated technology, such as education, tourism, advertising, architecture, and urban scheduling navigation such as indoor building and bus route guidance [3], [4], [5], [6]. It is worthwhile to implement AR in mobile application since mobile devices are renowned for the high mobility feature and easy to be used by everyone, particularly the young people [7].
There are some associated works regarding this topic. Hsu, Chen, and Wu [8] made use of video as an AR object for teaching the history of computing. They applied thirteen videos of historical figures.

571
Those videos would be appeared on tablet PC screen based on the corresponding figures in the printed textbook captured by the tablet's camera. Eighty-four high school students were involved in the research, divided into two groups. The data results show that students appreciated the method of learning history of computing using AR tools. Another related topic is research conducted by Bazzaza et al. [9]. They created iARBook, an immersive AR application along with a book. They used Vuforia and Unity3D as tools for creating AR objects. Also, the other related work is about interactive learning using AR book conducted by Jawad, Habib, and Ali [10]. They used Vuforia as well for creating the AR objects needed. This research is conducted to examine which AR objects design which is worthy to be implemented in an augmented book that tells about local Indonesian folktale. In this research, there are three designs of AR objects, and each of them has their specialties. The first design is AR objects that consist of pictures combined with the narrator's voice. The second design is AR objects consist of limited animation and the narrator's voice. And the last design is AR objects consist of short animated video based on the scene in the picture.
These designs will be examined based on subjective and objective measurements, respectively. The steps of the research are developing the augmented book, performing the subjective and objective measurements, and discussing the results. Also, this AR book is created in both the printed book and mobile application for Android platform.

RESEARCH METHOD
The primary focus of this research is to figure out which design of AR object that suits best for kids so that they have interactivity with the book as well as their parents or guardians.

System overview
The presented augmented book is a combination of printed book and mobile AR application, which runs on a smartphone using Android platform. The story of this book is about Lutung Kasarung, an Indonesian folktale which came from West Java. The main idea of the story is a romantic story between a sacred langur named Lutung Kasarung and a pure-hearted young woman named Purbasari. This folktale is chosen because, despite its romantic genre, there are some moral values within, such as maintaining a good relationship between sisters and preventing to judge anything only by its appearance. It also has a happy ending, which is enjoyable to be read by children.
The physical appearance of the book, which is printed A5 size, is shown in Figure 1. The story is divided into ten pages for the narrations, as well as the illustrations. As for the narration, exposed in Figure 2, the pages are designed in vintage style, yet using child-like fonts. All illustrations are colorized in grayscale as presented in Figure 3. It is widely known that there are two types of AR: Marker AR and Markerless AR [3]. In this augmented book, the type chosen is Marker AR, so that some markers need to be created. Illustrations in this book are the markers.
Since this augmented book also contains sounds as AR objects, the narrator's voices are included in every page. The narrator read every single line of the narration referring to the markers in the Indonesian language. As for the mobile application, the augmented reality presentation is categorized as a handheld display since it runs on a smartphone. The user interface design is simple yet colorful.

System implementation
There are four tasks of AR system on smartphone. The first task is to capture the markers through the smartphone camera. The second task is the precise AR objects appear based on the chosen marker. The third task is showing the specific AR objects (still image, short animation, or animated video) on smartphone screen. And the last stage is playing the narrator's voice (excluding the animated video) at the same time. When all those four tasks were efficiently running in the AR mobile application in the particular design, that system would be considered successful.

Tools and technology
The SDK used for handling the markers is Vuforia. It was chosen since it is a steady and proficient platform for creating AR mobile apps due to its capable technique of the image recognition and the proper technical limitations-handling [11]. It supports 2D as well as 3D contents and provides 512 unique markers for tracking use [12]. All illustrations are stored in the registered Vuforia account.
Amongst those well-known engines for creating augmented reality applications, this research used Unity3D to place the AR objects above the designated markers, processed using a PC. After that, the result was converted to a mobile application for Android. For the animation, Adobe Photoshop and Adobe After Effects were the utilized programs in this research. Moreover, for recording and editing the narrator's voice, the program used was Audacity.

Designs of augmented reality object
In this research, there are three designs of AR objects provided. The first one is designed in colorful picture format within the narrator's voice at the same time. The other one is conceived in limited animation format, also combined with the narrator's voice. And the last one is created in animated video format based on the scene of the picture. Figure 4 shows an example of the first AR object design. When the smartphone camera detects this marker, the AR object displayed on the screen is a colorful illustration of a young man, which is different with the marker that formed as a grayscale illustration of a langur. It is because this young man is the one who lived with Purbasari in a body of a langur for the time being. He could transform back into his human figure because of Purbasari's kindness. Hence, the marker is created as his former langur body while the AR object is his actual human figure. The narrator's voice comes out as well when the marker is detected.
In Figure 5, the AR object design is different from the former one. It is presented as a short animation of the langur transformation, also contained with the narrator's voice. The duration of the animation is indeed short, about 2 seconds, yet colorful, unlike the marker. The animation is played automatically when the marker is detected. Figure 6 shows the last AR object design, which is presented as an animation video based on the story that the narrator told beforehand. The duration of the animation is 20 seconds, showing the langur's transformation as well as the other characters' reactions who saw it. The animated video also contains background music. Like the second design, the video animation is automatically played when the marker is detected.

Participants
Participants in this research were people with wide range of age and consisted of five people based on usability testing required amount of individuals [13], [14]. Additionally, they were selected based on two characteristics: adults and familiar with smartphones.

Measurement
For the measurement, there are some methods regarding AR related research, such as informal evaluation, subjective measurements, objective measurements, qualitative analysis, and usability evaluation techniques [15]. This research used subjective and objective approaches, since the results from those approaches could be compared to each other [16].
The data gathered from subjective approach were counted using System Usability Scale (SUS). There were ten questions presented, and the average score would conclude the satisfaction degree from each of three designs. Questions from Questionnaire as shown in Table 1. These questions were given right after the participants tried each of the design, from only one illustration in the augmented book. Subsequently, the first thing they had to do was tried the first model and answered the questionnaire right after that. After they had finished, they were given the second design and also the questionnaire. The same actions were done for the third design.
After using the augmented book, the participants were asked to complete the questionnaire: Participants have been invited to indicate their satisfaction toward the presented augmented book. For answering the questionnaire, the participants were given 5-point Likert scale (ranging from 1 to 5, which means "strongly disagree," "disagree," "neutral," "agree," and "strongly agree" respectively) [15], [17].
Meanwhile, the objective approach is calculated based on the mobile application's technical elements. There are three instruments: storage, processing or success rate, and resource of the battery. The storage is counted in MB format while the processing and resource are in percentage.

RESULTS AND ANALYSIS
The AR application is formed as a mobile application. The mobile device used for this research was a smartphone having 3GB RAM, 32GB internal memory, and 16MP camera for detecting the markers. Additionally, the operating system was Android 6.0.1 (Marshmallow).

Subjective approach
Based on the designs shown in Table 2., this section is used to explain the results of the subjective approach which used SUS. From Table 2., the scores varied due to the designs. All of the participants' answers were collected and calculated with SUS calculation. The scores in Table 2. shows that every participant's satisfaction degree has improved from the former design to the next one. The highest satisfaction result comes from the third design with the average score 89,5. After that, the second design took the second place with the mean score 83. Moreover, the last place is the first design with average score 80. It means that people feel more engaged if the AR object in the augmented book is designed in third design (animated video and narrator's voice).

Objective approach
Based on Figures 7,8,and 9 this section is used to explain the results from the objective approach. Figure 7 shows the bar chart of how much AR mobile application consumed the storage space used on a smartphone when the specific design was applied. The first design, which consists of colorful, still image and the narrator's voice, used 106 MB of storage. Therefore, the first design made the mobile application consumed the least storage space.
Moreover, then, the second design used 110 MB of storage, which made it the moderate one. The short animations were created in a small size so that they did not consume much storage space. On the other hand, the third design, which consists of animated video and the narrator's voice, used 158 MB of storage. It made the mobile application required space the most when the third design was selected. Figure 8 shows the bar chart of the processing percentage referring to the AR mobile application. The application using first design has processing percentage of 80%. It has the highest score because there was the only minor amount of error. Then, the percentage of the application using the second design is 69%. It has a lower score from the first one due to some error occurred when the application was running. For example, the animation did not appear on the screen, and the narrator's voice came out late. Finally, the third design has the average percentage; it is 70%. There were some errors, yet lighter than the second design.  Figure 9 shows the bar chart of how much AR mobile application consumed the battery of the smartphone based on the designs. The first design consumed 7% of the battery life, which made it the least one needed resource. The second design used 10% of the resource. It has a higher resource yet still lower than the third design which used 15%. The third design had the highest consumption.
From the comparison charts in Figure 10, it can be concluded that the first design of the AR mobile application is running on a smartphone in a better performance than the second and third design since it needs the least storage, has better processing performance, and consumes the least resource.

CONCLUSION
Augmented reality is an excellent technology to be applied in a storybook so that the children have the urge to read and know more about local folktales. Besides, the best design of the AR object based on the result gathered in Table 2. is the third design, which is formed as an animated video and narrator's voice. The second best design goes to the second design, which is formed in limited animation and narrator's voice. However, the first and second design needs storage, processing, and resource more than the first design.
On the other hand, the first design makes the mobile application running on a smartphone in a better performance than the second and third design despite the low satisfaction level. Further research is required for the future works, for example, applying the same methods, yet using different designs of AR object. Furthermore, the objective approach for the mobile application could be more comprehensive, not only from storage used, processing, and resource.