Comparative study of user experience on mobile pedestrian navigation between digital map interface and location-based augmented reality

Received Aug 16, 2019 Revised Oct 18, 2019 Accepted Oct 30, 2019 Fast-paced mobile technology development has permitted augmented reality experiences to be delivered on mobile pedestrian navigation context. The fact that the more prevalent of this technology commonly will substituting the digital map visualization to present the geo-location information is still debatable. This paper comprises a report on a field study comparing about user experience when interacting with different modes of mobile electronic assistance in the context of pedestrian navigation interfaces which utilize location-based augmented reality (AR) and two-dimensional digital map to visualize the points of interest (POIs) location in the vicinity of the user. The study was conducted with two subsequent experiments in the Zhongli District, Taoyuan City, Taiwan. The study involved 10 participants aged between 22 and 28 years with different experiences in using smartphones and navigation systems. Navigation performance was measured based on a usability approach on pragmatic quality and hedonic quality like effectiveness (success rate of task completion), efficiency (task completion time) and satisfaction in real outdoor conditions. The evaluation findings have been cross-checked with the user’s personal comments. We aim at eliciting knowledge about user requirements related to mobile pedestrian interfaces and evaluating user experience from pragmatic and hedonic viewpoints. Results show that in the context of pedestrian navigation, digital map interfaces lead to significantly better navigation performance in pragmatic attributes in comparison to AR interfaces. Nevertheless, the study also reveals that location-based AR is more valued by participants in hedonic qualities and overall performance.

Fast-paced mobile technology development has permitted augmented reality experiences to be delivered on mobile pedestrian navigation context. The fact that the more prevalent of this technology commonly will substituting the digital map visualization to present the geo-location information is still debatable. This paper comprises a report on a field study comparing about user experience when interacting with different modes of mobile electronic assistance in the context of pedestrian navigation interfaces which utilize location-based augmented reality (AR) and two-dimensional digital map to visualize the points of interest (POIs) location in the vicinity of the user. The study was conducted with two subsequent experiments in the Zhongli District, Taoyuan City, Taiwan. The study involved 10 participants aged between 22 and 28 years with different experiences in using smartphones and navigation systems. Navigation performance was measured based on a usability approach on pragmatic quality and hedonic quality like effectiveness (success rate of task completion), efficiency (task completion time) and satisfaction in real outdoor conditions. The evaluation findings have been cross-checked with the user's personal comments. We aim at eliciting knowledge about user requirements related to mobile pedestrian interfaces and evaluating user experience from pragmatic and hedonic viewpoints. Results show that in the context of pedestrian navigation, digital map interfaces lead to significantly better navigation performance in pragmatic attributes in comparison to AR interfaces. Nevertheless, the study also reveals that location-based AR is more valued by participants in hedonic qualities and overall performance.

INTRODUCTION
A pedestrian traveler dislikes travel uncertainty. If he could get instant, personal and real-time information about public transportation or solutions that help him decide to reduce travel time to his destination, he would have much less uncertainty to dealing with. The common electronic navigation assistance which is being used to support pedestrian action is a mobile based 2D digital map interface. The main advantage of mobile digital maps compared to traditional paper-based map forms (in addition to offering multiple levels of zoom and map data search) is their ability to display layers of dynamic geo-locative information such as POI markers around the position of the user and the inclusion of navigation instructions for next waypoints [1]. Constant improvements also have been made to improve the usability of mobile map interfaces (from rendering, interaction and performance points of view) [2][3].
Recent improvements in the capabilities of smartphones are making the location-based augmented reality (AR) services a reality. Along with this trend, various concepts and prototypes have been proposed, focusing on basic implementation issues of mobile applications like tourism [4][5][6], health [7][8][9] and education [10][11][12][13]. Location-based AR provides a novel interface to smoothly link the physical location coordinate with data processing environments that make a user can see digital POI (Point of Interest) annotation of the particular landmark in the real environment. The location-based augmented reality view provides a direct view of reality enhanced with additional computer-generated data, as opposed to a conventional map, which provides an abstract view from above. Location-based AR can greatly improve the user action to reduce the travel time during a trip, especially for pedestrian travelers [14]. However, to the best of our knowledge, as the location based AR interfaces become more and more prevalent, the fact that this technology commonly will substituting the digital map visualizations is still debatable.
Given this state of affairs, we intend to investigate the user experience level of both the AR and 2D zoomable digital map interface, a fundamental difference between augmented reality and digital map from the pragmatic and hedonic point of view. The pragmatic utility reasons were related to functionality, durability, and practicality. The hedonic reasons were mostly related to the stimulation, identification and beauty. To support our research objectives, we used a self-implemented mobile location-based AR application that provides transit information such as the exact location of several bus stops in Zhongli District, Taoyuan City, Taiwan. This application also provides an approximate distance of the specific bus stop from current user location and step-by-step navigation guidance that are overlaid on a smartphone camera view. The same bus transit data location are offered in two application presentations, one is presented with a Google based 2D map interface and another is based on location-based AR to help a bus passenger understand a location of the nearest bus stop to the destination. The comparison result is obviously contributing to improve the usability and user experience of public pedestrian mobile navigation and that will be a big consideration to choose which technology will be used to presenting the geo location data to the users. The remainder of the paper is structured as follows: the next section describes the methods, study design and review of the related works. Section 3 discusses the study results of comparing the application interface of both smartphone applications for pedestrian navigation. Section 4 draws conclusions and provides a perspective.

RESEARCH METHOD
The methodology of this work follows the usability inquiry approach which is gathers a user's personal information such as likes, dislikes, needs and understanding of the system by talking to users, observing users, using the system in real-world scenarios or letting users answer questions.

Usability oriented approach
The definition of usability is often referred to as "efficiency, efficiency and satisfaction with which a specific set of users can achieve a specific set of tasks in a specific environment." According to this definition, a product's usability key factors are determined as follow [15]: Effectiveness -whether users can achieve what they want to do with the product, Efficiency -how long it takes them to do so and Satisfactiontheir feelings and product attitude.

Prototype description
The study is designed for evaluating the impact of using different application interfaces into the user experience when they are using both application interfaces for the same tasks. The mobile AR prototype which was developed in our research aims at providing assistance to the users with regard to accessing and receiving timetable information about the bus transport network of Zhongli District, Taiwan. The proposed prototype is using a sensor-based approach to tracking. Sensor-based tracking obtains geo-location information by fusing various sensors such as GPS, magnetometer and linear accelerometers to determine where POI annotations should appear in the camera view.
The application architecture of this prototype adopts the 3-tier architecture for data transactions. It involves three key concepts: use web service as data service, modularity in software development, for superior performance and portability, it separates the components of application servers that containing business logic functions and database servers that containing databases physically into different tiers/layers (due to its considerable transportation data size). There are hundreds of routes in Zhongli District, Taiwan and each route has many bus stop objects. As the study scope, this application only generates the augmented reality POI objects in one route that chosen by the user. Detailed methods and development processes of the proposed prototype clearly described in previous research [16]. The complete process flow is shown in Figure 1.

Figure 1. The Process flow of location-based AR that used in this study
We design UI/UX transition of our prototype based on [17] that reveal in location-based AR scenario, users are easily identifying the annotation objects when they have highlight on a landmark, direction arrows to next waypoint and the remaining distance information to a particular POI. The locationbased AR application sends a request to the server and then generates the POIs based on data in the database. After the application gets response data and phone location, it synchronizes the POI with smartphone orientation and movement sensors data. Thus, the POIs marker can be rendered and displayed accurately on the smartphone screen presentation through superimposing virtual objects over real-world bus stop locations. Figure 2 depicts the detail interface presentation in the location-based AR prototype which was used in this study.

Figure 2. The location-based AR user interfaces that used in this study
A radar feature in the upper left side of the screen informs the user about the next waypoints, including those which fall out of the user's point of view. A 'step by step' method has been employed to avoid overlaps among marker icons in environments overcrowded with POIs. This method also gives the user guidance about when turning left or right when arriving in a particular spot instead of display all markers in one screen. On the other hand, the map-based environment was developed based on a 2D Google Maps interface superimposed by markers that contain the same bus stop data that used in location-based AR environment. This application has used the Great Circle Haversine Formula as in (1) to determine the distance between the POI marker and user location [18]. Here, A corresponds to the position of the user and B to the one of the POI. Further, ϕ denotes the latitude and λ the longitude of one of these positions. Finally, Δϕ = ϕB − ϕA corresponds to the difference between two latitudes and Δλ = λB − λA corresponds to the difference between two longitudes. Parameters ϕ, λ, Δϕ, and Δλ must be available in radians to calculate the angle θ between these two points. To determine distance D, θ must be multiplied by the radius of the earth 6371 km.

Selection of testing participants
Faulkner prescribes that 10 participants as a sample will discover as many usability issues as more number of individuals will do. The involvement of 10 contributors is capable enough to give the proportional value-to-cost ratio in usability testing [19]. According to the literature studies, we recruited a total of 10 participants in this research. The participants were chosen based on four characteristics: (1) they have been using an Android smartphone, (2) understand English, (3) can read a direction and (4) not familiar with the exact location on the map. A background questionnaire was given to validate that all participants meet the characteristics. All participants are undergraduate and postgraduate students who have a good understanding of English and have been using Android smartphones in supporting their daily activities. Participants' ages are ranged from 20 to 28 years old.

User scenario
As a way to concretely embody a view of the user's actual and future activities, scenarios have proven very useful to give participants tasks [20][21][22]. Each participant will be responsible for completing all specific instruction scenarios. To avoid adaptation times, the participants used their own smartphones where the application was installed. Scenarios included in this research were divided into two tasks, Blind Search and Directed Search. We test the scenario in two iterations on the same distance location then we measure and compare the result between 2D digital map and our location-based AR prototype. In the first iteration, participants used the 2D digital map to complete the tasks then AR interface in the second iteration. In order to avoid any learning effects of participants, we used different POI locations with the same distance for each iteration randomly. Therefore, it could be ensured that participants navigated for them an unknown route in each experiment. Table 1 depicts the detail activities on the blind and directed search scenarios. Find the random location that given to participant in average 500 m radius.
Select one place that appear on the application screen when the distance are in 500 m average radius. 3 Use the application to find the exact location of POI marker.
Enter navigation mode, see the list of guidance instructions to the next waypoint.

4
Go to the given location that appear in the application interface Follow the instructions to go to POI location that appear in the application screen.
When they did the scenario's tasks, we observed the processes by using data collection forms to record each step transaction. We evaluated the time and success rate from each participant when they complete each scenario. Procedures to follow for each scenario: 1) Describe each step of the task to each participant as an application user. 2) Rate user personal success and elapsed time in completing each step. 3) Record all users' personal reactions to the experiences when they complete the scenario using questionnaires. 4) Let users use any and all resources at their disposal to complete these scenarios with one exception. Do not confer with each other.

User evaluation
The degree of user experience is defined as the value of each pragmatic and hedonic parameter. To gather detailed insights into navigation performance (pragmatic quality), time and success rate mentioned as the parameter for efficiency and correctness [23]. In fact, a user's general impression is recorded by the Attractiveness scale. Attractiveness is a valence dimension (emotional reaction on a pure acceptance/ rejection dimension). The hierarchy of Attractiveness can be seen in Figure 3. Perspicuity, Efficiency, and Dependability are pragmatic quality aspects, i.e. they describe interaction qualities that relate to the tasks or goals the user aims to reach when using the product. Stimulation and Novelty are hedonic quality aspects, i.e. they do not relate to tasks and goals, but describe aspects related to pleasure or fun while using the product [24]. The questionnaire that used for gathering participants' impressions about each version is the reduced version of the User Experience Questionnaire (UEQ-S) to focus on the only aspects of pragmatic and hedonic. Due to the both version of applications (2D map and AR) are presented to the participant in a sequential order one after the other, and they have to fill out a questionnaire concerning user experience for each of them. In such a setting, the number of questionnaire items must be kept to a minimum. Otherwise, the participant will be stressed and the quality of answers will decrease quickly [25]. Each item of the UEQ-S consists of a pair of terms with opposite meanings as shown in Figure 4. The first four items represent the pragmatic quality scale and the last four items the hedonic quality scale. Participants rate each item on a 7-point Likert scale. The answers are scaled from -3 (fully agree with the negative term) to +3 (fully agree with positive term). In order to simplify the instruction and make it easier to fill in the questionnaire, all items have the same polarity. The left side reflects the negative term and the right side the positive term. We have conducted a field trial of the two mobile application variants using as a case study in one of the bus transportation networks inside the National Central University campus that located in Zhongli District, Taiwan. The 10 participants were asked to do the same task scenarios in blind search and directed search subsequently with both application types, our proposed AR prototype and Google Map based interface. To avoid adaptation times, the participants used their own smartphones where the app was installed. While they were performing the tasks, their performance on the tasks was observed and any problems occurred or faced by users were noted. Experimental conditions in Table 2 are showing the same tasks completion in two different application presentations, 2D map based interface and location-based AR images superimposed on road scene.

RESULTS AND ANALYSIS
In this section we present all the results from the study to later interpret them from the user experience point of view. Direct comparison was conducted to explicitly compare the two evaluated application interfaces in term of time and success rate. Table 3 presents the comparison of mean and standard deviations (SD) of the required time to complete tasks and percentage of the success rate of two different task scenarios completed by 10 participants as sample users. ). With the little gap between required times, this data proves that presenting the route line and marker in the 2D map interface still familiar with the user daily usage but displays the POIs as a step by step navigation also understandable to users.
The UEQ-S results summary of the differences in user experience between both location-based AR and 2D digital map interface is presented in Table 4 (rated from −3 (lowest) to 3 (highest)). As already mentioned, the evaluation result can determine whether an app is satisfactory for the navigation tasks and whether it has better user experience scales. The means and standard deviations with the means comparison of the participants'user experience factors between 2D map and location-based AR environments are presented in Figure 5. The means of the pragmatic quality (perspicuity, efficiency, and dependability) scales are higher in the 2D digital map presentation. Even though the map-based interface has a significantly better score in navigation performance, it is noteworthy that there were significant deviations between the two interfaces concerning the hedonic quality. The location-based AR interface is more valued by participants in hedonic quality. The most interesting result of our evaluation is the differences in the hedonic quality scale which is resulted from the intuitive presentation of AR in navigation. The combination of real-world views combined with digital information can provide real-time prompts and supports to increase the immersion of the user. From the standard deviation (SD) value of the UEQ-S, it is evident that several users have a similar experience when using each of the applications in a pragmatic or hedonic parameter.

CONCLUSION
In this paper, we compared navigation performance and user experience of the two different user interface technologies augmented reality and digital maps in the context of GPS-enhanced pedestrian navigation. The usability approach was used in an iterative way during this study to gather insights from participants. In order to avoid any learning effects of participants from each experiment iteration, the different POI locations with the same distance were used for each iteration randomly. In such an experiment setting where both versions of applications (2D map and AR) are presented to the participant in a sequential order one after the other, and they have to fill out a questionnaire concerning user experience for each of them, the number of questionnaire items must be kept to a minimum. Therefore UEQ-S was used to minimize the level of stress and tiredness of each participant so the quality of answers would not decrease quickly. In general, participants succeeded in using both user interfaces presentation and could complete all navigation task scenarios. With the low deviation of the sample data, we conclude that the two interfaces tied with respect to usability criteria. This study also demonstrated that the practical and pragmatic qualities of the digital map-based interface have been valued more than the step-by-step AR interface by the majority of participants due to its familiarity. Nevertheless, participants highlighted the hedonic element and the natural interaction of User Experience in AR are promising for respondents. The combination of real-world views combined with step-by-step digital guidance information can provide real-time prompts and supports to increase independent access skills. Our field study revealed some of the strong and weak characteristics of each evaluated interface paradigm when it used in similar applications. Location-based AR interfaces still need to resolve major usability issues until they can be regarded as an indisputable substitute for traditional map-based interfaces. Due to this study only use the sample size of data, the findings should be interpreted with caution when implemented in real commercial application due to AR still have broad space to explore and several limitations. More respondents also need to be involved to improve the result data.