Development and testing of a thai website accessibility evaluation tool

Received May 23, 2019 Revised Jan 1, 2020 Accepted Mar 21, 2020 This paper deals with the important problem that there is no help with the accessibility evaluations of Thailand’s web by developing and evaluating a new method and tool online, WebThai2Access, with experts, developers, and users with disabilities. This tool was evaluated by 30 developers, 30 hearing impaired people, 30 visually impaired people, and 30 elderly people. The developers evaluated the websites whereas experimental tasks were given to each disabled group based on the problems they had accessing web information. The developers found WebThai2Access very usable and the 15 test criteria were reliable for evaluating websites. The lower and upper 95% limits for confidence ratings of developers were minus or plus 10% for YouTube and Pantip websites and minus or plus 3% with the blind association website. The 95% lower and upper limits of confidence were minus or plus 5% for hearing impaired users, minus or plus 2% for elderly users and minus or plus 0% for visually impaired users. The results therefore showed WebThai2Access was reliable and accessible for developers whose evaluations reasonably well predicted website accessibility for users with disabilities.


INTRODUCTION
The National Plan for the Development of the Quality of Life of the Disabled, No. 5, Year 2017 [1] reported statistics for disabled people on September 30, 2016. There are 1,534,763 people with disabilities (2.52% of total Thai population) with identity cards across Thailand. Disabilities can be divided into various categories as follows. There are 794,648 people (48.37%) with physical disability, 300,265 hearing impaired people (18.28%), 175,692 visually impaired people (10.69%), 127,850 mentally impaired people (7.35%), 117,887 intellectually disabled people (7.18%), 9,125 autistic people (0.56%), 7,432 people with learning disabilities (0.45%), 110,476 people with more than 1 type of disability (6.73%), and 6,432 people with none specified type of disability (0.39%). The National Statistical Office [2] stated there were 10 million elderly people (15.38%) approximately in the total population. Wapattanawong and Prasatku [3] state that the number of elderly people in the future is likely to continue to increase 6% per year but during 2015-2020, and the number of elderly people will increase by 7 %. A majority of these disabled and elderly people would gain from websites that were accessible as 62% of US working age people would gain from technologies that were accessible [4] and disabilities increase with age. Research in 2006 reported that 97% of websites in 20 countries did not meet minimum accessibility levels [5] and although no Thailand website figures were published for, it is not likely they are more accessible as unlike lots of the countries tested they do not have legislation. The important problem addressed by this unique research was the lack of an easy tool to apply and a reliable method to help developers who are not accessibility experts with the fixing and evaluation of Thailand's web site accessibility. An additional gain by giving disabled people access to websites is mobile device usability on [6]. At present, Thailand has regulations to support the development of accessible websites. The Ministry of Information Technology and Communication [7] has established a framework policy for information technology and communication of Thailand for the period 2011-2020. It mentioned that strategy 6 was reducing social and economic inequality by developing and applying ICT to creating equal opportunities to access public resources and services for all groups of people. It has prepared standards for government websites but the authors found no rigorous research study on how to apply a Thai web site development approach in practice and analyze the results. In addition, there is a lack of research on evaluating and developing of accessible websites in Thailand. There is only one site assessment tool which has been developed by a business unit, Thai Web Accessibility Team [8], but the evaluation data is not disclosed to the public.
The proposed method that would provide a solution to the identified problem and that was developed through this research was to create a Thai Web Accessibility Assessment method and web-based tool Webthai2access.org that helps developers who were not accessibility experts to assess and fix the accessibility of Thai websites. The research also experimentally validated the use of this method and tool for assessing access to Thai websites for visual impairment, hearing impaired or people who are elderly through experimental studies. From a literature review , to address the identified problem the authors decided to develop criteria for evaluating access to information on Thai websites for disabled people including testing the use of such evaluation criteria with web developers, visually impaired, hearing impaired, and elderly people. Although the criteria developed is relevant to all disabilities and the developers evaluated the websites for all disabilities, the time and resources available for this research project allowed only 90 disabled participants to be used in the evaluation. Since 30 people can provide reasonable statistical evidence in an experiment, it was therefore decided to use 30 visually impaired people, 30 hearing impaired people and 30 elderly people. The selection of these three groups is a practical way to cope with the time and resource limitations as blindness requires visual information to be made accessible through speech alternatives and a website to be navigated using the keyboard, while deafness requires auditory information to made accessible through text alternatives and as people grow older the proportion of the population with a disability increases whether sensory, cognitive or physical.

WEB ACCESSIBILITY
Disability covers impairments, activity limitations of activity, and restricted participation while only certain disabilities are related to the accessibility of websites. For example, physical disabilities that only affect the ability of a person to use their legs will not restrict the ability of a person to use a website. Since a blind person cannot use a mouse but uses a keyboard, an accessible website for a blind person will also be accessible for a physically disabled person who cannot use a mouse but can use a keyboard. A website accessible for a person with a physical disability who cannot use a mouse is however not always accessible for a blind person as they need all information provided in a non-visual form. The accessibility of a website does not depend on the particular type of keyboard a person uses and therefore although different users might use different sizes or designs of keyboard this study does not need to investigate different types of keyboard. Laaziri et. al. [9] proposed a model for assessing Accessibility, Usability, Emotionality, and Persuasivity but did not evaluate its use. Wahyuningrum and Mustofa [10] studied software quality measurement and identified the most widely used method was the empirical method involving users. Nayebpour and Rashid [11] investigated how service-oriented architecture might enable expert systems to respond to fast changing information. Inventor of the World Wide Web and Director of W3C Tim Berners-Lee, wrote that an essential aspect of the web was accessibility by everybody whatever their disability [12].
Therefore the Web was created for all, irrespective of culture, technology or ability and has the potential to remove barrier free for disabled people if accessibly designed. Web information access is a human basic right [13]. Making an accessible web assists people who are elderly and people in less wealthy regions in addition to disabled people with their lives including social activities, health care, employment and education. Accessible Websites that are accessible provide legal, financial, technical and social gains for education, government and companies, [14]. Many people cannot use the web. if not designed to be accessible. As an example, people unable to use a mouse need keyboard access and blind people require images alternative text which impacts on elderly people. 65 % of the approximately two million disabled people in Thailand are unemployed, the majority living in rural parts with over 50 % working in fishing or agriculture. There are no Thai website accessibility laws or regulations even though there is guidance for disability development practice and anti-discrimination laws. Web accessibility guidelines [15] have been  [16] based on international website accessibility research. Research has shown that following these guidelines can also benefit non-disabled users [17]. Ahmi and Mohamad [18] evaluated 25 Malaysian ministry websites and found low compliance with web accessibility standards.

WEBTHAI2ACCESS
Simply translating the original English Web Accessibility them doesn't necessarily solve localisation problems of Thailand context and language and. WebThai2access builds on a previous approach by the 2 nd Author [19] and was developed with a holistic view [20]. This research is the first experimental study to examine the ease of use or reliability of the method of using 15 criteria with accompanying techniques in a web-based tool. This journal article extensively expands on the brief information presented in the 2017 HCI International Conference poster extended abstract [21]. WebThai2Access, an interactive Thai website, was created to assist manually testing any Thailand Website through simplified checklists developed from WCAG 2.0, the more complex Web Content Accessibility Guidelines. Additionally, it provides evaluation tools and techniques. It is simpler to score and use than the WCAG guidelines which have more than 60 tests most of which can only be carried out manually by someone with expert accessibility knowledge and experience as only a few of the tests can be carried out by automatic checkers.
WebThai2Access has summarised and compressed the guidelines and tests into the following 15 criteria that can be followed by developers without accessibility expertise.
-Forms, Signup, Login, and Other: including site or service registration sign-in and working with any forms. -ALT Attribute of Image: enabling blind screen reader user to listen to image description.
-Definition of Link Target: understandable without accompanying button or sentence.
-Frame Layout and Titles of Frames: without titles screen reader users may not know or which won't know which information to next read. or page position. -Style sheet Removal: checking site appearance -Video/Audio/: sign language, captioning, transcripts, , and can help deaf or hearing impaired.
-Audio descriptions of video/animations -: providing spoken information about visuals for visually impaired -Tables: column and row and headers are vital.
-Logical Correct Tab Orderings: link and navigational element order is vital for keyboard users -Keyboard Page Functionality: following log-in if required -Text Editor Accessibility: enables users to alter the text feel and look like in a wordprocessor -Appropriate Forms Feedback: appropriate feedback to input prevents confusion.
-Colour and Contrast Check: appropriate colour contrast levels without distractions -Zooming Integrity of Page: enables resizing of images and text -Readability, blinking elements, style and Text size, and: avoid blinking causing seizures and small serif fonts and text or complex language The 4 scoring levels relate to the conformance levels of WCAG 2.0, where 0% is fail, 33% = A, 67% = AA, and 100% = AAA. The use of percentages has the advantage of allowing averaging compared to categories and understandability compared to level numbering (i.e. a level of 3 is meaningless without knowing the maximum is 4).

RESEARCH METHODOLOGY
There were 7 research phases. Phase 1 involved a literature review identifying Thai Website evaluation tools and differences between English and Thai accessibility guidelines to find web accessibility guidelines = changes for Thai tools used for Thai website evaluations. In the second phase, an interview was conducted with 30 Thai developers, 10 people with hearing impairments, 10 people with visual impairments, and 10 elderly people in order to know the barriers to accessing Thai websites, including information about the websites that each group of users use in their daily life. In phase 3, extensive documentation, tools, tests and guidelines, were developed for local Thailand Culture and Language . A validation study and review was carried out by accessibility experts of the documentation, tools, guidelines and tests. In phase 4, WebThai2Access was built based on the results of the expert review and validation and tested. Phase 5 involved a pilot study user evaluation of WebThai2 Access and the experimental design was finalized based on the results and a WebThai2Access user evaluation was conducted with 30 hearing impaired people, 30 elderly people, 30 visually impaired people and 30 developers.
A case study was conducted for phase 6 to verify that WebThai2access could successfully be used by a developer to first identify and then correct problems in a website and involved a blind student and The developer noted that there is some limitation of using Wordpress such as not being able to add code. The blind student used the website and identified problems they had with it and these problems were also identified by the developer using WebThai2Access. The developer then found solutions to the problems and the new version of the website was used by the blind student who verified that the identified problems had now been solved. Section 4 presents the results and analysis and discusses how developers' evaluations of Thai websites predicted how disabled users used the websites. 33%: Time limited form, few labels, CAPTCHA hard to access the, majority of forms accessible by screen reader and keyboard . 67%: No time limit, alternative CAPTCHA. majority of form accessible by screen reader and keyboard some errors i.e. wrong label identified by screen reader or does not read label or feedback 100%: no time limit or CAPTCHA, clear labels, Forms can be accessed easily through screen reader, and keyboard The WebThai2Access navigation tabs go to: -15 evaluation criteria, with further details -disabilities list with associated tests and descriptions -reviewed and approved products They enter website platform, URL, name, description, review information, reviewer's, email and nameof reviewerand.

EXPERIMENT DESIGN DETAILS
Six developers rated the usability and functionality of the WebThai2Access prototype at https://webthai2access.org using a 5 point Likert scale where 5 was the highest score and the average score was 4.7. Suggestions included more text space by increasing the character limit, having multiple text boxes and changing the word "test" (ท ดสอบ) to "testing" (การท ดสอบ) in the disability tab since when translated into Thai it is confusing for the user. Based on the evaluation results and feedback the WebThai2Access prototype was modified before conducting the experiment. Four groups of participants took part. Each group spent about an hour to do the task apart from the hearing impairment group which spent only 20 minutes on the task as there were only three criteria that affected this group. The developers evaluated all 15 criteria while experimental tasks were given to each disabled group based on the problems they had accessing information on web. The experimental task of each group was as follows in Table 1: The results of the Thai web developers' evaluations were compared with how well Thai elderly hearing impaired and visually impaired people carried out tasks related to their disabilities using the same three websites. This showed how developers' WebThai2Access evaluation scores predicted the accessibility of websites for disabled users (i.e. disabled people's task scores).

RESULTS, ANALYSIS AND DISCUSSION 6.1. Criteria changes
A Thailand Web Accessibility Guidelines official translation has not been done no has there been research regarding localisation of the guidelines for the Thailand's culture and language and. The literature review identified that changes required for the Thai criteria from the original English Web Accessibility Guidelines criteria were font types and sizes: U.K uses Sans-serif size 10-12 px, while Thai uses serif size 14-16 px. Criteria 15's readability, blinking elements, style and text size, was changed from sans-serif fonts to serif and 14 -16 px instead of 10 -12 px to suit Thai websites because of a publication [22]. It is consistent with the three research articles [23][24][25]. It was found that the different type of characters affect reading, but the differences in font sizes do not affect reading. Both young readers and the elderly like to read Thai characters with heads more than headless characters. The information from the literature review corresponds to the interview results that website users like to read headed characters for Thai.

Interview results
Interview results showed that developers have little knowledge about Web Accessibility with no or little experience in using a screen reader so do not know how to use shortcut keys. Problems accessing websites are personal problems. The criteria for evaluating a website that everyone can access is a checklist for predicting problems and obstacles that may occur for people with disabilities and the elderly. The results from interviews include: Visually impaired people have a problem using CAPTCHA as a screen reader could not read the picture and there was no alternative audio. The screen reader could not access some pdf files with no alternate text to explain pictures as they only hear "graphic". Therefore, the researcher has identified problems of "CAPTCHA" in the criteria 1 (Forms, Signup, Login, and ) and the no "alt tag" problem to be specified in the criteria 2 (Image ALT Attribute).
Many hearing impaired people have reading difficulties and so they found reading even plain text difficult and found it even more difficult to read text mixed between san serif and serif. Therefore, the researcher has specified this problem in the criteria 15. Hearing impaired people also found it difficult in filling information in the forms with no response feedback. Some websites provided very difficult content, so they needed sign language video or avatar to help. They did not know how to zoom in or zoom out text. It complies with UK website development criteria [19] and W3C website development criteria that everyone can access [15]. Therefore, the researcher has specified this problem in the criteria 14.
Elderly people found it difficult to read less than 10 point text. This complies with [23][24][25] who suggest that serif fonts, size 14-16 point are suitable for suitable for use in Thai websites. They also found it difficult in filling information in the forms with complex steps including signing up and signing. Therefore, the researcher has specified this problem in the criteria 1. There was not enough colour contrast between background and text. Therefore, elderly people could not see it clearly. There is blinking information on some websites which affect the readers. Therefore, the researcher has specified this problem in the criteria 13.

User evaluation results
As presented in Table 2 the 30 developers who evaluated the accessibility of WebThai2Access gave it an 99%. average score Criteria 15, 12, 8 were rated 99%, 95% and 98%, and respectively. whereas the remaining twelve criteria obtained ratings of 100%. The ages of the 30 visually impaired people varied between 13-23 with an average age of 19. Four had severe visual impairment, 7 became blind after birth while 19 were blind since birth and One used NVDA screen reader while 29 used Jaws. All used the Windows operating system versions 10, 8 or 7 and 20 had 1-2 years' screen reader experience while 10 had 5 years screen reader experience Three used Firefox 9 used Internet Explorer while 18 used the Google Chrome browser. The 30 hearing impaired users hand an average age of 20 ranging between 12 and 50 with had been deaf since birth. Five used mobile devices while twenty-five used a computer s. The 30 elderly users had an average age of 64.5 ranging from 60-89. Four had more than 3 years' experience using websites, 2 had 3 years' experience while 24 had 1-2 years' experience. Nine used a computer, 7 used a smartphone and 14 used a tablet, and. The 30 developers were trained to use a screen reader for the experiment and WebThai2Access and all had experience of developing websites using HTML.
Analysis of the results displayed in Tables 3-5, suggested the test criteria were reliable for developers evaluating websites as for the 15 criteria the average 95% lower and upper and limits of confidence of the developer scores were minus or plus 10% for both www.YouTube.com and www.pantip.com websites and minus or plus 3% for http://tabgroup.tab.or.th and they did not overlap the rating levels of 33% or 67%. This can be interpreted to mean that developers scored with low enough variability for agreement on rating levels using the following statistical analysis: standard error = standard deviation/sqrt 30; 95% confidence interval = 1.96* standard error; upper limit = mean + confidence interval; lower limit = mean -confidence interval. If upper and lower limits do not overlap 0.33 (i.e.33%) or 0.67 (i.e. 67%) then we can say it is reliable. For example, YouTube criteria 1 lower limit is 0.74 which is > 0.67 therefore average of 0.83 can be treated as nearest to rating of 1 (100%).
Using a similar analysis of the results for the disabled users shown in Tables 6-8, suggested that using the test criteria was reliable for evaluating websites as for the 15 criteria the average 95% lower and upper limits of confidence were minus or plus 5% for the hearing impaired, minus or plus 2% for the elderly and minus or plus 0% for the visually impaired, and they did not overlap the rating levels of 33% or 67%.

4907
The average difference between the average scores of the experts and developers was 18% (ignoring difference direction) and 2% when the difference in direction was considered. The mode ratings were the same for the experts and developers for all websites for criteria 3,5,7,8,13, and 14, for 13 criteria on tabgroup, 9 criteria on Pantip and 11 criteria on YouTube. Comparing the scores of the developers with the 3 groups of 30 hearing impaired, blind and elderly, users showed that the average difference for the hearing impaired was 8%, for the elderly was 17%, and for the blind users was 26%, and (ignoring the difference in direction), and-7% -17%, -1%, when the difference sign was considered. The greatest difference between blind users and developers were -58% 52%, 54%, respectively for criteria 1, 2, (www.pantip.com) and 4 (http://tabgroup.tab.or.th). Looking at the mode values there was agreement between the blind users and developers apart from criteria 4, 2, 1where the mode ratings for blind users were 100% 0%, 33%, and for the developers were 33% 67%, 100%, respectively. The mode ratings by the blind users were the same as the expert ratings for all criteria suggesting that the experts were better that the developers at predicting how the blind users would perform. This might be because of the lack of screen reader experience of the developers.
The greatest difference between and elderly users and developers were and 23% and -respectively for criteria 13 and 12 (www.pantip.com) and the only mode ratings difference were for criteria 13 where the developer mode was 67%. And the elderly mode was 100% The expert rating for criteria 13 was also 67% suggesting that the experts were not better than the developers at predicting how the elderly users would perform on criteria 13. The average differences between and developers were -2%, 12%, 12%for criteria 15, 12, 1, respectively and the mode ratings were the same, suggesting the developers predicted the hearing impaired people's performance quite well.

Case study results
For the Case Study, problems in accessing the website that had been identified and then solved were: a. No Image ALT Attributes and Text Alternative for all images in the whole website. Therefore, a screen reader could not read them: improved by adding ALT Attributes and Text Alternative for all images b. No Image ALT Attributes and Text Alternative for a map as it was placed in a frame and the URL of the map was copied from google map location therefore a screen reader couldn't read it. Tried to add code <frame name = "name"> but it didn't work because the codes were copied from the map, so when adding code, the picture of the map disappear: improved by adding alternative text for a map. c. There is no caption in the video. No Image ALT Attributes and Text Alternative for a video and also the video was placed in a frame. The video was created in the center in Bangkok so the developer couldn't add captions herself. The developer also didn't know how to create captions so the researcher taught the developer how to add captions for future work. The researcher taught the developer how to add captions on to the video. d. There is no explanation label in the "Other" field in the satisfaction form so a screen reader couldn't read it: changes by adding an explanation label on "Other" field. These results and analysis therefore show that it is possible to develop a unique easy to use and reliable method and online web tool WebThai2Access with 15 criteria that can be used by developers who are not accessibility experts to evaluate and fix the accessibility of Thai websites. Previous published research has involved accessibility experts using the complicated WCAG guidelines with some 60 test criteria and has not investigate Thai website accessibility.

CONCLUSION AND FUTURE WORK
This research developed, evaluate and verified a unique solution, WebThai2access, that addressed the important problem of a lack of an easy to apply way to help developers with the fixing and evaluation of the accessibility of Thailand websites. The results showed the good accessibility and reliability of WebThai2Access with developers' evaluations predicting the accessibility of websites for disabled users reasonably well. The average rating of 4.7 out of 5 showed developers found WebThai2Access very usable. For the 15 criteria the average 95% lower and upper limits of confidence of the developer scores were minus or plus 10% for both Pantip and YouTube websites and plus or minus 3% for blind association website and they did not overlap the tool rating levels which were 33% or 67%. The results for disabled users suggested the test criteria were reliable for website evaluation as for the 15 criteria the average 95% lower and upper limits of confidence were minus or plus 5% for the hearing impaired, minus or plus or 2% for the elderly, minus or plus 0% for the visually impaired, and they did not overlap the rating levels of 33% or 67%. The case study verified that WebThai2access could successfully be used by a developer to first identify and then correct problems in a website. Future work should research how to even further improve the predictions of website issues and also develop techniques and criteria for evaluating mobile device accessibility.

RECOMMENDATION
Based on the results of this research it is recommended that all website developers in Thailand use WebThai2Access to evaluate the accessibility of their websites to improve their accessibility. WebThai2Acess can be used when designing and developing a new website and also for evaluating existing websites to make them more accessible.