Comparative Review of the Features of Automated Software Testing Tools

Software testing is considered to be one of the most important processes in software development for it verifies if the system meets the user requirements and specification. Manual testing and automated testing are two ways of conducting software testing. Automated testing gives software testers the ease to automate the process of software testing thus considered more effective when time, cost and usability are concerned. There are a wide variety of automated testing tools available, either open source or commercial. This paper provides a comparative review of features of open source and commercial testing tools that may help users to select the appropriate software testing tool based on their requirements.


INTRODUCTION
Software development involves processes such as software programming, documenting, testing to develop a functional application finally. To check whether the software meets the user requirements and to deliver a functional application, software testing becomes a critical process [1][2][3]. It plays a significant role in the successful implementation of any system application.
Testing an application can either be done manually or be automated using software testing tools. A tester acts as an end-user and tests the correct behavior of most of all features of the application for manual testing. Because of this, manual testing is time-consuming and demanding, and it does not always get rid of all bugs effectively. It is an excellent choice for smaller companies that do not have sufficient financial resources for automated systems.
Automated testing addresses the challenges presented by manual testing. Automated testing allows tester the ability to create repeatable and reusable test scenarios. These test scenarios can then be executed as often as needed. Also with the increasing complexity of software development, it demands that software team use automated software testing tools to test the quality and functionality of the application. An automated test is more effective when time, cost, and usability are concerned.
There are a wide variety of automated testing tools available in the market, either open source or commercial. There are software tools that only perform a specific kind of testing and limited to specific type of language. While those software testing tools that support a wide range of applications, with better features and functionality may require additional costs. To know the differences of one from the other would allow the user to determine the right testing tool for their environment. This study aims to make a comparative analysis of available software automated testing tools by comparing the features of each tool in terms of type of testing, software support, licensing and cost and others.

REVIEW OF RELATED LITERATURE
Systematic literature review and practitioner survey discusses that the benefits of test software include reusability, repeatability, and effort saved in test executions [4]. The paper also supports the superiority of test automation when several regressions testing rounds are needed. Among the identified limitation of automated testing tools mentioned is the high initial cost in designing test cases, the initial investment in the purchase of the test automation tool and may require additional investment in the need to train staff. Test automation is more reliable, programmable, reusable, comprehensive and maintainable, saves money and time in the long run, has greater test coverage and is faster than human interactions compared to manual testing [5].
Efficiency and accuracy are the two main benefits of automated testing over manual testing [6]. Using automated test scripts yields a better return on equity over manual testing. Software testing is one of the most and time-consuming processes in software development [7]. Because of this, software developers have become increasingly interested in attempting to optimize testing to reduce development costs. A discussion of test metrics which serves an an important indicator of the effectiveness of software testing process was presented in [8]. These include organization metrics, project metrics, process metrics, product metrics and static and dynamic metrics. Organization metric refers to usefulness in overall project planning and management. Project metrics are useful in monitoring and controlling specific project. Process metrics use some test process while product metrics relates to a specific product like a compiler for a programming language. Static metrics are those computed without having to execute the product while dynamic metrics require code execution. Testing metrics are used to improve software productivity and quality. A methodological framework in [9] was used to evaluate testing techniques or tools that can be applied in various case studies. The implementation of the case studies is measured from their effectiveness, efficiency, and user satisfaction. Software test tools help software developer to examine software bugs, verify functionality and ensure the reliability of the software developed [10]. Various testing tools were grouped based on their types of applications and were analyzed by their cost and features. Software testing is still one of the most widely used approaches for checking and improving the quality of a software application [11]. One of the contributions in testing research is automated test input generation. Aside from this, there are new frameworks for test execution which promotes shorter cycles in the testing process. In [12], the paper experimented automation testing using three different software. Although automation have an initial high implementation and maintenance cost, test automation can give remarkable remarks in the long run when it is rerun multiple times. Because of repeatability and reusability of test scenarios, test automation increases the overall effectiveness of the testing process. For mobile testing, identify the mobile platform support, the lead time for the new OS, test coverage, text support (languages), test workflow, scripting capabilities, price and service support are needed when considering the tool to guide software developers and researchers [13].

RESULTS AND ANALYSIS
For this paper, the automated software testing tools chosen are the following: a. Selenium IDE. Selenium is an open source and portable testing tool to test web application that supports a different browser, platforms, and operating system [14]. b. QTP/UFT (Quick Test Professional/Unified Functional Testing). UFT(formerly QTP) is a graphical interface record-playback automation tool [15]. c. TestComplete. TestComplete is an application that helps automate software quality tests for websites, web applications, and Windows desktop applications [16]

CONCLUSION
The research is about the analyzation of the different features of automated testing tools such as Selenium, QTP/UFT, TestComplete, Ranorex, Watir, Sahi, and SoapUI. Since automated software testing has become a necessity for companies, based on the discussion, we can choose which among the automated testing tool is to be used for a certain type of testing purposes. In selecting tools, if the project cost is to be given higher consideration, open source tools such as Selenium is the better option. If the availability of support, ease of learning, report generation are to be considered, licensed tools such as QTP/UFT is a good option. For future works, other automated tools can also be included in the study, including their response time.