A systematic mapping study of performance analysis and modelling of cloud systems and applications

Received Nov 28, 2018 Revised Jun 29, 2020 Accepted Sep 23, 2020 Cloud computing is a paradigm that uses utility-driven models in providing dynamic services to clients at all levels. Performance analysis and modelling is essential because of service level agreement guarantees. Studies on performance analysis and modelling are increasing in a productive manner on the cloud landscape on issues like virtual machines and data storage. The objective of this study is to conduct a systematic mapping study of performance analysis and modelling of cloud systems and applications. A systematic mapping study is useful in visualization and summarizing the research carried in an area of interest. The systematic study provided an overview of studies on this subject by using a structure, based on categorization. The results are presented in terms of research such as evaluation and solution, and contribution such as tools and method utilized. The results showed that there were more discussions on optimization in relation to tool, method and process with 6.42%, 14.29% and 7.62% respectively. In addition, analysis based on designs in terms of model had 14.29% and publication relating to optimization in terms of evaluation research had 9.77%, validation 7.52%, experience 3.01%, and solution 10.51%. Research gaps were identified and should motivate researchers in pursuing further research directions.


INTRODUCTION
The cloud has been described as a network of systems that interconnect resources based on specific agreement between the use and the vendors [1]. CSPs offer services to users in a virtualized environment. Hence the need to ensure performance as indicated by service models which are: software-as-a-service (SaaS), infrastructure-as-a-service (IaaS), platform-as-a-service (PaaS) [2]. In SaaS, the CSP provides applications and makes them available to clients on the web. The user is not bothered about installations or licenses. In PaaS, the user benchmark on the structured infrastructure of the CSP to optimize the application set up. The user is only concerned with the application and data. In IaaS, the CSP offers resources over the internet for storage, computing and communication purposes. There are issues relating to trust management in spite of the efficiency and reliability provided by the CSPs [3].
Additionally, the cloud operates on four (4) models, these are public, private, hybrid and community cloud [2]. Private clouds is usually hosted on-site in an organization. Private cloud is considered more secure. Public cloud runs on the infrastructure available to the CSP and such infrastructure which is spread across geographical boundaries. Community cloud are hosted by a third party and utilized by organizations with common specialty. Hybrid cloud takes advantage available on the private and public Cloud. In addition, cloud computing is very broad and there are a plethora of subjects such as management optimization and identity management requiring significant discussion. Such issues are directly or indirectly related to the focus of this study [4,5]. Performance analysis on the Cloud involves metrics which are measured and this runs on both real and virtual machines [6]. For multiple users of webs application systems there is the need to understand that the performance of these systems is in real-time [7]. The commercial success of these platforms would depend on providing satisfactory results and quality of service (QoS), hence evaluation or modelling is essential [8]. In [7] web applications were analysed based on statistical distribution of relevant parameters and an experiment was conducted to validate the model. There are many methods for measuring performance centered on characteristics, criteria and evaluation, based on simulation [9]. Existing literature in the study shows some criteria for evaluating the performance affecting Cloud computing average response time. However [9] establish the framework for measuring the performance as a cloud system. The system was analysed using three sub-concepts namely: productivity, responsiveness, and utilization [10].
The outcome was categorized into three groups which are: perform service correctly, perform incorrectly and refuse to perform. [10] reports that techniques such as non-stationary time series analysis, model selection, and stochastic processes can be adopted to analyse Cloud workload; highlight the importance of performance. In addition, performance analysis of data storage is a critical issues especially for not structured query language (NoSQL) databases. Using the Yahoo cloud serving benchmark an analysis was carried out on the Redis and SSDB databases [11]. Performance evaluation of dataset is relevant because programmer expend time examining different classification techniques. The multilayer perceptron classifier offers the best accuracy performance for classifying the instances of datasets [12]. Furthermore, virtual machine utilization is an essential concept on the cloud, hence performance analysis during live migration becomes paramount. It is possible to compare performance value which can be used for controlling and halting unsuccessful migration and save significant amount of time in migration operation [13]. Clearly, the topic of performance analysis and modelling is an interesting one, hence several papers have been written on this subject. When writing an article, it pertinent for researchers to know it comes some technical knowhow. It also involves going through numerous high conference proceedings and journals. Getting a research direction would also involve perusing through numerous digital libraries and attending workshops. Finding out a research topic is indeed an arduous task for the researcher.
This challenge can be solved using a systematic mapping study categorizes materials in a research through the process of unique scheme and structure, which focuses on frequency of publications, thus facilitating a visual summary in form of a map. This study categorized the systematic mapping process into three phases, which are topic-based, research-based and contribution-based facets. The topic phase was used to determine the core aspect of studies in cloud performance analysis. The research phase dealt with the type of study, while the contribution phase is related to proposed method or process. The motivation for this work is that more researches are still required in the areas of performance analysis and modelling on the cloud. Hence, the problem this study seeks to address is identifying areas where there are dearth of papers and pointing it out to upcoming researchers. This paper therefore, aims at carrying out a mapping study based on cloud performance analysis and modelling. The major contribution of this review is that a visual map was created with percentages indicating the extent of work that has been done in various research types and research contributions such as framework and model in this field of study. The remainder of this work is structured thus: section 2 examines the related work. Section 3 focuses on materials and methods, while section 4 discusses the results. Finally, section 5 concludes the paper and provides recommendation for further studies.

RELATED WORK
The work in [14], is a systematic study that was conducted in relation to maps. The study provided guidelines for conducting reviews and offered relevant lessons based on existing studies. The multiple guidelines observed from existing work provided a different perspective for conducting systematic studies. The work in [15], is a systematic study that focuses on software patterns in the area of requirement engineering. The planning phase was used for this analysis, which provided further understanding of these patterns. The outcome was the design of a protocol to enhance research in this field.
The paper in [16], examined cloud policy languages and programming models. The systematic study selected six features for the survey and discussed two facets. The features used were framework, survey, privacy, security, accountability, reliability, and paradigms. The contribution facet and research facet was used in the methodology. The areas of research facet was opinion, experience, validation, evaluation, and solution. The contribution facet focused on the model, tool, and method. The paper in [17], conducted a study in the area of computer science concept diagrams. The systematic study focused on the usage, which resulted in the evaluation, and conceptualization of related work. A manual process and backward snowballing were employed in the studies. The paper added to knowledge in the learing and teaching of these concept maps. The publication in [18], did a study in the area of composition, virtualization, orchestration, and containerization of cloud resources. The systematic study used six features that were services in the aspects of orchestration, virtualization, composition, containerization, deployment, and rationalization. In addition, tool, metric, method, and model were used to examine the contribution facet. To produced maps in the research facet, the opinion, validation, experience, and research papers were examined. The publication in [19], offered a systematic study in the area of game software as it relates to education. The essence was to examine studies in software engineering domain, related to education. The systematic study examined educational software and provided focus for further studies. Using papers published between 1974 and 2016, game software usage, and evaluation was achieved.
In [20], the domain-specific language was the focus of systematic mapping study. The study produced outcomes based onn the research area, contribution type, and content scope. The research features utilized were conceptual papers, experience papers, conceptual and philosophical papers. The process of data extraction, classification were used in the study. The paper in [21] focused on designed and deployment models on the cloud. The systematic study used feature such as services deployment, configuration, security, privacy and implementation to produce the map. The features were used for model, method, tool, and metric in the contribution facet. In addition, the selected features were utilized on experience papers opinion, validation and evaluation research areas.
The work in [22] did a study on classifying cloud-based testing software. The systematic mapping study provided comprehensive empirical research in this field of study. Relevant and adequate papers were extracted, and the outcome of the investigations was statistically depicted. The work in [23] was centered on cloud-based mobile application testing. The systematic mapping study selected testing features such as graphical user interface, security, compatibility, and functional testing. The selected studies focused on research and contribution methods. The research method used the evaluation, solution, and validation research to produce the map. The contribution facet dwelled extensively on testing-as-a-service.
The paper in [24], is a systematic study was conducted in the area of software engineering. The study carried out extensive literature reviews for a unique outcome. The application of systematic literature reviews had experiences that were highlighted in work to enhance more studies. The publication in [25] discussed the topic of mobile and efficient use of cloud resources. The systematic mapping study also used the concept of research and contribution facets in their methods as discussed in [14]. The features extracted were directly related to the concept, and they include data storage, architecture, efficient transmission, computation offload, fault tolerance and collaborative mobile cloud. The areas of the solution, opionion, experience, validation, and evaluation research methods were the focus of the research facet. The features selected was also used to produced map for the model, tool, and method in the contribution facet.
The paper in [26] recommended evidence-based software engineering methods for gathering proofs by exploring conference papers and journals. The study has unique features relevant to the systematic review process. The work in [27] is a systematic review of aspects of distributed denial of service attacks (DDoS). The paper conducted the review using the topics of technique, variable, tools, deployment location and point in time. The paper examined the techniques of neural network and entropy. It concluded that the most used tools are matlab and network simulator, which are usually applied on a network in the detection of DDoS attacks. The work in [28] is a systematic literature review on designing trust-based security for wireless sensor networks (WSNs). The paper employed a quantitative approach and it utilized 24 articles in the review. The paper focused on the topics of designs, applications and protocols and trust factors. The work concluded that majority of the WSN applications prefer to implement trust-based secure routing. The work in [29] is a systematic literature review on enterprise architecture adoption issues and challenges. The paper used 16 articles to examine 19 problems and challenges such as enterprise architecture, overlapping organizational rules, unclear leadership as well as lack of business and IT alignment. The paper concluded that enterprise architecture adoption is still low especially for studies that relate to multidimensional factors and that no attempts were made to quantify the organizational pressure towards enterprise architecture adoption.

RESEARCH METHOD
The work in [14,26] were used as guidelines to carry out the research conducted in this paper. The guidelines provided the desired direction for the unique but repetitive process of conducting a typical systematic mapping study [30]. Figure 1 depicts the steps used in a systematic study, which begins with applying suitable research questions. Thereafter, a search is carried out on peer-reviewed digital databases for suitable papers. The papers are examined to determine the ones that will provide features suitable for the scope of the paper. The features are extracted using the process of keywording from various segments of the paper. Finally, the featues extracted are analyzed and used to produce a systematic map.

Primary studies search conduct
To achieve a thorough study, five major digital libraries were explored by carrying out an examination of journal and conference papers as shown in Table 1. The primary search focused on searching relevant literature from these online digital resources using the search string. Thereafter, a backward snowballing was carried out to complement the search process [31]. The scientific libraries for the study are shown below: Keywords taken from the title of the study were used in the search string and is presented as follows: ((TITLE ("Performance analysis") OR TITLE ( modelling)) AND (TITLE ("Cloud Systems") OR TITLE (applications))) KEY (Cloud). The research string was used on the metadata ensures that relevant papers was accessible. Results from selected databases/corpus were focused on computer science and the cloud. This study utilized 133 out of the population of 1908 papers found in the initial pilot search. This study covered the period 2010 to 2018. The list of included primary studies is at the Appendix.

Screening of selection criteria
In this process, a section criteria removes irrelevant papers from the study. The inclusion and exclusion criteria used in this paper as shown in Table 2. Publications that do not relate to the research questions or abstract, papers on editorials, panel discussions, presentation slides, prefaces, summaries and tutorials were expunged during extraction.

Table 2. Inclusion and exclusion criteria Inclusion Criteria
Exclusion Criteria The abstract explicitly motions performance analysis and modelling Furthermore such performance Analysis and modelling contributes to cloud systems and applications The abstract lies outside the domain of cloud systems and application in addition, the paper dies not focus on performance analysus and modelling.

Keywording
Keyword is a core aspect of systematic review process; key wording of improves the classification scheme. In building the classification scheme, a thematic analysis approach, which was aimed at identifying, analyzing and reporting themes within the studies under consideration was adopted [32]. During the keywording of abstracts, the abstracts of the selected studies were read, from which the concepts, main contributions and the areas of focus that reflected the research questions were identified. However, a large number of concepts and techniques were observed in the selected studies addressing cloud performance analysis and modelling. Thereafter, a combination of set of keywords was gathered from different studies according to thematic analysis approach to form a high-level understanding about the nature and contribution of the research.
Key wording was paramount to cutting down the time needed to develop a classification scheme. Moreover, this process ensured that only relevant articles were considered for the study. The process necessitated having adequate background insight of the study area. The keywords from the different papers on performance analysis and modelling of cloud systems were combined to provide appropriate understanding of the types and contributions of each article. This process was used to determine the various categories of the study. However, it was important to read the background or introductory part, in addition to the concluding parts of an article to ensure a reliable key wording for this study. The category is then determined by a cluster of words. This study utilizes three facets. The main phase focused on topics. The second phase centred on contributions to the topics in the first phase. The third phase dealt with the kind of research conducted.

Research types fact with categories and description
The work in [33] provides the classification approaches for a systematic study. These approaches for a sysytematic study. These approaches were adopted for this research work. The description of the classification approaches is discussed below. a. Philosophical papers: philosophical papers helps in identifying a problem and also providing solutions using new approaches. b. Opinion Papers: opinion papers offers insight into a researcher's mind by providing the researchs perspective in a field of study. c. Experience paper: there are outstanding researchers in various fields of study. Their experience is brought to the fore in conducting further studies. d. Evaluation Research: There are practical and theoretical concepts. Evaluation research offer outcomes from methods that have been implemented. e. Validation research: a procedure can offer valuable insight without practical implementation in the aspect of validation research. f.
Solution proposal: Most research is targeted to real-life solutions. Solution research solves problems. The pros and cons of such a solution are also made available. The studies in [33], were deemed acceptable for use in the research facet. Primary studies in this work were classified based on the categories and description above in determining the research facet.

Data extraction and mapping studies
The process of data extraction for this study was done on a Microsoft Excel sheet. The Excel sheet was used to enter each category of the classification scheme. The publications are then aggregated into tables containing either the "topic and contribution" or "topic and research extracts". Subsequently, the papers are displayed built from results of the sheet. The objective was to find out which aspect of performance analysis and modelling of cloud systems and applications were emphasized more in this work. This enabled the identification of gaps and in effect areas for further research. Excel tables are valuable tools from which bubble plots can be produced based on the results obtained. The bubble plots offers a pictorial representation of the frequency of the publications. The features extracted are used to provide maps along the contribution and research quadrants.

RESULTS AND ANALYSIS
The analysis of performance and modelling of cloud system and applications is discussed in this section. This study was able to identify the gaps in existing study and noticed that there was limited work being done in this research area.

Topic and contribution facet
The contribution facet focus on discussion in the areas listed below [23]: a. Framework : A well-structured and detailed method, with wide scope and purpose, focusing on a number of research questions or areas : Focuses on a more specific goal with a narrow research question or purpose. A lot of papers on topics and subject areas were identified during the research. The topics like optimization, quality of service, response time, architecture and impact were chosen because they were the ones mostly explored. In addition, the topics chosen for the research because they were the trending topics. Topics and contribution is also central to this study. These topics taken from the classification scheme are as follows:  Optimization  Quality of Service  Response time  Architecture  Impact Table 3 shows the primary studies listed for examining the contribution against the topics. Figure 2 displays the percentage of topics in the contribution category. Figure 3 indicates the percentage of topics in the research category. Figure 4 is the systematic map created in this study. The result showed that publications which discussed model in relation to performance analysis and modelling of cloud systems had 28.57% out of 105 papers in this category. While, the metric, tool, method and process had an overall percentage of 4.70%, 18.1%, 26.67%, and 21.9% respectively.
From the papers reviewed, 28.57% was from model discussion. Also, 0.95% of model discussions were on optimization, 1.9% was on quality of service, 11.43% were on architecture and 14.29% were on designs. No papers were found on model in terms of impact. Other contributions in this category are detailed in Figure 4.   Table 4 shows the primary studies listed for examining the research against the topics. Figure 3 displays the percentage of topics in the research category. From Figure 4, this research has shown that the evaluation research was 30.8%, out of 133 papers in this category. In addition, the results indicated that validation, solution, philosophical, experience, and opinion researches had 24.06%, 24.06%, 6.02%, 10.53%, and 4.51% respectively.

Topic and research facet
The discussions on evaluation research contributed 30.83% in this category out of the papers reviewed. The breakdown showed that 9.77% of the evaluation research discussions were based on optimization, with 5.26% on quality of service, 2.26% on response time, 4.51% on architecture, and 6.77% on impact.

Main findings
The first quadrant of Figure 4 shows is a 2 x-y scatter plot with bubbles for contributions and topics. There is also a similar plot on the researches and topics in the second quadrant. Figure 4 shows that in the first quadrant: a. There were more discussions on optimization in relation to tool, method and process with 6.42%, 14.29% and 7.62% respectively. In addition, analysis based on designs in terms of model had 14.29% and publication relating to optimization in terms of evaluation research, validation, experience, and solution were 9.77%, 7.52%, 3.01%, and 10.51% respectively. Hence, there is need for further research endeavours in areas such as impact and response time in relation to model. b. However, it was identified that there were very limited publications in the areas of architecture, response time and quality of service metric. In addition, to the best of the researchers' knowledge, there were no articles in the areas of design method and response time on philosophical research. c. The result on existing study based on performance and modeling of cloud systems and application showed that articles on architecture were the highest, while response time had the least number of papers. Furthermore, tool had the least publications in terms of quality of service with 0.95%. Publications based on response time, architecture and design were the lowest with 0.95% each. d. In the right quadrant, response time and design had the lowest publications on evaluation research with 2.26% each, while response time had the lowest publication in validation research with 0.75%. In addition, quality of service had the lowest number of articles in terms of solution research with 0.75%. From the foregoing, the visual map using bubbles offered a charting method with a clear approach for identifying gaps to guide future research effort. The importance of this study is that it serves as a basis to both researchers and industries practitioners for conducting more studies. This study is broadly divided into five main classes, firstly, the impact in relation to this study with its use; secondly, the architecture; thirdly, quality of service; fourthly, response time and lastly, the optimization. Furthermore, six topics were analysed under this study using contributions of tool, method, model, metric and processes, and researches such as validation, evaluation, solution, philosophical and opinion research. This study is specifically suggested for further consideration by researchers to focus more on the preliminary studies so as to identify possible gaps or effect that could be of a greater achievement in the course of the study.

CONCLUSION
Cloud computing is evolving very fast and so are the issues related to the services being offered. Cloud users and providers alike are benefitting from cloud activities. This has inspired researched efforts both within the industry and among research institutions alike. The use of cloud services will require appropriate resources by prospective clients. The cloud providers must also strive to perform at an optimum level. This has led to lots of studies on performance analysis and modelling of cloud system and applications. Several high impact studies are present in this research. Notwithstanding, a shortage of publications in several aspects still exists. The study conducted in this paper provides adequate features that will enable upcoming researchers in cloud computing to conduct further studies. The systematic map created in this research work identified numerous gaps in this particular field of study. Areas in cloud performance analysis and modelling that have not been explored are now available to interested researchers.