Measuring the quality of e-commerce websites using analytical hierarchy process

Improving website quality for e-commerce website is indispensable since it affects customer satisfaction. There are several aspects of website quality that should be considered. Unfortunately, what criteria that should be prioritized is still under research. This research aims to identify the priority of website quality criteria and incorporate these criteria to measure the quality of ten e-commerce websites in Indonesia using Analytical Hierarchical Process (AHP). This study interviews two experts to assign priority for each criterion. In the end, this study has found that availability is the uttermost aspect to consider. Furthermore, this study also found that OLX.com is the best Customer-to-Customer (C2C) e-commerce in Indonesia in terms of website quality. This research is useful for any e-commerce technical developer to improve his/her website in twofold: 1) criteria priority to improve the quality of C2C e-commerce websites and 2) website quality ranking of C2C e-commerce.


Introduction
The popularity of internet usage leads to higher online trading activity in Indonesia.In early 2018, the number of internet users in Indonesia reached 132.7 million and 40% of the population have made online transactions [1].Furthermore, Indonesian government expects that the e-commerce in Indonesia to grow to USD 130 billion in 2020 [2].With the high number of online transactions and search volumes, the competition among e-commerce companies in Indonesia is increasing.
In Indonesia, there are various Customer to Customer (C2C) e-commerce channels.Some of popular e-commerce channels are Tokopedia, Bukalapak, and Shopee [3].With the increasingly competitive market rivalry conditions, every all C2C e-commerce companies are necessary to improve the quality of their websites.This happened because it is a straight representation of the store, thus a website could affect the customer's perception [4].
To improve the quality of the website, all C2C e-commerce companies must invest their money and their time.Nevertheless, there is no guarantee that the corresponding efforts will result on the higher customer's acceptance [5,6].Also, a C2C e-commerce company need to continuously evaluate its relative position given other C2C e-commerce companies to survive.By doing so, a C2C e-commerce company will keep updated about its website's quality status.Hence, it could provide accurate immediate actions when necessary [7].Therefore, evaluating C2C e-commerce website becomes crucial.
There are plenty approaches for website quality metrics.Most studies in the website quality metrics have only focused on user's perspective [8][9][10][11][12][13][14][15][16][17][18][19][20].These studies do not attempt to consider website's developer perspectives as the basis of a website quality metric.At this point, website's developer perspectives put more emphasis on the technical aspects rather than either website's usability or functionality.Thus, it could consider this metric as a generic representation for many websites regardless their working purposes.
The idea to incorporate developer's perspective is not new.Several attempts have been proposed before.Dominic et al. conduct research using various criteria of website quality measurement on Asian e-government website, Asian airlines website, and Malaysian university website [21][22][23][24].One of them uses the Analytical Hierarchy Process (AHP) method [24].Nonetheless, these studies fail to consider that each website type is unique; hence, the criteria priority should not be given uniformly [21,22].For example, a boutique website should have a ISSN: 1693-6930  Measuring the quality of e-commerce websites using analytical hierarchy... (Umar Abdul Aziz) 1203 more colorful website, while a learning website should be more informative (not necessarily colorful) [4].This objective of this study is to find the priority of website quality criteria to help C2C e-commerce website developers in developing their website.Using the criteria priority, this study measured the quality of C2C e-commerce websites in Indonesia.The criteria used in this study were adopted from a study conducted by Dominic et.al [24].In the analysis process, AHP method is employed due two main reasons.First, it is considered more flexible and sustainable in measuring subjective and objective evaluations.Second, AHP also has a consistency test procedure [25].The criteria used when measuring the website quality are load time, page size, number of items, availability, broken links, traffic, design optimization, and markup validation [24].The measurement process of all C2C e-commerce websites on each criterion will use the help of web diagnostic tools.
This paper comprises of four sections.The first part contains the introduction that includes the background of the problem and the solution offered; the second part is the research method that includes the method used in this research; the third part is the result and analysis, and the last part is conclusion containing the conclusion of this research and suggestions for future researches.

Research Method
This section explains the methodology employed in this research.There are four steps namely problem formulation, designing analytical hierarchy process, data collection and data analysis.The research flow is available in Figure 1.

Problem Formulation
This stage formulates the problems to address.There two employed problem formulations: 1) What is the priority of quality criteria for C2C e-commerce website; 2) What is the ranking of C2C e-commerce based on their website quality?

Designing Analytical Hierarchy Process (AHP) Model
After formulating research problems, the next stage is to design a three-level hierarchical model to construct the research objective.The first level is the goal to find, the second level is the lists of assessment criteria and the third level is the lists of the websites that under investigate.The third level contains C2C e-commerce websites.The AHP model developed in this study is adopted from [24].Khan and Dominic developed an AHP model for selecting best quality of airline websites in Malaysia [24].Here, instead of selecting the best airline website, this study aims to investigate the best C2C e-commerce website.The result of the proposed model is available at Figure 2.

Data Collection
This research produces two data types: qualitative data and quantitative data.The quantitative data is obtained from tools and is not incorporating subjective judgment to produce data.In reverse, qualitative data relies on the individual subjective judgement.Qualitative data is conducted when a reliable tool to produce data is not available.This happened partially because of the intertwined complexity of the aspects among data sources.This research uses web diagnostic tools to yield quantitative data and expert judgement to generate qualitative data.In terms of qualitative data, this study uses questionnaire given to experts to develop a pairwise comparison among criteria.Website diagnostic tools used in this study can be seen on Table 1  The authors then give questionnaire to experts along with this following information: the filling instruction, the expert personal information, the criterion description, and the pairwise question for each criterion.Furthermore, the potential answers of the pairwise questions are the extension of that of AHP and are Likert-based.The scale of the pairwise question is available at Table 2 A little more important Experience and assessment leans a bit towards a factor than its pair 5 More important Experience and assessment leans heavily towards a factor than its pair 7 Much more important One factor is favored heavily and has shown dominance rather than its pair 9 Absolutely more important One factor is absolutely favored than its pair with high confidence rate

2,4,6,8 The middle value is between two adjacent decision values
If there is doubt between two of the above scales then take the middle value, which is 2,4,6,8 When web diagnostic tools have been successfully collected and questionnaires have been created, the data collection process can begin.The process of data collection using web diagnostic tools is conducted every day for a month (28 march-26 April 2018) at an unspecified time (random sampling).Then the process of data collection by using the questionnaire is done by purposive sampling method.Our experts (interviewees) are someone who has experienced ISSN: 1693-6930  Measuring the quality of e-commerce websites using analytical hierarchy... (Umar Abdul Aziz) 1205 at least three years in the field of marketing and website development.The interviewees will be asked to fill out the questionnaires and the authors conduct depth interviews related to the reasoning of the scale by them.

Data Analysis
At this stage, the authors perform data analysis that has been successfully collected using AHP method.Qualitative data that have been obtained will be processed into priority criteria.While the quantitative data that has been obtained will be processed into an alternative priority.Once the priority criteria and priority alternative are identified, a priority synthesis process is established to obtain overall priority.Based on the result of the overall priority, the writer can know the ranking of the quality of C2C e-commerce website studied.

Results and Analysis
In this section, the authors do an analysis of all data that has been obtained by using analytic hierarchy process method.In addition, the authors also do a comparison of the weight of the criteria obtained by weighting the criteria used in other studies.

Result of Data Search
At this point, authors have obtained all the required data from web diagnostic tools and questionnaires.The quantitative data are presented in Table 3 and the qualitative data are presented in Table 4. Table 3 shows the test result of C2C e-commerce website using web diagnostic tools.The first column shows the measurement criteria, the second column shows the unit of each criterion, and the other column shows the test results of the website for each criterion.Table 4 shows the results of pairwise comparison data for each criterion by the interviewees.In the first column, there is a measurement criterion that is compared, and the other column is a comparison scale of the criteria of all sources that have been generalized using geometric mean.

Analysis with AHP
After obtaining the quantitative and qualitative data, the next step is to calculate the weight of alternatives and the criteria weight.After that, the authors synthesize criteria weighting to the alternative weight.So, the authors get the final result of assessment analysis and ranking the quality of C2C e-commerce channel website.Table 5 displays the result of the analysis.
Table 5 shows the AHP results.The first row explains the used criterion of the website quality.Next, the second row describes the priority criteria obtained on the results of qualitative data analysis.Meanwhile, the first column list the C2C e-commerce websites.Furthermore, the second to twelfth columns clarifies the alternative priority obtained from the results of  ISSN: 1693-6930 TELKOMNIKA Vol.17, No. 3, June 2019: 1202-1208 1206 quantitative data analysis.On the other hand, the thirteenth column communicates the overall priority score for each website.Finally, the last column expresses a ranking of website quality based on overall priority order.Based on the results obtained in Table 5, it is known that the ranking of C2C e-commerce website quality are: 1) OLX; 2) Shopee; 3) Blibli; 4) Bukalapak; 5) Jualo; 6) Prelo; 7) Tokopedia; 8) Jakmall; 9) Mataharimall; 10) Blanja.OLX overwhelms other platforms in two criteria: page size and number of item along with Jualo.Interestingly, the vast majority of C2C platforms has similar availability with the value of 0,111.It implies that C2C platforms maintain the same standard of availability.Furthermore, this finding aligns with the expert point of view that give availability as the top priority.While OXL provides a good number of item and page size, Shopee as the second best C2C website is excellent in markup validation.Although this TELKOMNIKA ISSN: 1693-6930  Measuring the quality of e-commerce websites using analytical hierarchy... (Umar Abdul Aziz) 1207 criteria has the lowest priority given by the experts.Nonetheless, the performance of Shopee in other criteria is quite good in average compare to other websites.In contrast, Blanja as the lowest quality website exhibits poor performance in two highest rank criteria, traffic and response time.

Priority Criteria Comparison
Based on the results of the analysis obtained in the previous stage, the authors could obtain the priority of each criterion.This section compares and contrasts the proposed priority with that of proposed by other studies.The comparison result of criteria priority can be seen in Table 6.Table 6 shows that in this study, the criteria that have the highest priority are availability.On the contrary, in other studies, load time criteria have the highest priority [21][22][23].According to our experts, the low availability of C2C ecommerce website may lead to a decrease in sales and customer defection.Table 6 also reveals that the priority of markup validation criteria in this study has the lowest priority.Other studies also confirm the same [21][22][23].After a depth interviews with our experts, it is known that the markup validation criterion is an unimportant criterion.According to them, the website can still run even though there is an error in the markup.
Table 6 also indicates the similarity of priority response time criteria in this study with other studies.It shows the response time has a high level of importance.According to the interviewees, slow response time can cause customers to feel stressed and bored.So, customers can move to other competitor's website that has a faster response time.Table 6 also displays significant priority differences in broken links criteria from this study compared to other studies.In another study, the priority of broken links criteria has the eleventh priority [21,22].However, in this study, criteria broken links have the fifth priority.Corresponding to our experts, broken links are important criteria.According to them, broken links are the way to another page.If the link is dead, it can complicate the customer.

Conclusion
This research focuses on proposing new priority criteria for assessing the quality of C2C e-commerce websites.By using the new priority, this study also evaluating the quality of ten C2C platform in Indonesia using an AHP model.From the result, it can be suggested that OLX is the best C2C platform followed by Shopee.Furthermore, this study also reveals that the most important criteria for website quality in the context of C2C website is the availability, while the lowest priority is markup validation.In the end, this study may beneficial for C2C developers for maintaining their website quality to improve their customer satisfaction.For further research, it is interesting to conduct a C2C e-commerce channel assessment based on the quality of the mobile version of the application.Accordingly, it is necessary to research the framework of mobile application quality assessment to identify what factors can be assessed.

Table 1 .
. Online Web Diagnostic Tools

Table 3 .
Quantitative Data for C2C E-commerce Website

Table 4 .
Qualitative Data for Criteria Pairwise Comparison Matrix

Table 5 .
Final Result for C2C E-commerce Based on AHP

Table 6 .
Comparison of Criteria Priority