Benchmarking medium voltage feeders using data envelopment analysis: a case study

Feeder performance evaluation is a key component in improving the power system network. Currently there is no proper method to find the performance of Medium Voltage Feeders (MVF) except the number of feeder failures. Performance benchmarking may be used to identify actual performance of feeders. The results of such benchmarking studies allow the organization to compare feeders with themselves and identify poorly performing feeders. This paper focuses on prominent benchmarking techniques used in international regulatory regime and analyses the applicability to MVFs. Data Envelopment Analysis (DEA) method is selected to analyze the MVFs. Correlation analysis and DEA analysis are carried out on different models and then the base model is selected for the analysis. The relative performance of the 32 MVFs of Western Province, Sri Lanka is evaluated using the DEA. Relative efficiency scores are identified for each feeder. Also the feeders are classified according to the sensitivity analysis. The results indicate that the DEA analysis may be conveniently employed to evaluate the performance of the MVFs. The evaluation is carried out once or twice a year with the MV distribution development plan in order to identify the performance of the feeders and to utilize the available limited resources efficiently.


Introduction
Most of the Provinces of Ceylon Electricity Board (CEB) assures 100 % electrification programs and successfully achieved the targets [1,2]. In Western Province South-I (WPS-I) of CEB almost 100 % of the area is electrified [3,4]. After this electrification, work the main focus is given to improve the performance of the network. Many proposals are identified for improving the performance of the MVFs, but with limited resources only some of them are implemented in a given time period. The present practice is implementing proposals which are essential to keep the network at stipulated margins [5] (voltage levels at + or -6 present, feeder and transformer loading at rated values, etc) and select few other network improving proposals [6,7]. Various issues and contradictions occurred among Area Engineers, Distribution Maintenance Engineers and Planning Engineers; while assigning targets to be achieved for feeders due to lack of proper method to evaluate the current performance of the feeders. Performance benchmarking is widely used and it is very important for any type of organization. The results of such benchmarking studies allow the organization or the unit to compare itself with the best organization or unit and to develop strategic plans for improvements in their performance. Distribution Networks are benchmarked on Geographical or Area-wise in the world [8][9][10][11][12][13][14]. Other than that few Distribution feeders are benchmarked to evaluate the capability of integration of Distributed energy Resources [15][16]. But those methods cannot be used directly in Sri Lanka because many MVFs go through many consumer service centers (CSCs) and Areas. Therefore poor performance or failures in one CSC or Area may affect the performance indicator of the other CSC or Area. It is vital for any type of organization to evaluate the performance and to possess a clear idea regarding their performance. If a proper method is available to evaluate the performance of feeders then the limited resources are fully utilized and system improvements are done to the most affected feeders. In this paper a methodology is introduced to benchmark MV feeders in Sri Lanka and results of a case study for WPS-I of CEB using the introduced methodology are discussed. Section 2 discusses about the prominent benchmarking techniques and justifies the choice of Data Envelopment Analysis (DEA). Section 3 elaborates about the mathematical formulation and characteristics of DEA technique. Section 4 evaluates the DEA efficiency scores on benchmark MV feeders in Sri Lanka. It also analyzes this methodology on WPS-I of CEB. Section 5 discusses about the sensitivity analysis and efficiency scores of feeders. This section also recommends a methodology in benchmarking study of MVFs.

Prominent Benchmarking Techniques
In assessing the most appropriate benchmarking methodology, the following principles are considered [17].
-Practical application -Robustness -Transparency and verifiability -Ability to capture business conditions adequately -Restrictions -Consistency with economic theory -Regulatory burden The summary of characteristics of benchmarking techniques are given in Table 1. Partial Performance Indicators (PPI) may be avoided since it is not capable of accommodating the differences in operating environments and also unable to describe the overall performance of feeders. DEA, Corrected Ordinary Least Squares (COLS) and Stochastic Frontier Analysis (SFA) accommodate the differences in operating environments and also describe the overall performance of the feeders. COLS and SFA require functional relationship with inputs and outputs but it is very difficult to find functional relationships in the properties of the MVFs. The COLS and SFA are difficult to be interpreted for multiple inputs and multiple outputs. Thus DEA is the only suitable method to benchmark MVFs as per Table 1. Various surveys of benchmarking methods incorporated in few countries for Electricity Transmission and Distribution utilities [18] strongly indicate that DEA is the most widely used benchmarking method. In this paper, DEA was selected for benchmarking study.

Data Envelopment Analysis 3.1. Introduction to DEA
DEA is considered as a non-parametric programming technique which creates an efficiency frontier by optimizing the weighted output to input ratio of each Decision Making Unit (DMU). This is subject to the condition that this ratio is equal to 01, but never exceeds 01 for any DMU considered. DEA is a linear programming type technique and it is based on an optimization platform. DEA evaluates the relative efficiencies considering the input and output variables used for the analysis. It also identifies the most efficient units and inefficient units which demands improvements. The efficient and inefficient units are categorized after analyzing the inputs used and the outputs produced by of all the units or divisions. DEA evaluates the amount of resources or the properties to be reduced in order to become efficient as other units. The targets are given to the relatively inefficient units by DEA analysis, which in turn enables it to become relatively efficient. By implementing various system development techniques, the units or organization are able to achieve the best practice or relatively efficient unit's performance. By that system will be developed gradually in the most economical way.

Mathematical Formulation of DEA
In order to obtain the highest possible value for efficiency rating (θ) for the DMU being considered, the set of values for the coefficients u's and v's are evaluated using linear programming technique. In this model the following notations were used. j : number of DMUs considered for DEA DMUj : DMU number j θ : relative efficiency rating of the DMU being evaluated by DEA yrj : amount of r th output produced by j th DMU xij : amount i th input consumed by j th DMU i : number of inputs used by the DMUs r : number of outputs generated by the DMUs ur : coefficient or weight assigned by DEA to output r vi : coefficient or weight assigned by DEA to input i λ : weights If the value obtained for the efficiency rating (θ) for a particular DMU is less than 100%, then that DMU is called as relatively inefficient. That means it has the capability to produce the same level of output with lesser amount of inputs. The following objective function is used in the study to maximize the efficiency rating θ for the DMU.
The above mentioned objective function is subjected to the constraint that when same set of u and v values are applied to all the DMUs being considered the efficiency rating θ is always less than or equal to unity.  In order to run DEA on a standard linear program package it is algebraically reformulated as follows: Assume that there are n DMUs. Then the dual linear program of above model can be interpreted as follows. Minimize θ subjected to: In (10) is the weighted sum of inputs of other DMUs, which is less than or equal to the input of the efficiency rating of the DMU that is considered. It is evident that the weighted sum of outputs of other DMUs is greater than or equal to the output of the DMU that is considered. This model is referred to as "envelopment model".

Orientations in DEA
In performance evaluation, DEA basically comprises of 03 orientations. According to the type of organization, their service or main task, the most appropriate orientation is selected. There are three main orientations in DEA namely input-oriented, output-oriented or base oriented models. In input-oriented models, given amount of output are produced that consumes the smallest possible amount of inputs. Thus the outputs are uncontrollable and inputs are controllable. In output-oriented models, the DMU produces maximum number of outputs with given amount of inputs. Here the inputs are uncontrollable and outputs are controllable. In base oriented models, the DMUs are expected to utilize the minimum level of inputs to produce maximum level of outputs. It implies that both inputs and outputs are controllable. Figure 1 depicts the projection of an inefficient unit on the frontier with the three possible orientation of a DEA model.

DEA Analysis Software
Typically simple problems are solved using equations and graphical methods. But slightly advanced problems are solved using solver parameter in excel [19,20]. DEA use detailed studies and hence the conventional methods are not suitable. Licensed software (Warwick DEA software, DEA Frontier Analyst Software, Performance Improvement Management Software (PIM-DEA), etc.) are utilized to solve advanced and detailed DEA analysis. Other than that free and open source software (DEA Frontier Analyst Free software & etc) are available for DEA analysis. The characteristics of above methods are summarized in Table 2 and this inclines the advantage of using DEA Frontier Analyst Free software in this paper. This software is capable of evaluating 20 Feeders at any instant of time.

. Introduction
The selection of suitable input and output variables are very significant in DEA analysis. The criteria of selection of these inputs and outputs are quite subjective. A DEA study ideally starts with an exhaustive initial list of inputs and outputs that are considered relevant for the study. At this stage, all the inputs and outputs that possess a bearing on the performance of the DMUs to be analysed should be listed. Screening procedures, which may be quantitative or qualitative, may be used to pick the most important inputs and outputs. A rule of thumb (from international practices) is that for 'm' number of inputs and 'n' number of outputs, there are n x m number of feeders. Otherwise all the Feeders would get closer to 100% efficiency and discrimination may be difficult. Factors to be considered when selecting input and output variables are [21][22][23][24][25][26][27]: a. Availability of data b. Easiness to collect data c. Relevant to electricity distribution d. Accuracy e. Common usage in available literature f. Transparency

Data Collection
The information of MVFs is obtained under the following categories:

Facts Regarding Data Collection
A feeder rearrangement was done in July 2014 in Panadura GSS due to addition of a new Power Transformer to the GSS. The data was obtained from 2013 July to 2014 July. The peak power loss and maximum voltage drop are obtained from SynerGee software for the normal feeding arrangement of the feeders. That software is used by CEB for Power Distribution planning purposes. The reliability indices are not calculated separately because the summation of feeder off duration due to auto tripping and feeder off duration due to manual tripping is equal to SAIDI value of a particular feeder. The summation of number of auto trippings and the number of manual tripping is equal to SAIFI value of a feeder.

Correlation Analysis
All the available data cannot be used for DEA analysis as the multiplication of the number of input and number of output should be a minimum value. Therefore most appropriate inputs and outputs are to be selected. Correlation analysis is done to identify the relationship between available data categories. The relationship between two numerical variables is also measured by correlation. Here the target is not to use one variable to predict another variable. But it shows the strength of the linear relationship between two variables. Table 3 shows a guideline to the correlation analysis. When correlation coefficient r = ± 1 it indicates that there is a perfect positive or negative correlation between those two variables. If the value of r=0 that means there is no relationship that exists between the two variables. All other values of r fall between -1 & 1 and the value indicates the strength of the relationship between two variables. Table 3 may be used as a guideline to the adjective that may be used for values of r obtained after calculation to describe the relationship.
According to the study, maximum voltage drop percentage and peak power loss possess strong positive linear relationship. In SynerGEE software, the voltage difference between two nodes (small section of a feeder modelled in the software) is used to calculate the power losses. The feeder length, number of manual trippings, number of manual trippings < 5 minutes and feeder off duration due to manual trippings < 5 minutes showcase strong positive linear relationships. In a normal feeder comparatively higher portion of them are less than five minutes because most of the manual trippings are done for load transfer and switching operations (for switches like DDLO and Air circuit Breakers where on-load operations are not possible). Most of the switching operations take less than five minutes time period. With the increase of feeder length, the number of switches of the feeder increases and switching operations increases. That is the reason for having strong positive linear relationship among the above categories.
The number of substations, bulk consumers and consumers of a feeder also highlight strong positive linear relationships. In areas like Dehiwala and Rathmalana, both the consumer density and the power demand are high and hence the substations are situated closer to each other. The bulk consumers is also situated closer to each other when compared to places like Agalawatta and Kalutara areas (where the consumer density is lower and the bulk consumer density is also lower). Feeder off duration due to auto trippings and Feeder off duration due to auto trippings > 5 min display strong positive linear relationships. This is because, for transient faults and momentary faults the feeder outage durations are very less compared to other faults. It is important to consider the results of Correlation analysis when selecting Inputs and outputs for DEA analysis.

Input and Output Variables used in Literature
The feeders are expected to supply the demanded power and also to maintain the network by minimizing the feeder tripping, power losses, etc. Input-oriented model is required for this kind of study because good quantum of output is produced by consuming the smallest While computing the relative efficiency scores of the feeders, the items that are to be controlled should to be added as the inputs. Under normal conditions, the manual trippings are taken for load transferring situations and line maintenance work. In this kind of situation, power for the consumers in the feeder is supplied by extending other feeders. Any power interruption is informed well ahead to those consumers. But the information of the partly interrupted section is difficult to calculate as substantial information is not available. Manual trippings are done to improve the condition of a feeder. Hence the feeder off duration due to manual trippings is not considered as an input to the model. When a feeder is switched off all the consumers of a feeder are affected. Therefore the number of trippings is to be controlled. The number of feeder trippings is a major parameter while deciding the condition of a feeder. Therefore it is better if it can divide feeder trippings into few categories. But the amount of variables cannot be increase as per choice, because if the number of inputs to the model increases then more number of feeders tends to obtain relative efficiency score of 1. Therefore the number of feeder tripping is only divided into number of auto trippings and manual trippings. Therefore the number of feeder trippings is divided into two categories namely the number of auto trippings and number of manual trippings. Table 4 shows the input and output variables that are used for the study.

Selection of Inputs and Outputs for the Base Model
DEA analysis relies heavily on the initial choice of inputs and outputs. The efficiency scores tend to be sensitive to the choice of input and output variables and in some circumstances inappropriate choices may lead to inaccurate relative efficiency scores. To select the suitable base model, DEA analysis is carried out for several models in order to analyse the variation of the results for different input and output combinations.

Evaluation with Peak Power Loss and Maximum Voltage Drop
Even though the maximum voltages drop percentage of a feeder and peak power loss are major parameter when deciding a condition of a feeder, according to the correlation analysis maximum voltage drop percentage and peak power loss showcase strong positive linear relationship (correlation index of 0.913). Therefore only one parameter is selected for the model. A DEA Model is evaluated with both peak power loss and maximum voltage drop percentage in the model. Another model number is evaluated with peak power loss and without maximum voltage drop percentage. Another model is evaluated without peak power loss and with maximum voltage drop percentage. According to the results, some feeders show significant variations in the relative efficiency scores in the above evaluated models. Therefore both variables are significant variables and both the peak power loss and maximum voltage drop percentage are taken to the base model.

Evaluation with Reliability Indices
In international benchmarking practices, the use of reliability indexes as a variable is rare, but at present the reliability indexes (SAIDI & SAIFI) are the most widely used variable to measure the performance of power distribution sector. But when a feeder is considered, the SAIDI value is equal to the summation of feeder off duration due to auto trippings and manual trippings. The number of auto trippings and manual trippings is equal to SAIFI value of a feeder.
A separate study is done by using reliability indices. Therefore the only difference that occurs between the model with reliability indices compared to the model with feeder of durations and feeder trippings is that the feeder trippings are taken as two variables (manual trippings and auto trippings) in the first model while the number of feeder trippings are taken as only one variable in the second model. In both the models, some feeders show significant differences in their relative efficiency scores. In the normal model, the data is observed in detail especially when obtaining the targets to be achieved to become relatively efficient and when doing sensitivity analysis it is more advisable to have more information. Therefore the normal model is more suitable than the model with reliability indices.

Study Done to Select Suitable Output
Even though few items are utilized beyond our control as outputs, the selection of output is to be done carefully. The relative efficiency scores obtained from the evaluation needs to be justified. Therefore to select the most accurate output, various combinations of feasible outputs are studied. It is observed that the relative efficiency score changes considerably with the selected outputs. Those evaluated outputs are uncontrollable variable. Also changing the relative efficiency score regard to the number of substations of the feeder, Number of consumers of the feeder, Maximum demand of the feeder and Energy supplied by the feeder cannot be justified. Other than that deciding the relative efficiency score based on Number of Sub stations of the feeder, Number of consumers of the feeder, Maximum demand of the feeder and Energy supplied by the feeder are cannot be verified. Therefore from feasible outputs only feeder length is taken as output for the evaluation.

Analysis with Base Model and With Exclusion of One Variable at a Time
To check the suitability of selected inputs relative efficiency scores were checked with different models upon exclusion of one variable at a time. In all the models relative efficiency scores changes significantly with regard to selected base model. Therefore all the selected input variables for the base model are significant.

Justification of the Selected Base Model
The selected base model for the analysis is justified from the results of the analysis done after running the different DEA models. Table 5 depicts the justification of the selected base model.

DEA Analysis 5.2.1. Relative Efficiency Score
The DEA model was solved using DEA Frontier Free Software. The relative efficiency scores for the Input Oriented model is obtained and tabulated as in Table 6. It is witnessed that out of 32 feeders (seventeen feeders in urban category and fifteen feeders in rural category), 14 feeders (eight feeders in urban category and six feeders in rural category) possess the ISSN: 1693-6930  efficiency score of 1.0. The Mathugama Feeder 8 has the lowest efficiency score in rural category and Dehiwala Feeder 7 has the lowest efficiency score in urban category. Figure 2 depicts the Relative Efficiency Score obtained by each DMU. All the inefficient feeders are given some targets to be achieved to become relatively efficient. Efficient Input Target assigned by DEA analysis for feeders are shown in Table 7. These values can be used to get an idea regarding how the relatively inefficient feeders needed to be improved. In-depth analysis needed to be done observing the practical constraints, field issues and the efficiency targets assigned by the study.  Figure 2. Relative efficiency score plot

Sensitivity Analysis
In order to check the stability of the efficiency scores obtained from DEA analysis according to the variations in inputs and outputs, it is required to carry out a sensitivity analysis. Here one input variable or output variable is removed from the base model at a time and the DEA analysis is carried out to find the efficiency score. Then the obtained efficiency score is compared with the base model efficiency scores. When carrying out the sensitivity analysis it is noted that the efficiency scores of the feeders never increase upon removal of input and output variables from the model. The result from the sensitivity analysis is used as a base for the classification of feeders. Considering the pattern obtained from the graph of efficiency variation with different models upon removal of variables at a time the feeders are classified in to five categories. a. Robustly efficient: DEA efficiency score stays at one or decrease very slightly when the variables are removed from the model one at a time. b. Marginally efficient: Efficiency score is 01 for the base model and remains at 01 in some situations, but drops significantly in other situations. c. Marginally inefficient: DEA efficiency score is below 1 but above 0.9 for the base model and stays in that range during the sensitivity analysis d. Significantly inefficient: DEA efficiency score is below 1 but above 0.9 and drops to much lower values during the sensitivity analysis. e. Distinctly inefficient: DEA efficiency is significantly low (below 0.9) in all the situations.
According to the study nine feeders are robustly efficient, five feeders are marginally efficient, two feeders are significantly inefficient and sixteen feeders are distinctly inefficient. Table 8 shows the summery of feeders categorized in the sensitivity analysis.  50  22  24  3  0  Rath F7  5  30  1  16  0  Rath F8  7  22  3  7  0  Rath F9  14  60  4  25  1  Pana F1  9  26  17  14  0  Pana F2  4  60  14  57  1  Pana F3  6  14  5  2  0  Pana F4  94  323  92  45  1  Pana F5  131  390  37  62  1  Matu F1  27  157  30  77  1  Matu F2  11  609  230  215  5  Matu F3  19  219  123  4  1  Matu F4  16  24  27  0  0  Matu F5  36  681  127  42  3  Matu F6  19  177  63  14  1  Matu F7  13  299  60  296  5  Matu F8  4  63  12  59  1  Matu F9  24  176  30  129  2  Matu F10  6  26  13 18 0 Sensitivity based classification is important when improving the performance or increasing the efficiency scores of the feeders. That is for a particular unit or to an organization it is essential to know its strength and weaknesses in order to achieve their targets. In distinctly inefficient Feeders, the efficiency score is below 0.9 for all the cases including base model. That kind of feeder needs special attention to improve their performance. In-depth studies are to be carried out about these feeders and methods are to be identified to improve the performance of these feeders. Existing limited resources are to be focused mainly to these distinctly inefficient feeders and give priority for these feeders in implementing system development work. Significantly inefficient feeders require detailed studies to find methods to improve its performance. These feeders are given good attention in implementing system development work. At the same time marginally efficient feeders are very sensitive to changes in some variables only. Therefore it is important to identify important variables for these kinds of feeders and prevent them from becoming inefficient. Studies are also done to identify and improve the sensitive variables. Robustly efficient feeders perform well in comparison with all the available feeders. These feeders does not demand close monitoring. It is more beneficial to do system development work to other categories of feeders than this category. Steps shown in Figure 3 can be used as a guide line/methodology for benchmark medium voltage feeders in Sri Lanka. Similar type of model can be developed to Benchmark 132/ 220 kV Transmission Lines of the country.

Conclusion and Recommendations
In this paper, the thirty two medium voltage feeders of Western Province South-I of CEB is evaluated. The feeders are categorized into two categories as urban and rural based on the homogeneity of the feeders. After the evaluation, the feeders are categorized into five categories called robustly efficient, marginally efficient, marginally inefficient, significantly inefficient and distinctly inefficient. Finally it is recommended to do in-depth analysis for inefficient feeders considering the practical constraints, field issues and the efficient targets assigned by the study to improve their relative performances. Further, the same study is recommended for all distribution provinces in Sri Lanka annually or biennially (Parallel to Medium voltage Distribution Development Plan).