An Emprical Evaluation of Problems

AN EMPRICAL EVALUATION OF PROBLEMS

IN PERFORMANCE MEASUREMENT SYSTEMS

OF BIG SIZED FIRMS IN TURKEY

(002-0428)

Second World Conference on POM and 15th Annual POM Conference,

Cancun, Mexico,

April 30 - May 3, 2004.

Hilmi YÜKSEL, Phd,

University of Dokuz Eylül,

Department of Business Administration

İZMİR/TURKEY

Keywords: Performance Management, Performance Measurement Systems

Abstract The design of performance measurement systems has received considerable attention in recent years. However, many firms still clarify that they meet many problems in their performance measurement systems. These problems may be sourced from many reasons and for designing performance measurement systems effectively, these reasons must be identified clearly.

This paper investigates the problems that big sized firms mostly meet in their performance measurement systems. For this aim, a questionnaire has been designed and mailed to the big sized firms in Turkey. In the questionnaire, some questions are related about the characteristics of the performance measurement systems of the firms and some questions are related about the problems, firms may meet in their performance measurement systems. According to the results of the analysis, the problems that big sized firms meet in their performance measurement systems has been identified and relationships between the problems that firms mostly meet and the characteristics of the performance measurement systems have been determined. After determining the problems that firms mostly meet, the reasons of these problems have been evaluated. This paper also aims to emphasize that in today’s condition, the traditional performance measures are often inappropriate and insufficient to evaluate the performance of the firms. This claim has been tried to be verified by demonstrating the relationships between the problems that firms meet in their performance measurement systems and tendency to use financial and non-financial measures for evaluating the performance of firms.

Introduction

The traditional performance measures, mostly, are based on financial measures. However, recently, the production techniques and philosophies and competitive environment changed significantly. If the environment of the firms was stable and didn’t change so rapidly , then the limitations of the traditional performance measurement systems wouldn’t be so significant.

Many researchers showed that performance measurement systems that base on financial measures, are insufficient to measure and to integrate all factors that are significant for the successes of firms (Suwignjo vd.,2000). In order to eliminate the limitations of the traditional performance measurement systems, new performance measurement frameworks were proposed. These performance measurement frameworks are multi- dimensional and focus non-financial measures in addition to financial measures. The main objective of designing these performance measurement systems is to balance the measures that provide information about the future performance of the firms and to determine external success as well as internal success (Bourne vd, 2000).

However, firms, still meet very different problems in their performance measurement systems and these problems can be sourced from very different reasons. According to Kennerley and Neely (2002) , yet few firms appear to have systematic processes in place for managing the evaluation of their measurement systems. Before 20 -30 years ago, because, the environment of the firms was more stable, providing information about past could be an indication about the future performance of the firms and providing feedback wasn’t so important as today’s. The most problems that firms meet in their performance measurement systems source because of using traditional performance measurement systems that base on financial measurements. Provided that using traditional performance measures, the firms provide information about the past and are not able to provide information as rapid as required. These are only some factors that affect the efficiency of performance measurement systems, there are so many problems that firms meet in their performance measurement systems and these problems and the factors that affect to these problems must be determined for increasing the efficiency of the performance measurement systems.

In order to eliminate the deficiencies of traditional performance measurement systems, firms, firstly, must determine the problems of their performance measurement systems. After determining these problems, they must redesign their performance measurement systems if necessary. Without an efficient performance measurement system that is redesigned according to today’s requirements, the performance of the firms can’t be evaluated exactly, and for the firms to be successful in the competitive environment will be so difficult.

Performance Measurement Systems

Neely et al. (1995; 80) stated that “performance measurement is a process of quantifying the efficiency and effectiveness of action, performance measure is a metric used to quantify the efficiency and/or effectiveness of an action and performance measurement system is the set of metrics to quantify both the efficiency and effectiveness of actions.” According to this definitions it can be stated that one of the main objective of the performance measurement is to help the firms determine their performance problems and focus to the areas that effect the efficiency and effectiveness of the firms.

At the base of the performance measurement system, the information system of the firms stays. This information system must be a performance measurement system that integrates the information among the systems. So that, performance measurement process can be considered as a closed loop control system that spreads the strategies and objectives and provides feedback from different areas (Bittici et al ,1997).

The development of performance measurement system can be divided into three main phrases. These phases are designing performance measures, implementing the performance measures and using performance measures. The design phase can be subdivided into identifying the key objective to be measured and designing the measures themselves (Bourne et al.,2000; 757). In the design of the performance measurement system; why do we want to measure and what do we want to measure are the main questions that must be answered. The reasons for measuring the performance can be stated as where have we been?, where are we now?, where do we want to go?, how are we going to get there? and how will we know that we got there ? (Lebas ,1995; 24). According to Arthley and Stroh (2000); a performance measurement that provides information about how we are good about our activities, if we are able to achieve our objectives, if the customers are satisfied, if the processes are in control and in which areas we need improvements can be stated as efficient. An efficient performance measurement system must have the ability to provide information about the activities and the activities that must be improved rapidly.

Provided that the performance measurement system shares input and produces output for the other systems, it mustn’t be considered as a separate system. As Bititchi et al (1997); clarified performance measurement system has got an important role in order to spread the objectives of the firms and to encourage for integrating different areas in firms.

Ghayalini and Noble (1996; 63) stated that the literature concerning performance measurement has had two main phases. The first phase began in the late 1880s and went through the 1980s. In this phase, the firms focused to the financial measures to control, monitor and improve their activities. However, in today’s competitive conditions and production management techniques and philosophies, traditional performance measurement systems that base on financial measures are not sufficient and appropriate to evaluate the performance of the firms. Neely et al. (1995) stated that lean production is imposing changes on the performance measurement systems too. With the changes in the production techniques quality, flexible and lead time became as important as cost (Ghalayini and Noble,1996, 63, Ghalayini et al., 1997, 208, Bititci et al., 2001). Time based competition strategies requires performance measures that do not emphasize individual operation time standards but instead stress reduction of the set up times, the flexibility of the workforce and the capability of producing high quality products by a specified completion time (Toni et al., 1997; 180).

In the literature, there are many researches that emphasize the deficiencies of traditional performance measurement systems according to many dimensions and discuss the differences between traditional performance measurement systems and non-traditional performance measurement systems (Ghalayini and Noble, 1996, Ghalayini et al.,1997, Toni et al, 1997, Manoochehri, 1999, Bititci et al. , 2001, Medori and Steple, 2000).

Over recent years many researches have been done by practitioners and researchers to develop new performance measurement systems that try to take account financial and non-financial measures in balance (Bititchi et al. 2001). In the late 1980s and early 1990s, the dissatisfaction about the traditional performance measurement systems lead to the development of “balanced” or “ multi-dimensional” performance measurement frameworks. (Bourne et. al ,2000;754-755). The balanced scorecard (Kaplan and Norton,1992), the performance prism (Kennerly and Neely, 2000), the performance measurement matrix (Keegan et al, 1989), the results and determinants framework (Fitzgerald et al. 1991) and the SMART pyramid (Lynch and Cross, 1991) can be given as examples of these frameworks The objectives of such frameworks is to help firms define a set of measures that reflect their objectives and assess their performance appropriately (Kennerley and Neely; 2002;1224). With proposing new performance measurement frameworks, the question of which measures must be used by the firms has been tried to be answered. However, these performance measurement systems haven’t been able to present a proposal (Bourne et. al, 2000). It mustn’t be overlooked that the integrated performance measurement system, proposed in order to evaluate the performance of the firms and to prevent local optimizations, have got disadvantages in addition to important advantages (Ghalayini and Noble, 1996;77).

In addition to these Btitichi et. al (1997) proposed an integrated performance measurement systems reference model and Ghayalini et. al (1997) proposed an integrated dynamic performance measurement. In the performance measurement system that Ghayalini et. al (1997) proposed, the performance measures at all the levels can be integrated and success areas and related performance measures can be updated.

Non-Traditional Performance Measures Versus Traditional Performance Measures

Traditional performance measures have many limitations that make them less applicable in today’s competitive market. The most signification limitation of traditional performance measures is that they are based on management accounting systems which focus on controlling and reducing direct labor costs (Ghayalini et al, 1997). In addition to this, traditional performance measures are based on outdated traditional cost management systems, lagging metrics not related to corporate strategy, inflexible, contradict continuous improvement, a predetermined format and not applicable to the new management techniques (Ghayalini and Noble, 1996;77). In today’s, providing feedback rapidly is a significant factor for the firms to enhance competitive advantage. However, traditional performance measurement has got limitations in this point because of providing information about the past (Ghalayini, Noble, 1996).

As stated, in today’s conditions, firms must evaluate their performance in the terms of satisfaction of customer, quality, flexible and innovation. However, traditional performance measurement systems are not able to measure to these strategic objectives (Manoochehri, 1999). In addition to these limitations of traditional performance measurement systems, it must be considered that, shortening the time of supply chain and adapting to the production planning have got important roles in the success of the firms but these success factors can not be easily evaluated with financial measures (Ghalayini et al . 1997). In spite of these limitations of traditional performance measurement, one of the important disadvantage of non-traditional performance measurement is that there are so many non- financial measures that can be used in firms and so that the firms may meet with difficulties while determining non-financial measures (Medori and Steple, 2000).

Critical Factors For Determining The Measures

Determining inappropriate measures may let the firms focus to unnecessary activities and determine the priorities wrongly. Focusing to inappropriate activities may prevent firms to maintain required improvements in firms. Gunesakaran et. al (2001;72) stated that the firms that use fewer measures can evaluate their performance better. Using so many measures may prevent the firms to focus to the activities that must be improved. However using insufficient measures may cause the firms to provide information that are inaccurate. For example, firms may focus to decrease their costs and for this aim, their performance measurement system basing on financial measures will be sufficient. In this situation, the expectations of the customers may be neglected and satisfaction of the customer may be decreased. Using measures that discords, may also prevent firms to focus to the activities that must be improved. For this reasons, the priorities of the firms must be determined accurately and the performance measures must be determined throughout the strategies of the firms. One performance measure can’t be appropriate for all the departments and all the time. As the environment is very dynamic , the measures can’t be considered as static. Ghayalini et.al (1996) stated that one measure that is significant for the firm today doesn’t mean that it will be significant tomorrow as today. Bititchi et al. (2000; 694) clarified the factors that prevent the performance measurement system of the firms to be dynamic as “lack of a structured framework that allows organizations to differentiate between improvement and control measures, and develop causal relationships between competitive and strategic objectives and processes and activities, absence of a flexible platform to allow organizations to effectively and efficiently manage the dynamics of their performance measurement systems and inability to quantify the relationships between measures within a system”.

The term of integration for performance measurement system refers to the ability of the performance measurement system to promote integration between various areas of the business. Integrated performance measurement systems should enable correct deployment of objectives and should provide a structured framework to allow the relevant information to feedback to the appropriate points (Bittici et al.; 1997; 47).

As literature reviewed, it can be stated that, measures must be determined from the strategies of the firms (Bourne et al., 2000;757). There may be significant differences among the performance measurement systems of the firms. Mainly, these differences are sourced because of the performance measures of the firms relating to the strategies and the strategies are different among the firms (Medori, Steple,2000). In the literature, there are many performance measurement frameworks and different criteria developed for the design of performance measurement. However, a systematic hasn’t been proposed for the performance measurement yet. Different systems requires different measurement system criteria and this prevents to develop an approach appropriate for all the firms (Beamon vd.,1999; 277).

Research objectives and Methodology

The objective of this research is to examine the problems that firms meet in their performance measurement systems and the factors, such as using non-financial measures and measuring all the dimensions of the firms, that affect these problems. A questionnaire was designed based on literature review. Respondents were asked to indicate , using a five-point Likert scale (1= Completely disagree, 2= Rarely Agree, 3= Partly Agree, 4 = Rather Agree, 5= Completely Agree) their evaluations about their performance measurement systems and the problems that they meet in their performance measurement systems. After the questionnaires were sent to 160 firms that were chosen randomly from the 500 big firms in Turkey. Respond rate was resulted as 33,21 percent (53 out of 160), which may be accepted, generally, relatively low, but in the context of Turkey, it may be accepted because of the reluctance of Turkish firms to respond to these kinds of questionnaires.7 firms of respondents indicated that they don’t have got a performance measurement system to participate in the survey and the indications of 3 firms were found as illogical and so 10 questionnaires weren’t included to the analysis of the survey. Consequently, the analysis of the survey were realized with the indications of 43 firms.

The survey consisted of three major sections. The first section gathered information about the performance measurement systems of the respondents. The second section provided information about the problems that firms meet in their performance measurement systems and third section focused the factors that affect the level of meeting with these problems.

An Overview of the Performance Measurements Systems of the Firms

First of all , the respondents were asked to indicate, using five-point Likert scale, the areas that they use their performance measurement. The results of these evaluations were given at figure 1

Figure 1: Uses of the Performance Measurement

According to figure 1, the area that firms mostly use performance measurement is evaluating the employees. The least area that firms use their performance measurement is benchmarking between the firms. This may cause because of the firms being unwilling to share their information. As seen from the figure 1, evaluations of the firms that they use performance measures in order to provide information for the external reporting supports this propose.

In this section, the respondents were asked to indicate the statements about their performance measurement systems. The statements and the mean and standard deviation values were given at table 1.

Table 1: The Evaluations of the Firms About Their Performance Measurement Systems

According to table 1; the respondents stated that they have got an efficient performance measurement system. For this statements, no firms indicated as “completely disagree” and only four firms indicated as “rarely agree”. This result mustn’t be overlooked in the evaluation of the results of the analysis. In addition to these, we can say that the objective of their performance measurement system of respondents is improvement not at all control and their performance measurement system is capable to measure all the dimensions of their firms. The respondents stated that they also use non-financial measures in spite of their performance measurement system base on financial measures of the firms.