Factor Structure of Work Practices and Outcomes 1
Paper Under Peer Review
Please do not publish or distribute widely without approval from the author.
Psychometrics of the Voice Climate Survey: Evidence for a lower and higher-order factor structure of work practices and outcomes
Peter H. Langford
Voice Project, Department of Psychology, Macquarie University
Author’s Note:
Correspondence should be directed to Peter Langford, Voice Project, Department of Psychology, Macquarie University, NSW, 2109, Australia; email: ; phone: +61 2 9850 8020. The Voice Climate Survey presented in this paper is copyrighted by Access Macquarie Limited, the commercial arm of Macquarie University. University researchers involved in non-profit research can use the tool without seeking permission from the author. For all other enquiries or to have data benchmarked against the existing database contact the author.
Keywords: Employee opinion survey, job satisfaction, organizational commitment, turnover, work practices.
Psychometrics of the Voice Climate Survey: Evidence for a lower and higher-order factor structure of work practices and outcomes
Abstract
This study presents evidence supporting the psychometric properties of the Voice Climate Survey – an employee opinion survey that measures work practices and outcomes. The tool is tested across 13,729 employees from 1,279 business representing approximately 1,000 organizations. An exploratory factor analysis, a confirmatory factor analysis and internal reliability analyses support 31 lower-order practices and outcomes that aggregate into seven higher-order work systems labeled as Purpose (including lower-order factors such as direction, ethics and role clarity), Property (including resources, facilities and technology), Participation (including employee involvement, recognition and development), People (teamwork, talent, motivation and initiative), Peace (wellness and work/life balance), Progress (achieving objectives, successful change and innovation, and satisfied customers) and Passion (representing the construct of employee engagement currently popular among practitioners, incorporating subscales of organization commitment, job satisfaction and intention to stay). External validation of the tool is demonstrated by linking scores from the employee survey with independent manager reports of turnover, absenteeism, productivity, health and safety, goal attainment, financial performance, change management, innovation and customer satisfaction.
Psychometrics of the Voice Climate Survey: Evidence for a lower and higher-order factor structure of work practices and outcomes
This article presents psychometric support for an employee opinion survey designed to give researchers and practitioners a robust and efficient measure of a wide range of work practices and outcomes.
Employee surveys are one of the most common forms of data collection used by researchers and practitioners. Such surveys are used widely for describing the nature of an organization, assessing how well an organization is performing, benchmarking organizational performance against other organizations, and estimating the potential causal relationships between work practices and outcomes (Kraut, 2006).
Among both researchers and practitioners, employee surveys are being used increasingly to simultaneously measure a broad range of work outcomes (such as job satisfaction or the now popular construct of employee engagement) as well as a multitude of potential determinants of those outcomes. It is hoped that such a multi-dimensional approach will provide insight into a hierarchy of relative importance of work practices, enabling organizations to better allocate resources to development initiatives that will in turn maximize desired work outcomes. Unfortunately, there is a lack of published and psychometrically robust multi-dimensional employee surveys. This paper reviews the recent development of research using multi-dimensional employee surveys, and presents a survey with strong psychometric support and a practically useful and theoretically innovative factor structure.
There are several ongoing debates and directions in the literature regarding the concept of organizational climate and its measurement. Some of the most theoretically significant topics involve (1) the relationship between climate and culture, (2) the distinction between psychological versus organizational climate, (3) the argument for and against general measures of climate, and (4) the need for a consolidation of the vast range of climate dimensions that have been studied. These issues will now be discussed.
Climate and Culture
There are two main models describing the relationship between climate and culture. The first and older model sees climate and culture as hierarchically equivalent and distinct. This tradition separates climate (employees’ evaluation of their work environment including structures, processes and events) from culture (a more subjective description of the fundamental values of an organization; Denison, 1996; Meyerson, 1991; Schnieder & Snyder, 1975). This separation reflects the differing historical development of these constructs, with climate developed largely by organizational psychologists and culture developed through anthropology and sociology.
Increasingly though, with the advent of management science as a new and separate research domain, culture is being seen as an overarching construct, within which climate is a subset. Researchers such as Hofstede (2003), Schein (2004) and Rousseau (1990) have described culture has having different levels. While the number of levels varies (Hofstede has two, Schein has three and Rousseau has five) they can all be broadly seen as differentiating values and practices (which are indeed Hofstede’s two levels). Values, in this sense, are seen as fundamental, often unconscious, ways of understanding and evaluating the world. Practices, in turn, are seen as the tangible and observable behaviours and practices. In this model, climate can be equated with the measurement of employee’s description and evaluation of workplace practices. As such, climate is treated as a subset of culture, in the same way that values are regarded as a subset of culture.
This paper follows this more recent direction, equating climate with practices which are a subset of culture. The Voice Climate Survey described here focuses on measuring organizational practices rather than values.
Psychological Versus Organizational Climate
Although not an active ongoing topic, there has historically been a concern among some researchers regarding the existence of climate above the level of an individual. Authors such as James and Jones (1974) differentiated psychological climate (an individual’s perceptions) with organizational climate (measured by aggregating many individuals’ perceptions), and argued that an organizational climate should perhaps only be regarded to exist if the variance between the many psychological climates was low. If the variance between psychological climates was high then perhaps no single organizational climate should be thought to exist.
While acknowledging the theoretical contribution of this debate between psychological and organizational climate, the current paper assumes that the aggregation of an organizational climate has value, albeit perhaps having greatest value when the aggregated scores are interpreted alongside measures of variance within the organizational climate. Researchers find value in comparing aggregated organizational climates across different organizations, industries and countries. Similarly, practitioners are interested in measuring and improving aggregated organizational climate, regardless of the variance within the organizational climate.
General Versus Domain-Specific Climate
Schenider has been a leading critic of the generalized approach to measuring climate (1975, 2000). He has argued that the dimensions and content of climate measures should differ depending upon the organizational outcome that is of greatest interest. If the researcher is interested in improving safety, the measure of organizational climate should differ to one used if a researcher was interested in improving customer service.
The current paper acknowledges that a higher variance of an outcome variable is likely to be predicted by measuring key determinants of that outcome in greater detail, rather than measuring a wide range of practices, some of which may not be of predictive value, and all of which would need to be measured in less detail if the researcher is to keep the same length of survey.
Nevertheless, there remains substantial theoretical and practical value in general measures of climate, in much the same way there is value in general measures of personality. First, limiting the range of dimensions being measured prevents an easy comparison of the relative importance of each practice. If a researcher or practitioner is unsure which dimensions of climate are most strongly associated with a specific outcome, limiting the range may exclude important predictors. Huselid (1995) introduced the term high performance work practices in an attempt to direct research towards examining which of the extremely broad range of possible work practices best predict organizational outcomes. Work practices such as recruitment and selection, training and development, performance management and appraisal, compensation and benefits, career development, teamwork, customer orientation, occupational health and safety, to name only a few, have been consistently linked to various measures of organizational effectiveness (e.g., Patterson, West, Lawthorn & Nickell, 1998; Paul & Anantharaman, 2003; Pfeffer, 1994; 1998; Von Glinow, Drost & Teagarden, 2002).
The vast majority of past research, however, has examined such practices in isolation, which hinders our understanding of the relative efficacy of these management practices. Hence, there is a growing interest among researchers in studying a broad range of practices within a single study to enable direct comparisons of effect sizes (e.g., Huselid, 1995; Pfeffer, 1998). Practitioners are also interested in being able to compare the performance and potential importance of the many management practices undertaken within their organizations.
A second benefit of general measures of climate is that researchers and practitioners are often unsure which organizational outcomes they wish to improve, or they may wish to study simultaneously two or more outcomes. For example, in a particular organization or industry, it may be unclear whether stress or satisfaction is in greater need of improving, or indeed the researcher may be interested in estimating the strongest predictors of both stress and satisfaction. In such circumstances, it is necessary to sacrifice the specificity and depth of a survey in order to increase the breadth of dimensions being measured.
Finally, a third benefit of general measures of climate is that they facilitate comparisons across organizations and studies. Researchers like being able to conduct meta-analyses and practitioners like to be able to benchmark results across companies and industries. To be able to make such comparisons with confidence, measures need to be sufficiently general in focus to be of value in a variety of work environments.
In all of these circumstances, a general measure of organizational climate meets the needs of both researchers and practitioners. In the same way that a general measure of personality can be of greater value for some purposes, a general measure of organizational climate can at times be superior to a specific measure. It was for such purposes that the Voice Climate Survey was developed.
Work Systems
Given the wide range of work practices that have been identified and studied, there is also a growing call for the investigation of a smaller set of higher-order categories that can be used to group work practices and enable comparison across studies (e.g., Huselid, 1995; Niehaus & Swiercz, 1996; Tomer, 2001; Pfeffer, 1998). Following Huselid, the current paper uses the term systems to refer to the grouping of work practices and outcomes. In a meta-analysis of measures of organizational climate, Parker et al. (2003, p. 389) stated there is a need “to find a means of categorizing the enormous number of psychological climate scales into a logical set of core categories”. Similarly, van den Berg and Wilderom (2004, p. 573), in a recent review of the climate and culture literature, argued that “convergence on the [higher-order] dimensions is very much needed and may stimulate research, as is the case in the development of the Big Five personality traits”. Identifying a simpler, higher-order set of systems may help integrate existing research and provide a language and structure to coordinate future research into management practices, in much the same way that the Big Five personality characteristics provoked substantial development of the study of individual differences.
To-date, unfortunately, this research has been largely unsuccessful. Huselid’s pioneering studies (Delaney & Huselid, 1996; Huselid, 1995) found modest support for two systems. The first comprised the following diverse range of practices suggested to contribute to employee skills: information sharing, job design, training, work/life quality, enhanced selectivity, quality circles, labour-management teams, grievance procedures, and incentive compensation. The second system comprised the following practices suggested to contribute to employee motivation: performance appraisal, meritocracy, and recruitment. However, the face validity for these two distinct systems is unclear (some of the skills-focused practices could easily be argued to be motivation-focused practices, and visa versa) and indeed Huselid emphasized that this structure was a preliminary model developed with exploratory measures.
Another prominent researcher in the area of high performance work systems is Guest (1997, 2004). Along with other writers, and building upon the systems suggested by Huselid, Guest (1997) suggested that work practices may fall into the three higher-order systems associated with employee skill, motivation and opportunity to contribute. While conceptually elegant, this division of systems was only loosely based on empirical support and has not yet been validated. In a recent article Guest (2004) used sequential tree analysis and factor analysis and found support for a single system which included practices such as job design, appraisal, teamworking and employee involvement.
Other researchers have also presented empirical support for a single system. Investigating production systems in the automotive industry, MacDuffie (1995) found that all measured management practices clustered onto a single factor. More recently Den Hartog and Verberg (2004), examining work practices across multiple industries in the Netherlands, found a single high performance work system consisting of a combination of practices with an emphasis on employee development, strict selection and providing an overarching goal or direction. Unfortunately, the finding of a single system does not contribute to the previously discussed call for a small number of multiple systems to simplify theory and improve comparisons across studies.
Recently, Patterson et al. (2005) published a proprietary tool they called the Organizational Climate Measure (OCM). The OCM demonstrated sound psychometric qualities for 17 lower-order work practices. While hypothesizing a higher-order factor structure mirroring the four factors of the Competing Values Framework (including factors for human relations, internal processes, open systems and rational goal; Quinn & Rohrbaugh, 1983), Patterson et al. found only weak empirical support for such a higher-order structure. A possible explanation for their limited success may have been their use of a model of values to develop a measure of climate. If, as discussed earlier, practices and values are separate subsets of culture, and if climate can be equated to the measurement of practices, it is perhaps not surprising in hindsight that a factor structure for values may not map neatly on a factor structure for climate.