Why do Organizations Collaborate?
Empirical Evidence from Chicago Public Schools

Carolyn J. Hill

Assistant Professor

Georgetown Public Policy Institute

Georgetown University

3600 N Street, NW

Washington, DC 20007

Tel: (202) 687-7017

Fax: (202) 687-5544

Laurence E. Lynn, Jr.

George Bush Chair and Professor of Public Affairs

George Bush School of Government and Public Service

Texas A&M University

1064 Academic West Building

4220 TAMU

College Station, TX 77843-4220

Tel: (979) 458-8033

Draft: November 1, 2003

This version is preliminary and incomplete: please do not cite or quote without permission.

Prepared for the 25th annual research conference of the Association for Public Policy Analysis and Management, November 6-8, 2003, Washington DC. We thank the Consortium on Chicago School Research for providing the data used in this paper and for answering our questions about the data; and Constance Lindsay and Celeste Lum for excellent research assistance.

Why do Organizations Collaborate?
Empirical Evidence from Chicago Public Schools

Abstract

Organizations are under increasing pressure to collaborate in order to improve their performance. From each organization’s perspective, however, what motivates decisions to collaborate? Failures of collaboration, which are more common than successes, may reflect failures on the part of sponsors, administrators, and participants to understand the interests that motivate participation and to govern in the light of those interests. In this paper, we provide insight in this issue regarding collaboration between public schools and external organizations. Using rational choice, socialized choice, and resource dependency theories to develop testable hypotheses, we analyze a large sample of public schools in Chicago, and model their decisions to participate in collaborative relationships. We find that characteristics of schools that reflect resource dependency motivations tend to explain participation in collaborative relationships relatively more often than do the variables reflecting the other two types of theories. In particular, resource dependency variables predict schools’ participation in collaborations with foundations, mental health organizations, parks, other schools, and cultural organizations. This preliminary analysis, which is based on data from the 1998-1999 school year, provides a basis for generating hypotheses and refining measures for further analyses of collaboration by Chicago Public Schools using data from additional years.

Why do Organizations Collaborate?
Empirical Evidence from Chicago Public Schools

Collaboration among human service organizations is a popular, seemingly uncontroversial governance concept. Potential beneficiaries of such collaboration include clients and customers, the organizations themselves, sponsors and administrators (Weiss 1981), and also the wider community of citizens and stakeholders. However, collaborative ventures often encounter difficulties that prevent the potential benefits from being realized (Bardach and Lesser 1996; Hassett and Austin 1997; Meyers 1993; USDHHS 1991; USGAO 1992; Weiss 1981). The question is: why?

Failures of collaboration may reflect a lack of understanding or acknowledgement of the interests that motivate organizations to participate in collaborative relationships in the first place. Assuming that all participants in a collaborative have similar motives is an insufficient basis for sustaining collaboration and realizing the potential benefits to the many actors involved. A number of theoretical lenses provide insight into service organizations’ incentives to collaborate with other organizations, and various researchers have reviewed these theories and some of their implications (Hill and Lynn 2003; Meyers 1993; Reitan 1998). These theoretical perspectives can be used to conceptualize and operationalize aspects of organizations that are associated with motivations to collaborate with other organizations. The understanding of organizations’ propensity to collaborate is crucial for designing governance mechanisms that promote and sustain productive collaboration.

To the best of our knowledge, few studies have put these kinds of questions to an empirical test using a large sample of organizations. In the current study, we address this gap in a particular setting: the Chicago, Illinois public school system for the 1998-99 school year, where we can take advantage of the naturally-occurring variation in school collaborative activity. Surveys of principals from a sample of 297 schools provide information about characteristics of the schools and collaborations with external organizations. Because collaborating is voluntary (and not mandated by some external governing authority), the teachers, principals, and schools that collaborate with external organizations may be different in both observable and unobservable ways from those that do not collaborate. The large sample of schools, and information from the principal surveys and administrative data, allow us to control for many (but not all)[1] factors that are likely to affect the collaboration decision: e.g., characteristics of students, schools, and community.

Our preliminary findings indicate that characteristics of schools that reflect resource dependence motivations explain participation in collaborative relationships relatively more often than do the variables reflecting rational choice or socialized choice theories. In particular, resource dependency variables predict school participation in foundations, mental health organizations, parks, other schools, and cultural organizations. This preliminary analysis, based on data from the 1998-1999 school year, provides a basis for generating hypotheses and refining measures for further analyses of collaboration in Chicago Public Schools using data from additional years.

The paper proceeds as follows: first, we discuss conceptual and theoretical explanations for organizational participation in collaborations. Next, we review types of collaborative relationships found within public school systems in the U.S., and provide background on the Chicago Public Schools. We describe the data and measures used in this study, and the hypotheses regarding factors associated with collaboration. Next, we describe our methods and findings. The final section concludes and discusses future plans for this research.

WHY DO ORGANIZATIONS COLLABORATE?

Conceptual and Theoretical Bases

We define collaboration as voluntary participation in interorganizational relationships that involve agreements or understandings concerning the allocation of responsibilities and rewards among the collaborators.[2] Each potential collaborator must decide how to allocate its scarce resources toward organizational goal attainment. Specifically, the organization can allocate effort toward an independent product or service (i.e., the production influence of other organizations is limited to arm’s length exchange transactions); or to a collaborative product (i.e., the provider’s productivity is affected by the efforts of other providers); or to both.[3]

In an earlier paper, we reviewed primary classes of theories that can be helpful in analyzing the decision to collaborate (Hill and Lynn 2003). Rational choice theories are concerned with exchanges (e.g., of service for payment) or with other interactions (e.g., providing or withholding information) between autonomous actors seeking to attain pre-existing organizational goals. Theories in this class include transaction cost theory, principal-agent theory, the theory of teams, game theory, and collective action theory.

Socialized choice theoriesare concerned with relationships other than exchange relationships that might further shared values. In contrast to rational choice theories, socialized choice theories tend to view the production strategies of human service providers as partially or wholly endogenous with respect to socially constructed patterns of interaction. Theories in this class include organization theory (which involves aspects of embeddedness, power, culture, leadership, and an emphasis on primary work), institutional theory, structuration theory, and network theory.

Resource dependence theory (Aldrich and Pfeffer 1976) shares aspects of both rational and socialized choices theories. This theory holds that organizations interact with their environments and respond to available opportunities and constraints, but they are not completely determined by such external forces (Aldrich and Pfeffer 1976). Survival is an overarching goal (reflecting a rational goal-based perspective), but “not all internal decisions are relevant to survival, and thus not all are affected by the environment” (Aldrich and Pfeffer 1976, p. 84). Governance mechanisms might encourage either independent production (e.g., by manipulating funding streams or conditions to require segmentation of services within an organization) or collaborative production (e.g., by offering support for “innovative” projects that require cooperation and coordination) (Kramer and Grossman 1987).

Specific theories in these broad classes focus on different factors that are often assumed to influence an organization’s incentives to collaborate. Thus, they may have different implications for the initiation and management of collaborations. Our strategy in this paper is to develop measures, and derive and test hypotheses, that are consistent with these major classes of theories. In principle it would be preferable to develop and test full specifications for each component theory. The data we use are not adequate for this strategy, however. Instead, we construct measures and specify models that test aspects of the major theoretical classes that yield unique predictions concerning motivations to collaborate. With this strategy, the predictive value of each class of theory can be isolated. Findings to the effect that the different classes of theories have different explanatory utility have significant implications for promoting effective collaboration; getting the theory wrong can doom a collaboration that might otherwise be successful.

Collaboration in Public Organizations

Research interest in interorganizational relations has been a topic of research for a number of years across a range of disciplines and substantive fields (Agranoff 1991; Galaskiewicz 1985; Powell 1990; Scott 1998; Whetten 1981). Further, as noted earlier, a rich theoretical and conceptual base exists for thinking about why an organization might collaborate with an external organization. Surprisingly little empirical evidence exists, however, regarding why particular organizations choose to collaborate.[4]

Most studies that do examine such questions for public sector organizations analyze a relatively small number of organizations or networks (e.g., Weiss 1987; Linden 2002; Milward and Provan 1994; Provan, Milward, Isett 2002); or collect and interpret information from interviews (e.g., Linden 2002). Other studies that examine networks or interorganizational relationships in the public sector tend to take collaboration as given, and do not model the collaboration decision itself (Meier & O’Toole 2001). Galaskiewicz and Shatin (1981) studied determinants of collaboration in 181 human service organizations, however they focused on leadership and personal relationships of managers with potential collaborators and how these interact with the turbulence of their environment.

In the current paper, we attempt to address a gap in this literature by developing hypotheses regarding different possible motivations to collaborate, and test these hypotheses using a large sample of public schools in Chicago.

Collaboration in Public Schools

Collaborative relationships in the realm of K-12 schools are encouraged between schools and parents, public and private organizations, businesses, universities, and social service agencies in the community (NCREL 1996). These efforts are supported by school leadership and research organizations (e.g., Council of Great City Schools, and regional educational laboratories supported by the U.S. Department of Education), which establish task forces on, issue reports on, and encourage school-community collaborations. For example, a 1998 report from the Council of Chief State School Officers (CCSSO) described the potential value of collaborations with external organizations, especially for schools in low-income areas:

Through school-community partnerships and school-linked services efforts, educators have found ways to connect with and integrate services and supports that help low-income children achieve academic success and develop into independent, educated, self-sufficient adults. These partnerships have become much more widespread as schools have sought out allies to provide additional support for students and their families to improve education-related results. Schools also have increasingly become more active partners in efforts to revitalize low-income communities and neighborhoods (CCSSO 1998, p. 3).

Examples abound of consortia and initiatives to promote collaboration among schools and organizations in their communities. These initiatives often have a number of goals, such as improving educational outcomes, improving the efficiency and effectiveness of health and social services, addressing a broader spectrum of the developmental needs of youth, and building the human, social, and economic capital of communities (Melaville and Blank 2000).

Unlike many areas of social service, however, where continuity of care, wrap-around concepts, and multi-problem clients provide prima facie arguments for organizational interdependence, the interdependence of individual schools with other local organizations is not strongly supported by either theory or practice. Rather, individual schools typically are viewed as largely autonomous within a hierarchical framework of policy direction. For this reason, public school collaborations offer a particularly interesting opportunity to investigate the various motivations to collaborate, since inclinations to do so cannot be taken for granted.

The Chicago Public Schools

A large Midwest urban school district, the Chicago Public Schools (ChiPS) in 2002 comprised 493 elementary schools (grades K-eight), 95 secondary schools, 14 charter schools (nine elementary and five secondary), and 30 alternative programs (for disruptive students and dropouts). Over 315,000 students were enrolled in K-8 schools, and over 100,000 students in secondary schools. Over half the students are African-American, 36 percent are Latino, and about nine percent are white. Eighty-five percent of all students enrolled in ChiPS come from low-income families, and 14 percent are limited English-proficient (ChiPS 2003).

The ChiPS implemented a number of major reforms during the late 1980s and 1990s, which included:

The School Reform Act of 1988, an Illinois law that abolished job tenure for Chicago school principals and made them accountable to newly-created Local School Councils (LSCs) that award four-year performance contracts. Additionally, the LSCs play a key role in setting priorities and budgets for each school.

The School Reform Act of 1995,an Illinois law that essentially granted the Mayor of Chicago control over the city’s school system, including the power to name school board members (including the chair), and a Chief Executive Officer. The law also ended civil service and union protections for non-teaching employees; and overhauled the school district’s financial system (mostly granting the mayor more discretion over the use of funds).

The 1996 reforms, which ushered in a number of curriculum and support reforms. A major element of these reforms was a policy that required students in third, sixth, and eighth grades to pass a standardized test in order to be promoted to the next grade. If they failed to do so, they were required to attend summer school, then re-take the test. If they passed, they were promoted to the next grade; if they didn’t pass, they were retained in the grade for another year, promoted regardless of their performance, or sent to transitional programs. Furthermore, schools were held accountable for student performance: the school was placed on probation if fewer than 15 percent of its students met national norms on the reading test; and faced reconstitution if performance did not improve.

Hypotheses

Schools are typically regarded as either under hierarchical control of district or state officials, or alternatively, as loosely-coupled systems that foster local autonomy (March and Olsen 1976). Under either of these views, the idea that an individual school would collaborate with organizations or individuals outside of the hierarchy or outside of the school itself might seem anomalous. The gains from doing so are not self-evident.

Yet the broader school reform movement, emphases on academic accountability and improvement, and the interconnected and intractable problems that face students in many schools—especially those in low-income urban areas—motivate the push for collaboration (Wang and Boyd 2000). Understanding empirically the characteristics associated with a school’s participation in a collaboration is fundamental to identifying the governance structure(s) that will best accommodate what individual schools hope to gain from a collaboration. It is likely that schools’ incentives vary across type of school, across time, and across institutional settings. Thus, it is unlikely that findings for a particular type of school at a particular time in a particular institutional setting will generalize. The question remains, however, as to whether it is even possible to identify the incentives to collaborate for a particular sample of schools. Can their incentives be characterized as being consistent with predictions from rational choice, socialized choice, or resource dependency theories (or some combination)?

The readily ascertainable facts of a situation may provide clues about organizations’ incentives for participating in collaborative relationships (Hill and Lynn, 2003). Figure 1 shows examples of such characteristics, and classifies as being consistent with either rational or socialized choice theories. (We omit resource dependency theory from this table because it represents a specific theory, unlike the broad classes of rational and socialized choice theories). Some kinds of information (for example, transactions costs, organizational values and affiliations) are distinctive to a particular class of theory, while other kinds of information (for example, interorganizational relationships, interests of partner agencies) have ambiguous implications. We use these ideas to identify evidence of ChiPS schools’ concerns and behaviors that can be consistently characterized as suggesting rational choice, socialized choice, or resource dependency approaches to collaboration.

[Figure 1]

The ChiPS reforms discussed earlier were not directly focused on initiating or sustaining collaborations among schools and external organizations (though expanded local participation through LSCs and other forms were encouraged – Bryk et al. 1998). However, these reforms create an environment for schools with explicit incentives, and reflect a rational, command-driven approach to governance. Thus, while not a direct target of the ChiPS reforms, we predict that the governance environment created by the reforms will prove most effective in altering the inclination to collaborate of those schools that tend to be motivated primarily by organizational self-interest and least effective with those schools whose internal and external relationships tend to be socially constructed.Under these circumstances, we predict that schools that respond relatively more to motives based on self-interest would experience more ultimate “success” (i.e., higher student achievement). Schools that do not respond in ways that are consistent with the policy’s intent are likely to be less successful in the longer run.[5] Figure 2 depicts these predictions.