Discussion Paper and Draft Proposal
for
Conducting an Evaluation Study
of the
Discovery Simulation Programme
within the
Discovery Organisation of ABC Corp Pharmaceuticals
Submitted by:
The Synapse Group, Inc.
PO Box 9715-267
Portland, Maine, USA
Telephone 207-761-4221
Facsimile 207-874-0456
Email <>
Principal Consultants for this Proposal:
Bernard J Mohr (207-874-0118 voice direct)
and
Jane Watkins (202-332-0958 voice direct)
Table of Contents
I. OUR UNDERSTANDING OF YOUR GOALS & SITUATION P.2
II. HOW WE CAN RESPOND TO YOUR GOALS/SITUATION:
Criteria for and Benefits of our proposed approach P.3
III. EMBEDDED EVALUATION:
The Approach P.4
IV. APPRECIATIVE INQUIRY:
The Underlying Framework & Process for Embedded Evaluation P.6
V. PROPOSED PROJECT PLAN:
An Overview P.11
VI. PROPOSED PROJECT PLAN:
Key Activities, Required ABC Involvement & Approximate Timing P.12
VII. PROPOSED PROJECT PLAN:
Estimated Costs, Underlying Assumptions and Alternatives P.14
VIII. RELEVANT BACKGROUND AND EXPERIENCE
OF THE PRINCIPAL CONSULTANTS FOR THIS PROJECT P.16
I. OUR UNDERSTANDING OF YOUR GOALS AND SITUATION
(Please let us know immediately if we have misunderstood anything and we will find a way to adapt our proposal as appropriate)
In order to help meet the challenge of increasingly difficult quantitative and qualitative targets for your Discovery Organisation, ABC CORP-HROD in conjunction with XYZ Inc. have developed and run a three day internal workshop based on a dynamic computer simulation of the Discovery Research process.
ABC CORP-HROD would now like external support in:
• evaluating the impact that this intervention has had on the Discovery organisation
• identifying opportunities for consolidation and learning and follow-up;
• learning from the experiences of working with systems thinking concepts and
simulation technologies
Specific questions that you want to address fall into three categories:
i) Future Application
• What can be learned from this experience of using systems modeling and simulation methodology ?
• How can HROD improve the effectiveness and expand the use of these technologies within ABC CORP?
• What are the future development areas and opportunities within Discovery ?
ii) Programme Impact Evaluation
• To what extent has the Discovery Simulation programme achieved its specified objectives ?
• In what ways has it increased the knowledge/understanding/awareness of leaders in Discovery?
• What differences has it made to their behaviours ?
iii) Assessment of Systems Impact
• What impact has the Discovery Simulation programme had on the systems operating within Discovery
• What impact has it had on overall performance ?
Your time frame for this evaluation process is based on awarding the contract by Sept. 26, 1997 and you anticipate that a staged approach (during Oct. to Dec.) will be useful, resulting in delivery of a final report by Friday January 16, 1998
II HOW WE CAN RESPOND TO YOUR GOALS/SITUATION
(Criteria For And Benefits Of Our Proposed Approach)
Given our understanding of your goals and constraints (as described in the “Invitation to Tender” and based on the initial teleconference with Elizabeth Smith), we believe that the most effective evaluation strategy and process is one which meets the following criteria:
a) has the capability of responding in an integrated fashion, to the whole range of evaluation goals and objectives, and,
b) is implementable within the anticipated time frame, and,
c) can be shown to be rigorous and is based in reputable scientific theory as it applies to human behaviour and organizational learning, and,
d) collects data not only from Discovery Simulation participants, senior managers and other key stakeholders , but also,
e) recognizes the highly limited time availability for involvement by ABC personnel (i.e. Discovery Programme “graduates”, ABC -HROD personnel and other key stakeholders to the Discovery Programme initiative) and,
f) addresses the reality that ABC is a transnational organization, where things are done differently in different parts of the organization,
....while simultaneously, , using the evaluation process as a conscious organizational intervention for achieving the following key benefits:
g) deepening and reinforcing the learnings and applications from the Discovery Programme to the functioning of the overall ABC Discovery process, and,
h) creating a highly receptive environment (within the Discovery organization) for the implementation of conclusions and action recommendations emanating from the evaluation process, and,
i) transferring, to the ABC -HROD group, a powerful new process (embedded Evaluation combined with Appreciative Inquiry) for generative organizational learning (vs. adaptive or single loop), for use in future transformation and/or evaluation initiatives.
In order to maximize the probability that the benefits listed above will be fully realized, we are recommending the use of an approach to evaluation called Embedded Evaluation coupled with an Appreciative Inquiry framework and methodology. Both the Embedded Evaluation approach and the Appreciative Inquiry process which brings it to life, are described in the following sections.
III. EMBEDDED EVALUATION: THE APPROACH
SYNAPSE uses a customized approach to evaluation called Embedded Evaluation (based on the work of the Cambridge Partnership for Organizational Transformation - of which Jane Watkins is also a principal) coupled with an Appreciative Inquiry framework and methodology.
Embedded Evaluation is a process that involves the organization in discovering evaluation data from a wide range of stakeholders. This process will not only specifically address the questions that the organization has identified for the evaluation of the computer simulation workshop in the Discovery organization, but will also transfer into ABC Corp a capability to use the process in all aspects of it’s work. A group of employees chosen to participate in evaluating the computer simulation for the Discovery organization will participate in and learn to create a customized interview protocol for data collection; to carry out the interviews; to discover key learnings from the data; and, to create
recommendations for action. The data, narrative in form, provides a rich context from which to draw specific information that can be used to modify, improve and/or develop innovative approaches for additional uses of the computer simulation workshop.
A primary advantage of this approach is that it helps the organization create a culture that values examination of processes, relationships and tasks as a natural and creative part of the day-to-day work of the organization. It is an approach that emphasizes “generative” evaluation; that is, evaluation as an on-going learning
process which is a critical tool for managing change and generating continuous improvement in the organization’s work.
This approach to evaluation can be taught using any project or task evaluation as a case study so that the organization learns Embedded Evaluation by doing it in a real-time process. Once the organization has experienced this approach, the skills to apply it to all facets of their work are embedded in the organization. This involvement of employees of the organization insures: (1) a high level of ownership of the data by the organization itself; (2) enhanced accuracy of data because of the multi-level activity and capacity to reach close to or even all of those impacted by the project; and, (3) ease of implementation of changes because of the embedded nature of the information in the system.
To ensure that the emphasis shifts from “finding what is wrong and fixing it” -- a process that is far too slow in today’s rapidly changing environment -- to “finding what is going well and enhancing it,” we use the Appreciative Inquiry approach. AI focuses on that which is creative, generative and life-giving in the project or organization. The power of Appreciative Inquiry is that it emphasizes what has been done well and challenges those involved to think creatively and provocatively about ways to change and enhance that which is not working as well as desired. By combining the concept of Embedded Evaluation with the Appreciative Inquiry approach, we find that the energy of the people involved shifts toward creative approaches for improvement. This approach removes the fear of failure, defensive strategies and overly cautious behaviors that accompany a culture where evaluation is “summative” and used to find fault, pass judgment and/or punish. It also accepts that there is no way to discern the “one right way” to do a task or process and recognizes that the most creative approach to improvement is to have those involved feel free to assess their work from the vantage point of learning for the purpose of improving.
Finally, Embedded Evaluation becomes a key element in supporting an organizations transformation into a learning institution which can implement the lessons of evaluation quickly, smoothly and with minimal resistance. At a time when the nature and speed of change are so bewildering, the speed of responsiveness that embedded evaluation encourages is crucial to success -- and even to survival.
Embedded Evaluation done from an Appreciative Inquiry perspective will provide ABC Corp with relevant data that reflects widely shared reactions to and suggestions for the computer simulation workshop itself and innovative ideas for ways to use it creatively. In addition, we believe that this approach to evaluation has the capacity to be a transformational process for any organization that recognizes that the linear methods appropriate to a less complex and slower paced workplace are no longer adequate. The challenges of a global culture, the proliferation of information technologies and resulting flood of knowledge, and the increasing complexity of all aspects of life and work require organizations that are agile, creative and able to learn continuously. Embedded Evaluation provides a process capable of creating a learning organization that will thrive in the future.
IV. APPRECIATIVE INQUIRY: 1
The Underlying Framework and Process for Embedded Evaluation
NOTE: The following description of AI is intended to give the reader of this proposal some context for the customized project plan which follows. It is assumed that the scientific research and theory base for AI would be presented in the early stages of the project itself. The term “organizing” is used here in the largest sense - specifically including the meaning of how work in the Discovery organization occurs and how the learning from the Discovery Simulation programme is applied
Appreciative Inquiry is an approach to organizational analysis and learning that is uniquely intended for discovering, understanding, and fostering innovations in social organizational arrangements and processes. Appreciative Inquiry refers to both a search for knowledge and a theory of intentional collective action which are designed to evolve the vision and will of a group, organization or society as a whole. It is an inquiry process that affirms our symbolic capacities of imagination and mind as well as our social capacity for conscious choice and cultural evolution. The art of appreciation is the art of discovering and valuing those factors that give life to an organization or group. The process involves interviewing and collection of narrative data to draw the best of the past to set the stage for effective visualization of what might be. Critical perspectives guiding the practice of Appreciative Inquiry include the following:
1. Inquiry into "the art of the possible" in organizational life should begin with appreciation. Every system works to some degree, and a primary task of management and organizational analysis is to discover, describe, and explain those "exceptional moments" which give life to the system and activate members' competencies and energies. The appreciative approach takes its inspiration from "what is." Valuing, learning, and inspired understanding -- these are the aims of the appreciative spirit.
2. Inquiry into what's possible should be applicable. Organizational study should lead to the generation of knowledge that can be used, applied and validated in action.
3. Inquiry into what's possible should be provocative. An organization is an open-ended, indeterminate system capable of:
a. becoming more than it is at any given moment, and,
b. learning how to actively take part in guiding its own evolution.
Appreciative knowledge of "what is" becomes provocative to the extent that the learning takes on a normative value for members. In this way appreciative inquiry allows us to use systematic management analysis to help the organization's members shape an effective future according to their own imaginative and moral purposes.
4. Inquiry into the human potential of organizational life should be collaborative. This principle assumes an inseparable relationship between the process of inquiry and its content. A unilateral approach to the study of social innovation is a direct negation of the phenomenon itself.
In its most practical construction, appreciative inquiry is a form of organizational study that selectively seeks to locate, highlight, and illuminate what are referred to as the "life-giving" forces of the organization's existence.
In this sense, there are two basic questions behind any appreciative inquiry:
i. What, in this particular setting and context, makes organizing possible?
ii. What are the possibilities, expressed and latent, that provide opportunities for more effective (value-congruent) forms of organizing?
Appreciative inquiry seeks out the exceptional best of "what is" to help ignite the collective imagination of "what might be." The aim is to generate new knowledge which expands "the realm of the possible" and helps members of an organization envision a collectively desired future and to carry forth that vision in ways which successfully translate images into possibility, intentions into reality and belief into practice.
As a method of organizational analysis, appreciative inquiry differs from conventional managerial problem solving. The basic assumption of problem solving seems to be that "organizing-is-a-problem-to-be-solved." The task of improvement involves removing deficits. The process usually involves:
(1) identifying the key problems or deficiencies; (2) analyzing the causes;
(3) analyzing solutions; and (4) developing an action plan.
In contrast, the underlying assumption of appreciative inquiry is not that organizing is a "problem to be solved" but rather that it is a "solution to be embraced." The steps are: (1) discovery and valuing; (2) envisioning; (3) dialogue; and (4) co-constructing the future.
First, you discover and value those factors that give life to the organization. The challenge of valuing is to discover, for example, the commitment of the organization and to find out when that commitment was its highest. Regardless of how few the moments of highest commitment, the task is to zero in on these and to discuss the factors and forces that created the possibility for them. The list of positive or affirmative topics for discovery are endless: high quality, integrity, empowerment, innovation, customer responsiveness, technological innovation, team spirit, etc. In each case the task is discovery of the positive exceptions, successes, and most vital or alive effective moments. Discovery involves valuing those thing of value worth valuing. And it can be done both within and across organizations (in a bench marking sense) and across time (“organizational history as positive possibility”).