ArgosyUniversity
COURSE SYLLABUS
R7036
Program Evaluation Methods
Faculty Information
Faculty Name: Edward Balli, DBA
Campus: Chicago
Contact Information: 815-501-0175
Office Hours: 5:00 p.m. to 6:00 p.m. Fridays
Short Faculty Bio: I am a partner in Financial and Managerial Solutions in Dekalb, Illinois. I earned my Doctored in Business Administration in Accounting with a minor in MIS from ArgosyUniversity in August 2007. My experience is in working with small to medium size private and public firms in the Midwest with accounting and information systems. The recommendations may include a complete audit of the operations to recommendations of proposed application software, telecommunications, and hardware. I have over 25 years of experience. I have taught part time for over 12 years at AuroraUniversity in business courses.
I am a member of American Institute of Certified Public Accountants (AICPA); Information Systems Audit and Control Association (ISACA); Certified Internal Auditor (CIA); Special Interest Group on Knowledge Discovery and Data Mining (SIGKDD); and Society for Industrial and Applied Mathematics (SIAM); American Mathematical Society (AMS) and Illinois CPA Society (ICPAS).
Course description: This seminar emphasizes the acquisition of knowledge and skills in program evaluation methodology. Six alternative evaluation approaches are surveyed, with a focus on developing a management/decision-oriented evaluation plan. This seminar also serves as a practicum for the conceptualization and development of a doctoral research study that employs a program evaluation model.
Course Pre-requisites: PC6521 or equivalent (does not apply to business students)
Required Textbook:
Fitzpatrick, J., Sanders, J. R., & Worthen, B. R. Program Evaluation: Alternative Approaches and Practical Guidelines (3rd ed.). Allyn & Bacon. ISBN: 0321077067
Recommended Reading:
Wholey, J., Hatry, H., & Newcomer, K. (2004). Handbook of practical program evaluation (2nd edition). Jossey-Bass. ISBN: 0-7879-6713-0.
Online Resources:
Technology: Pentium III CPU/ Windows XP; 128MB RAM, printer; Microsoft Office 2000 or XP, Acrobat (full version), Microsoft Internet Explorer 6.0 (PC) 5.0 (MAC) or Netscape Navigator 8, and Norton Antivirus.
Course length: 7.5 weeks
Contact Hours: 45 Hours
Meeting Time: Friday 6P.M. to 10P.M.
In Class Meetings
Session DATESTIMES
Session 1 10/24 6 PM to 10 PM
Session 210/316 PM to 10PM
Session 311/076 PM to 10PM
Session 411/146 PM to 10PM
Session 511/216 PM to 10PM
Session 612/056 PM to 10PM
Session 712/126 PM to 10PM
Credit Value: 3.0
Program Outcomes:
- Research and Theory - Educational Leaders make decisions based on research and supported theory. They use their knowledge and skills to:
- Performing - Analyze, design, conduct, and defend research in an educational context using action research and other appropriate designs.
- Performing - Demonstrate knowledge of the role of research in professional development of faculty and students.
- Understanding
- Evaluate and apply research and reflective practices to complex educational problems, including the assessment of improvement and accountability systems.
- Exhibit a broad and systematic understanding of the role of research and theory in professional education.
- Analysis
- Critically analyze current research in education and related fields to inform curricular and instructional decisions
- Analyze student achievement data for program improvement.
- Applying - Evaluate and apply theory and research to select, integrate, implement and assess educational practices and materials
- Decision Making - Make decisions about curriculum, instructional strategies, assessment and professional development that are based on sound research, best practice, school and district data and other contextual information
- Communication and Information Literacy - Educational leaders effectively communicate a vision of educational excellence to the learning community. They use their oral and written language skills and their knowledge of informational literacy and organizational communication to:
- Oral and Written Communication - Orally or in writing, present educational documents (including recommendations, critiques or justifications) that are clear, concise, organized and well supported in a professional manner using media appropriate to the education context and audience, using appropriate media and technology.
- Organizational Communication - Exhibit skills in building trust, gauging climate understanding, assessing perception, facilitating systemic processes associated with organizational communication and promoting a positive learning culture.
- Information Literacy - Demonstrate multiple literacies (including information technology, knowledge of media, conflict resolution, etc.) to effectively support improved achievement, communicate educational messages and promote harmony.
- Using Technologies - Promote the use of a broad range of software applications or Web-based tools to facilitate systematic investigation, to enrich curriculum and instruction, to monitor instructional practices and to provide staff the assistance needed for improvement.
- Integration - Demonstrate effective integration of instructional technology into the teaching and learning process and to manage the assessment and reporting of student learning.
- Accountability - Use technology to measure, assess, and evaluate information related to institutional effectiveness and accountability.
- Critical Thinking and Problem Solving - Educational leaders develop systems to check their own thinking, to reflect on present and past practices in considering the future and to analyze complex issues and to evaluate potential solutions. They use their knowledge and skills to:
- Critical Thinking - Analyze contemporary theoretical concepts at all levels of educational leadership, in design, implementation, and evaluation of personnel preparation programs at pre-service and in-service levels.
- Problem Solving/Decision Making - Identify a problem, diagnose its underlying causes, evaluate possible solutions in relation to foundation theoretical foundations in education, and determine and defend an appropriate course of action.
- Evaluation -Evaluate relevance of established theory to current education practice and identify gaps in current literature.
- Collaboration - Educational leaders promote democratic values by respecting the ideas and gifts of those with whose care they are charged. They continuously seek improvement and use their knowledge and skills to:
- TeamBuilding - Build teams to accomplish the goals of complex organizations with educational missions.
- Families and Community - Work with families and various stakeholders of the educational community in addressing, selecting and implementing an optimal course of action to deal with the issues that face educational leaders, including, but not limited to, resource acquisition and management, policy development and analysis, program development and evaluation, staff selection and development, school-community relations, and curriculum development.
- Focus on Achievement - Use observation and collaboration to design, implement and evaluate meaningful and effective experiences that improve student achievement.
- Leadership - Educational leaders have an evolved leadership style that supports the development of others, is harmonious with democratic principles and best educational practices and focuses on student achievement as the ultimate goal of the learning community. They use their knowledge and skills to:
- Leading - Provide leadership in various roles and responsibilities to work effectively in achieving common goals.
- Motivating, Coaching, and Delegating - Train, coach, delegate, and motivate faculty, staff, students, parents and other members of the educational community.
- Supervision - Evaluate teaching and staff performance using a variety of supervisory models.
- Professionalism - Meet research, teaching, and service requirements of faculty positions in public, private or higher education.
- Change Management - Develop clear initiatives and adapt his/her leadership behavior to assist an educational institution, district or school reach its vision, mission, and goals in a changing environment.
- Contemporary Issues - Promote the success of all students by understanding, responding to and influencing the larger political, social, economic, legal and cultural context on the local, state and national levels.
- Curriculum, Instruction, and Assessment - Use best practices and sound educational research to improve instructional programs.
- Data-driven Decision Making - Apply data and information to solve educational problems.
- Managing Resources - Manage the organization, operations and resources in a way that promotes a safe, efficient and effective learning environment
- Ethics/Principles - Educational leaders are bold in exhibiting and expecting accountability in integrity, fairness and ethical behavior. Their actions
- Integrity - Demonstrate a respect for the rights of others with regard to confidentiality and honesty
- Fairness – Demonstrate their ability to combine impartiality, sensitivity to diversity and a concern for others in dealing with all constituencies
- Ethical Behavior - Demonstrate personal and professional values of their organizations and the ideals of our democratic society
- Diversity – Educational leaders appreciate the value of every individual and are committed to their success. They use their knowledge and skills to:
- Managing – Design and implement educational policies, instruction and curricular reform that reflect culture and diversity.
- Theory & Research - Apply human development theory, proven learning and motivational theories, and concern for diversity to the learning process
- Leading -Capitalize on diversity to create a culture that promotes respect and success for all students.
- Internship - Through a program of field study, educational leaders have had the opportunity to demonstrate their knowledge and skills situations that contribute to their professional development. They have used their knowledge and skills to:
- Applying - Participate in a sustained and extensive internship activity that provides opportunities to apply skills and to develop new skills in real world settings.
- Facilitating and Engaging - Arrange and engage in course embedded field experiences that use best practices and sound educational research to improve instructional programs.
- Human Growth & Development - Educational leaders consider the personal and professional continuum of development of their various constituencies and use this knowledge in all phases of their work from curriculum planning to staff development. They use their knowledge of human intellectual, physical, social and emotional growth to:
- Human Development – Help form comprehensive professional growth plans that focus on authentic problems and tasks, mentoring, coaching, observation, conferencing and collaborative reflection, and other techniques that promote new knowledge and workplace skills.
- Organizational Development – Design procedures and techniques of inquiry-based organizational development and leadership to assist school and district personnel in understanding and applying best practices for student learning.
- Personal Leadership – Promote the success of all students of a school, district or educational institution of higher learning of learning by facilitating the development, articulation, implementation, and stewardship of a vision supported by the educational community.
- Lifelong Learning - Develop personal professional growth plans that reflect commitment to lifelong learning and best practices. Continuously access, reflect on, and integrate knowledge to remain current in educational profession and practice.
- Program Specific – Curriculum and Instruction
- Application of Curriculum & Instruction – Formulate instructional design competencies related to strategies for growth in curriculum planning, decision making, problem solving and change management.
- Analysis of Curriculum & Instruction Principles - Use supported research and theory to plan, implement and evaluate curricular and instructional practices.
- Curriculum Theory & Design - Demonstrate advanced skills in synthesizing and merging curriculum and instructional theory to design, implement and assess instructional programs that lead to improved student achievement.
- Evaluation - Evaluate the instructional processes, strategies, techniques and methodologies (e.g., strategies for growth in planning, decision-making, problem solving, and change management) in an educational institution.
- Monitoring – Monitor and evaluate operational systems to ensure they enhance student learning and reflect accountability to the community.
- Promote Technology - Use and promote technology and information systems to enrich district curriculum and instruction, monitor instructional practices and provide assistance to administrators who have needs for improvement
- Staff Development - Assist school and district personnel in understanding and applying best practices for student learning
- Professional Responsibility - Apply theoretical concepts for all levels of curriculum design, implementation and assessment of educational preparation programs
Course Objectives:
- Define curriculum and program evaluation.
- Discern the differences between research and evaluation and identify which is more appropriate to solving an educational problem.
- Identify an appropriate assessment tool and apply it to a program or curriculum to determine feasibility of evaluation.
- Construct a blueprint for a curriculum or program evaluation
- Conduct appropriate “scoping” activities to focus and inform the evaluation plan.
- Identify appropriate sources of information to be used for data collection in a program or curriculum evaluation.
- Identify the most appropriate data collection method to be used in a program or curriculum evaluation.
- Determine the reliability and validity of data collection instruments to be used in a program or curriculum evaluation.
- Identify the purpose of a study, research methodology, data collection methods, statistical analyses used, and results of the study and will be able to discuss the implications of the study as related to best practice in program and curriculum evaluation.
- Identify and apply commonly used quantitative procedures to measure program or curriculum outcomes.
- Identify threats related to the internal and external validity related to the research methodology used to evaluate a program or curriculum and use this information to determine the most appropriate research design.
- Use statistical analyses to determine the effectiveness of a program or curriculum and will be able to translate the numbers into words so that users can read and understand the information to improve the program or curriculum being evaluated.
- Identify the best presentation method of the results of a program or curriculum evaluation and present the information in a manner that is understood by users.
- Identify an appropriate method to analyze data collected during a program or curriculum evaluation in order to glean the most important results to communicate to users.
- Lead an evaluation team in identifying information to be summarized and revealed to users related to a program or curriculum evaluation.
- Identify potential ethical issues related to the evaluation of a curriculum or program and apply appropriate problem-solving strategies to reduce the likelihood that these issues will effect the implementation or results of the evaluation.
- Construct a timely work plan and cost-effective budget related to the implementation of a program or curriculum evaluation.
Assignment Table
Week / Module Topics / Readings / Assignments1 /
- Approaches to Curriculum and Program Evaluation
- Difference Between Research and Evaluation
- Standards of Effective Evaluation
- Issues and Problems Associated With Evaluation
- Course Project:
Overview:
This project is designed to provide you with on-the-job training. In this project you will develop an evaluation design paper for a curriculum or program. You will be expected to conduct the scoping activities and design the evaluation. Scoping out the evaluation entails collecting information on the program or curriculum through interviews with key contacts (decision-makers, staff, etc.) on current information needs, and conducting a synthesis of past-related research and evaluation studies. After the focus of the evaluation is identified, the project will involve laying out an evaluation design, data collection plan, analysis plan, budget, and briefing and presentation plan. The design should be developed with clear awareness of the political aspects of the situation and tailored to the needs of the agency, organization, or school leadership. Strategies for encouraging the use of the resulting evaluation findings should also be discussed.
- Research an organization of your acquaintance (you might chose your educational university). Identify at least three evaluation objects that you believe would be appropriate for study. For each of the objects, identify:
ii)The basic focus of the evaluation
- Next, identify a program to evaluate. In one or two pages, provide a broad overview of the evaluation, including its purpose, methods, and likely uses.
ii)Who are the users of the program?
iii)Who are the stakeholders?
iv)Why does the program exist?
v)What are the goals and objectives of the program?
vi)What are the fundamental elements of the program?
2 /
- Deciding Whether to Evaluate or Not: Evaluability Assessments
- Types of Evaluation
- Curriculum Evaluation
Chapter 4
Chapter 5
Chapter 6
Chapter 7
Chapter 8
Read Hypothetical Project:
Science Teacher Enhancement: Introducing Faculty in Middle Schools to New Concepts and Methods in Science Instruction / Discussion Questions:
- Describe how performance measurement or standards-based education has influenced the work of your school or organization. Are these “evaluation measures” useful for educators, students, and stakeholders? Give reasons.
- Analyze the advantages and disadvantages of an evaluator acting as an advocate? Describe a situation in which it would be appropriate for the evaluator to serve as an advocate and a situation in which it would be inappropriate. Justify your position.
- As a potential evaluator, how do you think evaluability assessment might help inform your process? How might it help the users of information?
- Application
Also, post a summary of your findings to the Discussion Area.
- The city jail in a mid-size Florida city has experienced two cases of suicide by inmates in the last few months. As a result, the city council has hired a new director for the jail. The new director recognized that the jail has quite a few problems with officers’ and guards’ morale, physical facilities, training, security, and other issues and has decided to call in experts to advise him on the status of the jail. You’ve been hired to coordinate the evaluation. Provide detailed information on the types of evaluation question you will be asking, what types of data will be collected, and your justification for the type of evaluation model you will employ.
- Imagine that you have been asked to conduct a project evaluation of a confidence-building program for adult graduate students, who have returned to school after being out for several years. The program includes components that have students identify what their academic strengths and weaknesses are from their perspective as well as from the perspective of their prior teachers, friends, and employers. Students are also asked to identify what types of instructional activities they prefer and to create short-term and long-term goals related to their academic career. Based on your background, philosophy of education and methodological preferences, identify an evaluation model that you would use. List three specific questions you would be interested in asking the students of the institution. Also, discuss whether you would focus more on the individual or the group?
- Project