PUAD 3002: Program Design, Evaluation, and Decision-Making

PUAD 3002: Program Design, Evaluation, and Decision-Making

Instructor:

Sheila M. Huss, Ph.D.
School of Public Affairs; 1380 Lawrence Street, 5th Floor (525P)
Office Hours: Mondays and Wednesdays from 9:15 to 10:45; by appointment

Phone: (719) 660-8511 (cell); (303) 315-0526 (office)
E-mail: (preferred method of communication)

Welcome to PUAD 3002! I am looking forward to a great semester with everyone! The syllabus lays out what to expect in terms of course material, expectations, objectives, and assessments that will evaluate how well you are accomplishing the course objectives.

Course Description:

PUAD 3002 is a core course in the BAPS program. Effective public service program outcomes are systematically managed, monitored, and evaluated. Students in this class will learn the analytical, critical thinking, and problem solving skills required for program design, implementation, evaluation, and evidence-based decision making.

Service Learning:

Service learning is a large part of this course. That means that we are working with a partner in the community, Children’s Outreach Project. Children’s Outreach Project is a preschool in Denver, and many of the children in the school have special needs and come from low income families. The preschool has a number of different programs within it to provide services for the children. In this class, we will evaluate part of their program. Extensive details about Children’s Outreach Project and the program evaluation we will conduct for the school will be provided in many ways, including announcements, reading material, assignment descriptions, etc.

In the School of Public Affairs BAPS Program, graduates should participate in and contribute effectively to public service; analyze, synthesize, and think critically; solve problems; and make decisions. Students should demonstrate integrity and communicate and interact productively with a diverse and changing workforce and citizenry. Outcomes associated with this goal include field specific knowledge, personal and professional growth, quantitative and qualitative research skills, and communication skills. This goal and its outcomes are developed through coursework and experiential learning, specifically service learning. Service learning “…involves students in community service activities and applies the experience to personal and academic development.” Essentially, it involves real-world, applied experience and will build students’ resumes. Students who do not reside in Denver will participate in the service learning experience remotely.

Student Learning Objectives:

1.  Demonstrate knowledge of different public service programs while learning how to plan, implement, and evaluate these programs.

2.  Discuss various methods of evaluating public service programs.

3.  Explain the major concepts in program design and evaluation.

4.  Perform skills required in conducting program evaluations.

5.  Discuss the importance of technology in public service program design and identify specific ways that technology is integrated into public service programs.

6.  Apply theory to a real life setting involving the design, implementation, and/or evaluation of a public service program.

7.  Explain the connection between course content and your service learning experience (service learning objective).

8.  Participate in and reflect upon strategies for mitigating methodological challenges and substantive problems existing in the evaluation of a community partner’s program (service learning objective).

*See the last page of the syllabus for each objective and how it is fulfilled within this course.

Required Textbook and Readings:

Rossi, Lipsey, and Freeman. (2004). Evaluation: A Systematic Approach. Thousand Oaks, CA: Sage Publications.

Other readings will be required, and they can be accessed on the Canvas course page.

Most weeks, there will be an oral lecture for you to listen to—you are required to listen to the lectures. The material that is delivered in them will appear on the weekly quizzes.

You need to have an active CU Denver account (including e-mail); you need to be able to access the Canvas course page, as this page is where you will find readings, assignment descriptions, course announcements, etc. Please pay special attention to announcements, discussions, and modules. Canvas also is where you will submit assignments. If you have difficulties with Canvas, please contact the Canvas Help Desk.

Disability Accommodations:

CU Denver is committed to providing reasonable accommodations for all persons with disabilities. The syllabus is available in alternate formats upon request. If you have a disability that may impact your work in this class and for which you may require accommodations, please contact Disability Services at (303) 556-3450. Students who receive accommodation letters, please contact me to discuss the provisions of those accommodations as soon as possible. If you do not have a disability, but are dealing with an issue that may affect your performance in this course (e.g., English is your second language), please let me know, so we can work together to manage the issue in a way that will facilitate your success in this course.

Academic Dishonesty:

All students must be honest and forthright in their academic studies. To falsify the results of research, to steal the words or ideas of another, to cheat on an assignment, or to allow/assist another to commit these acts corrupts the educational process. Students are expected to do their own work and neither give, nor receive, unauthorized assistance.

Because academic dishonesty seems to be a systemic problem at CU Denver, you are required to review the Canvas module on plagiarism and take a multiple choice quiz (built into the module) on each section.** This exercise is a way for you to learn about academic dishonesty in a proactive, non-threatening manner. You must complete all three of the section quizzes and either copy and paste them (with your results showing a perfect score) or take a screen shot of each one (again, showing a perfect score). I just need to see evidence of all quiz questions answered correctly (screen shots or copy/paste or a certificate of completion, as I said in the previous sentence)—no need to stress about how you submit the documentation! The academic dishonesty quizzes are an ungraded assignment on Canvas, but required. Failure to complete this exercise will result in half of a letter grade deduction in your final grade.

http://www.ucdenver.edu/academics/CUOnline/FacultyResources/AcademicHonesty/Documents/student/introduction/index.htm
**Click “Academic Integrity Course” to get started

Additionally, here is a link that discusses how to avoid plagiarism. This document gives specific examples of how to summarize and synthesize information and how to appropriately cite the information. You are required to familiarize yourself with this information.

http://www.ucdenver.edu/academics/colleges/SPA/CurrentStudents/academicresources/Documents/Plagiarism%20Workshop.pdf

Finally, I use a program on Canvas called turnitin. Turnitin provides a report of originality of your work, along with a score, which reflects the percent of your work that is not original. You should submit your assignments early, so that you will be able to see your turnitin report. If you submit an assignment and notice a turnitin score of (for example) 10%, I recommend reviewing the report and, if necessary, revising and resubmitting your assignment. There may be reasons other than plagiarism for a high turnitin score, so you should review the report to see what material is identified as unoriginal. For example, in classes where I have given essay exams, students were instructed to copy and paste the essay questions above their answers. Everyone had a turnitin score of about 15% because the essay questions were identical on everyone’s exam. Viewing the turnitin report will show you this type of thing. You have an opportunity to self-correct—please take it. Once an assignment is due, you may not resubmit it, so if you would like to take advantage of the opportunity to check your own work for plagiarism, you need to turn in the document before the due date/time.

Because you have multiple opportunities before you turn in an assignment to learn what academic dishonesty is, to ask questions about it, and to correct your own work, if I catch academic dishonesty, the consequence very likely will be an automatic F in the class and notification of the dean. Your intent is irrelevant (i.e., “I didn’t know that was cheating” is not an excuse). Part of being fair is being consistent, so I will not consider individual circumstances—all instances will be treated the same.

Late Assignments:

For each day (24-hour period) an assignment is late, 5 points will be deducted from your total score on the assignment.

Email Etiquette:

I frequently check e-mail and will make every effort to get back to you in a timely manner; however, I also maintain an active social and professional life outside of class and may not be able to provide as prompt a response as you desire. Typically, I respond to e-mails within 24 business hours and almost always within 48 business hours. If you wait until the day before an assignment is due to ask a question, you risk not receiving an answer in time—please do NOT send multiple e-mails in a short period of time. I do not ignore e-mails, so when I receive your e-mail and have a chance to respond, I will.

Evaluation of Student Learning:

Students will be evaluated according to the following grading scale and criteria:

93% -100% (555-600) A
90%-92% (537-554) A-
88%-89% (525-536) B+
83%-87% (495-524) B
80%-82% (477-494) B-
78%-79% (465-476) C+
73%-77% (435-464) C
70%-72% (417-434) C-
60%-69% (357-416) D
Below 60% (356 or below) F

You will be graded on weekly quizzes, weekly threaded discussions (which take many forms), and a service learning project. The total number of possible points is 600; extra credit is built in as we go. Detailed assignment descriptions are available on Canvas—you should read them prior to submitting assignments.

Weekly Quizlets (15 weeks x 10 points = 150 points/25%): Each week, you will create a quizlet to demonstrate your understanding of the required reading. More details are discussed in a separate assignment sheet. If you read what was assigned, you should not have difficulty with the quizlets.

Threaded Discussions (15 weeks x 15 points = 225 points/37.5%): Each week, you are required to participate in a threaded discussion. The discussion will be in various formats—sometimes respond to a prompt; other times, you will complete a 1-page formative assessment and respond to others’ work.

Experiential Course Project (225 points/37.5%): This project is an opportunity for you to engage in evaluation research with a community partner. Specifically, you will conduct a program evaluation for Children’s Outreach Project, the components of which include: an introduction and theoretically-based literature review that includes defining the problem and describing the intervention (35 points); developing indicators (20 points); writing a methods section that includes both process and outcome evaluation components (35 points each – total 70 points); and analyzing data and discussing the findings (50 points). The final product will be a written report to be submitted to our client (which is just all of the previous components combined) and a presentation for Children’s Outreach Project (participation can be at the site or via Zoom, a video conferencing system that I will set up). The presentation should include power point slides and supplementary materials. The final product, which also will incorporate how well you engaged with Children’s Outreach Project, is worth 50 points.

Extra credit is built-in! If you guys do an awesome job on your course project, I will award extra credit points on each assignment and the final report and presentation.

Due dates are noted on the course calendar—all assignments, including quizzes and weekly discussions, are due on Saturdays by 11:59 PM, unless otherwise noted.

Class Schedule and Reading Assignments:

Week 1: Overview of Program Design and Evaluation Concepts (1/17-1/21 – short week)
Readings: Rossi et al. Chapter 1; See Canvas Module 1
Assignments: Weekly quiz; threaded discussion

Week 2: Overview of Concepts Continued and Tailoring Evaluations (1/22-1/28)
Readings: Rossi et al. Chapter 2; See Canvas Module 2
Assignments: Weekly quiz; threaded discussion

Week 3: Overview of Concepts Continued and Identifying Issues and Formulating Questions (1/29-2/4)
Readings: Rossi et al. Chapter 3; See Canvas Module 3
Assignments: Weekly quiz; threaded discussion

Week 4: Developing a Conceptual Framework and Introduction to Formative Research (2/5-2/11)
Readings: See Canvas Module 4
Assignments: Weekly quiz; threaded discussion; draft of Introduction and Literature Review,
including problem definition and intervention

Week 5: Pre-Testing and Needs Assessment (2/12-2/18)
Readings: Rossi et al. Chapter 4; See Canvas Module 5
Assignments: Weekly quiz; threaded discussion

Week 6: Developing Indicators and Participatory Evaluation Methods and Program Theory (2/19-2/25)
Readings: Rossi et al. Chapter 5; See Canvas Module 6
Assignments: Weekly quiz; threaded discussion

Week 7: Process Evaluation and Measuring and Monitoring Program Outcomes (2/26-3/4)
Readings: Rossi et al. Chapter 6; See Canvas Module 7
Assignments: Weekly quiz; threaded discussion; Draft of Indicators

Week 8: Special Topics: Technology in Program Design and Evidence-Based Management (3/5-3/11)
Readings: See Canvas Module 8
Assignments: Weekly quiz; threaded discussion

Week 9: Measuring and Monitoring Outputs and Outcomes and Introduction to Study Design (3/12-3/18)
Readings: Rossi et al. Chapter 7; See Canvas Module 9
Assignments: Weekly quiz; threaded discussion

CU Denver spring break is from 3/19 to 3/25/17. Have an awesome break!!

Week 10: Experimental and Quasi Experimental Designs (Randomized Control Trials and Randomized
Field Experiments) (3/26-4/1)
Readings: Rossi et al. Chapter 8; See Canvas Module 10
Assignments: Weekly quiz; threaded discussion

Week 11: Assessing Program Impact (4/2-4/8)
Readings: Rossi et al. Chapter 9; See Canvas Module 11
Assignments: Weekly quiz; threaded discussion; draft of process evaluation

Week 12: Using Qualitative Methods for Monitoring and Evaluation (4/9-4/15)
Readings: See Canvas Module 12
Assignments: Weekly quiz; threaded discussion; draft of outcome evaluation

Week 13: Detecting, Interpreting, and Analyzing Program Effects, Including Statistics (4/16-4/22)
Readings: Rossi et al. Chapter 10; See Canvas Module 13
Assignments: Weekly quiz; threaded discussion

Week 14: Measuring Efficiency and Cost Effective Analysis (4/23-4/29)
Readings: Rossi et al. Chapter 11; See Canvas Module 14
Assignments: Weekly quiz; threaded discussion; draft of findings and discussion

Week 15: The Social Context of Evaluation: Monitoring and Evaluation in Action (4/30-5/6)
Readings: Rossi et al. Chapter 12; See Canvas Module 15
Assignments: Weekly quiz; threaded discussion; Final Evaluation Report