UCF School of Electrical Engineering and Computer Science
Course Assessment Report
COP 4331
Processes for Object Oriented Software Development
Spring 2014
The following reports the course learning outcomes, expected performance criteria, mapping of the course outcomes to the program educational outcomes, assessment measures, assessment results, and reflective statements. This report encompasses the spring 2014 offering of COP 4331 Section 1. Only one section was offered for the spring 2014 semester.
1) Course outcome:CO-COP4331-1 states that “The students shall be able to construct UML diagrams of the following types: Use Case Diagram, Class Diagram, Activity Diagram, Collaboration Diagram, State Transition Diagram, Sequence Diagram and Data Flow Diagram.”
Performance criteria:75% score on homework assignment #1: a collection of UML diagrams for a defined system, 75% score on homework assignment #2: a collection of UML diagrams for a defined system
Student outcomes: SO-CpE-03, SO-CS-09
Assessment measures: 70% of students with a passing score of 75% or higher for the course outcome.
Assessment result:
COP 4331-Section 1:
Processes for Object Oriented Software Development Course outcome 1 was measured for Spring 2014 semester among 127 students. For the first performance criteria, 114 students met the threshold for passing this outcome, which represents 89.8% (114/127) that meets the performance criteria. For the second performance criteria, 103 students met the threshold for passing this outcome, which represents 81.1% (103/127) that meets the performance criteria.
In previous review cycle2013, 79.1% met the first performance criteria, resulting in 10.7% gain. The result in gain was due to more lab time allocated to the topic, different instructors, and different performance criteria.
2) Course outcome: CO-COP4331-2 states that “The students shall be able to work in a group environment and function on multidisciplinary teams.”
Performance criteria:75% score on the Individual Lessons Learned document: a reflective paper on the efforts of each team member and the group environment.
Student outcomes: SO-CpE-04, SO-CS-04
Assessment measures: 70% of students with a passing score of 75% or higher for the course outcome.
Assessment result:
COP 4331-Section 1:
Processes for Object Oriented Software Development Course outcome 2 was measured for Spring 2014 semester among 127 students. 119 students met the threshold for passing this outcome, which represents 93.7% (119/127) that meets the performance criteria.
In previous review cycle 2013, 91.4% met the performance criteria, resulting in 2.3% gain. The result in gain was due to different instructors. The performance criteria in question is graded based on the student’s responses to the reflective questions and their teammates’ comments on their performance. There is some subjectivity to the grading that may cause slight variations from instructor to instructor and semester to semester.
The program ensures consistency by maintaining the same text, instructor presentations, project deliverables, and project templates for each offering of COP 4331. The performance criteria in question is part of the project suite and is assigned each semester.
3) Course outcome: CO-COP4331-3 states that “The students shall be able to work on a project for a client, performing all the steps associated with the object-oriented software development life cycle including elicitation of the requirements from a client, preparation of software requirements specification, project management plan and test plan of a software system.”
Performance criteria:75% score onProject Deliverables 1: a collection of documents for project planning, 75% score on Project Deliverables 2: a collection of documents for project planning, Midterm Exam, and Final Exam.
Student outcomes: SO-CpE-05, SO-CS-02, SO-CS-03, SO-CS-10, SO-CS-11, SO-CS-12, SO-CS-13
Assessment measures: 70% of students with a passing score of 75% or higher for the course outcome.
Assessment result:
COP 4331-Section 1:
Processes for Object Oriented Software Development Course outcome 3 was measured for Spring 2014 semester among 127 students.
The first performance criteria is the Project Deliverables 1 assignment. For the first performance criteria, 127 students met the threshold for passing this outcome, which represents 100% (127/127) that meets the performance criteria. In previous review cycle 2013, 100% met the first performance criteria, resulting in 0% gain.
The second performance criteria is the Project Deliverables 2 assignment. For the second performance criteria, 127 students met the threshold for passing this outcome, which represents 100% (127/127) that meets the performance criteria. In previous review cycle 2013, the second performance criteria was not included.
The third performance criteria is the Midterm Exam. For the third performance criteria, 80 students met the threshold for passing this outcome, which represents 62.9% (80/127) that meets the performance criteria. In previous review cycle 2013, 90.5% met the third performance criteria, resulting in 27.6% decrease. The result in decrease was due to different instructors and not enough time allocated to topics.
The fourth performance criteria is the Final Exam. For the fourth performance criteria, 89 students met the threshold for passing this outcome, which represents 70.1% (89/127) that meets the performance criteria. In previous review cycle 2013, 82.6% met the fourth performance criteria, resulting in 12.5% decrease. The result in decrease was due to different instructors and not enough time allocated to topics.
The program ensures consistency by maintaining the same text, instructor presentations, project deliverables, and project templates. The homework assignments focus on the same topic areas: UML diagrams and academic research. Similarly, some rubrics are shared by instructors.
However, to prevent students from sharing answers from previous semesters, the exams must change, at minimum, the design case study on which many questions are based. In future semesters it may be beneficial for instructors of COP 4331 coordinate more closely on the development of midterm and final exams to ensure consistent emphasis on topics covered in the course.
4) Course outcome: CO-COP4331-4 states that “The students shall be able to write technical high level design and detailed design of a software system.
Performance criteria:75% score on Project Deliverables 2: a collection of documents for project planning, 75% score on Project Deliverables 3: a collection of documents for project planning and implementation.
Student outcomes: SO-CpE-11, SO-CS-06
Assessment measures: 70% of students with a passing score of 75% or higher for the course outcome.
Assessment result:
COP 4331-Section 1:
Processes for Object Oriented Software Development Course outcome 4 was measured for Spring 2014 semester among 127 students. For the first performance criteria, 127 students met the threshold for passing this outcome, which represents 100% (127/127) that meets the performance criteria. For the second performance criteria, 127 students met the threshold for passing this outcome, which represents 100% (127/127) that meets the performance criteria.
In previous review cycle 2013, 95.7% met the first performance criteria, resulting in 4.3% gain. As the instructors use the same materials for preparing and presenting lectures, use the same project templates, and maintain similar homework assignments and exams it is very likely that variations are due to statistical fluctuations.
In previous review cycle 2013, the second performance criteria was not included.
5) Course outcome: CO-COP4331-5 states that “The students shall be able to produce implementation of design and write user’s manual including build instructions of a software system.”
Performance criteria:75% score on Project Deliverables 4: a collection of documents for project implementation and testing
Student outcomes: SO-CpE-11, SO-CS-06
Assessment measures: 70% of students with a passing score of 75% or higher for the course outcome.
Assessment result:
COP 4331-Section 1:
Processes for Object Oriented Software Development Course outcome 5 was measured for Spring 2014 semester among 127 students. 127 students met the threshold for passing this outcome, which represents 100% (127/127) that meets the performance criteria.
In previous review cycle 2013, 94.8% met the performance criteria, resulting in 5.2% gain. As the instructors of COP 4331 use the same materials for preparing and presenting lectures, use the same project templates, and maintain similar homework assignments and exams it is very likely that variations are due to statistical fluctuations.
6) Course outcome: CO-COP4331-6 states that “The students shall be able to prepare and successfully give oral presentation of a software system design and operation of the class project deliverables.”
Performance criteria:75% score on Final Presentation and Demonstration
Student outcomes: SO-CpE-07, SO-CS-06
Assessment measures: 70% of students with a passing score of 75% or higher for the course outcome.
Assessment result:
COP 4331-Section 1:
Processes for Object Oriented Software Development Course outcome 6 was measured for Spring 2014 semester among 127 students. 82 students met the threshold for passing this outcome with their individual score, which represents 64.5% (82/127) that meets the performance criteria. 127 students met the threshold for passing this outcome with their group score, which represents 100% (127/127) that meets the performance criteria.
In previous review cycle 2013, 94.8% met the performance criteria for Final Presentations. Scores were not split based on individual and group grades in the previous review cycle. The difference in scores between the individual and the group grades indicates that a stronger emphasis on individual work is necessary in future semesters.
The group presentation score is used as the deciding performance criteria, as it has greater weight in students’ overall grade. The target is met and the result is a 5.2% gain. As the instructors of COP 4331 use the same materials for preparing and presenting lectures, use the same project templates, and maintain similar homework assignments and exams it is very likely that variations are due to statistical fluctuations.
7) Course outcome: CO-COP4331-7 states that “The students shall be able to read and critique a research paper on a specific area such as requirements, design, implementation and integration, maintenance, testing, and metrics.”
Performance criteria:75% score on Homework 3: a research paper summary and critique in the field of Software Engineering
Student outcomes: SO-CpE-10, SO-CS-01
Assessment measures: 70% of students with a passing score of 75% or higher for the course outcome.
Assessment result:
COP 4331-Section 1:
Processes for Object Oriented Software Development Course outcome 7 was measured for Spring 2014 semester among 127 students. 77 students met the threshold for passing this outcome, which represents 60.6% (77/127) that meets the performance criteria.
In previous review cycle 2013, 88.6% met the performance criteria, resulting in 28% decrease. The result in decrease was due to different instructors.
The program ensures consistency by maintaining the same text, instructor presentations, project deliverables, and project templates. The homework assignments focus on the same topic areas: UML diagrams and academic research. Similarly, some rubrics are shared by instructors.
However, to prevent students from sharing answers from previous semesters, the exams must change, at minimum, the design case study on which many questions are based. In future semesters it may be beneficial for instructors of COP 4331 coordinate more closely on the development of midterm and final exams to ensure consistent emphasis on topics covered in the course.
Reflective statements:
RS-1. Successes:
ANS: The performance criteria were achieved for section COP4331-01 for all course outcomes except CO-COP4331-7. The average measure of all the assessment results is 86.3%. The teams gave very good presentations and turned in exceptional documentation throughout the semester, with 100% of project deliverables above 70%.
The primary reason for changes in assessment results are the different instructors for the course. The program ensures consistency by maintaining the same text, instructor presentations, project deliverables, and project templates. The homework assignments focus on the same topic areas: UML diagrams and academic research. Similarly, some rubrics are shared by instructors.
Likewise, the material and the course project support one another in theory. In practice, there are small variations across semesters. Students, Teaching Assistants, or Instructors will focus more on one of these areas throughout the semester. In the spring 2014 course, the emphasis was on project deliverables over the material assessments like the Midterm and Final Exams.
RS-2. Lessons Learned:
ANS: CO-COP4331-7 relates to the students ability to read and critique a research paper. Some preparatory information may need to be included in the course, specifically to assist students in finding and evaluating appropriate research papers. There are modules in Webcourses@UCF offered by the UCF library that may be useful here.
There continues to be group work struggles that become apparent at the end of the semester. It may be useful to add more group evaluation documents like the Individual Lessons Learned throughout the semester or hold biweekly meetings with the student teams to spot trouble earlier.
Team structure remains a concern. As noted in previous assessments, the stronger programmers of the team tend to lead the coding efforts and allow other members of the team to adopt support roles.
RS-3. Proposed Improvements:
ANS: Implement a schedule of meetings with student teams for every other week, or every third week to monitor progress and communication. Proposed improvements should be implemented in the spring 2015 semester to measure the effect on student performance. The purpose is to be able to more accurately assess an individual’s contribution to the group project and identify those students who need a stronger background in object oriented development and coding practices.
Recitations should continue host lectures on UML examples in the first few weeks and lectures on Object Oriented code practices in the following weeks. A recitation on research in software engineering should be added to support CO-COP4331-7. Alternatively, modules from the UCF library could be completed to support this course outcome.
RS-4. Dependencies and Impact on Other Courses:
ANS: This course is dependent on COP 3330 – Introduction to Object Oriented Programming. However, students continue to arrive in COP 4331 with minimal knowledge of UML and Object Oriented coding practices.
RS-5. Overall Summary Statement (reflect on closing of the loop in assessment process)
ANS: The proposed improvements should be implemented to ensure that more interaction between students and TA/Instructor occur. This should allow deeper insight into coding, communication, and UML issues throughout the semester. Additionally, we should be able to measure an individuals’ contribution to the main project. There is room for improvement with the semester project to support the concepts taught in COP 4331 that are not considered in Senior Design. Now that Senior Design is becoming a requirement for both CpE and CS students, the emphasis of COP 4331 should be reevaluated.
Previously implemented improvements should continue to maintain the performance increase in CO-COP4331-1 and additional practice in coding should improve each student’s knowledge of Object Oriented Coding.