Assessment Plan

Department of Information Technology

Radford University

May 2015

Table of Contents

1. Introduction

2. Assessment Mechanisms, Procedures, and Reports

3. Assessment Timeline and Personnel

4. Program Educational Objectives

5. Student Outcomes

6. Review of Course Outcomes and CO to SO Mappings

6.1 Course Outcomes for New Courses

7. Review of Program Educational Objectives and Student Outcomes

8. Review of Assessment Process

9. Assessment Timeline

1. Introduction

Thisplan, which replaces the June 2009 Department of Information Technology Assessment Plan (which itself replaced the June 2006 Assessment Plan for the Computer Science Program),describes the department’s process forcontinuous improvement of its programs. The plan describes the mechanism used to assess each program educational objective (PEO) and each student outcome (SO) and the procedure used to determine the extent to which the program meets each PEO and SO. It also describes other assessment activities, including the mechanism and procedure for reviewing the PEOs and SOs themselves, for reviewing the assessment plan itself, and for making recommendations for improvement.

The 2006 Assessment Plan was originally patterned after similar documents developed and published by the U.S. Naval Academy Computer Science Program, and information from Pacific Lutheran University was also helpful. Assessment information found at the University of Delaware provided input into the development of course and program assessment mechanisms and procedures.

2.Assessment Mechanisms, Procedures, and Reports

The table below lists the mechanisms used to assess specific PEOs and SOs, and itsummarizes the procedure used to determine the extent to which each PEO and SO is being met. It indicates the person or group responsible for conducting the assessment and for preparing a report that summarizes the results of the assessment, as well as the group responsible for evaluating the results and when the activity is to occur. The assessment committee has oversight of the entire assessment process.

Assessment
Mechanism / Responsible for Conducting the Assessment and for Preparing a Report Summarizing Results of the Assessment / PEO andSO
Assessed / When Assessment Conducted / Responsible for Evaluating Assessment
Alumni Survey / Assessment
Committee (AC) / PEO 1-6 / Every Spring / Leadership Team
IAB Meeting / AC / PEO 1-6 / Every spring / Leadership Team
Course/Student Outcome Assessment / Course Instructors and
Coordinators conducts; AC summarizes / All SOs / Every Semester / Leadership Team
Senior Survey / Chair conducts;
AC summarizes / PEO 1, 3 / Every Semester / Leadership Team

The table below lists the mechanisms used to review course outcomes (COs), CO to SO mappings, PEOs, SOs, and SO to PEO mappings and summarizes the procedure for using collected data to develop recommendations for changes in the COs, CO to SO mapping, PEOs, and SOs.

Assessment
Mechanism / Responsible for Conducting the Assessment and for Preparing a Report Summarizing Results of the Assessment / Item Assessed / When Assessment Conducted / Responsible for Evaluating Assessment
Report
Faculty Survey / AC / Infrastructure and Student Readiness / Every Spring / Leadership Team
Review of Outcomes for Individual Courses / Curriculum Committee and Course Instructors and Coordinators / Course outcomes for all courses / Every three years / Curriculum Committee, Course Coordinators, and AC
Review of CO to SO Mappings / AC / All CO to SO mappings / Every three years / Leadership Team
Review of PEOs and SOs / AC / All SOs and PEOs / Every three years / Leadership Team
Review of Assessment Plan / AC / Assessment Plan / Every three years / Leadership Team and AC

Assessment Reports

After each assessment mechanism listed in the two tables above is used, a report is to be prepared that

  1. describes the procedure followed in the assessment,
  1. specifies which PEOs, SOs, or other items (eg review of COs) were addressed by the assessment mechanism, and
  1. summarizes data that can be used to determine the extent to which each relevant PEO and/orSOis being met or to evaluate the state of other items

The reports can also contain recommendations for changes to thePEOs, SOs, program, and/or the assessment process.

Copies of reports are to be stored in the department accreditation archive.

Assessment Summary and Leadership Team Recommendations

The table below lists the assessment mechanism for reviewing the results from the assessment mechanisms listed in the first two tables above, and it summarizes the procedure for conducting these assessments.

Assessment
Mechanism / Responsible for Conducting the Assessment and for Preparing a Report Summarizing Results of the Assessment / Item Assessed / When Assessment Conducted / Responsible for
Evaluating Assessment
Assessment Summary / Assessment Committee / All other assessment reports / Every summer / Leadership Team
Leadership Team Review / Leadership Team / Assessment Summary and, as needed, other assessment reports / Every fall / Department Chair

Assessment Summary Report

The annual Assessment Summary Report contains a summary of the results given in the reports describing the assessment mechanisms in the first two tables above. It is to contain the following items:

  1. A chart showing the assessment mechanism used during the year and the PEOs,SOs, and other itemsthat were addressed by each of these mechanisms.
  1. A summary of the data from the other assessment reports organized by the PEOs, SOs, or other items that the data describes.
  1. A listing of any course based assessments in the current year or in any earlier years in the 3-year cycle whose value fell below 3.5.
  1. A listing of any recommendations for changes to the program, PEOs, SOs, and/or assessment process. The listing is to be organized by what is recommended to be changed and the source of any recommendations should be noted.

The Assessment Summary Report should be given to the Leadership Teamas a basis for developing plans to effect continual improvement of the program and for evaluating the progress of the implementation of these plans.

Leadership Team Recommendations Report

Based on the data in the Assessment Summary Report and the other assessment reports, the Leadership Team should provide to the department chair a set of recommendations on ways to improve the department and its programs. In particular, the LT should analyze results from that year’s course-based assessment and should recommend whether action should be taken before the end of the three-year cycle. Actions could include reassessing the CO in the course earlier than scheduled in the cycle.

Recommendations should be organized by the party that the Leadership Team recommends should be responsible for responding to the recommendation. Recommendations should be accompanied by an indication of the assessment data that supported then. Where possible, three years worth of assessment data should be used in creating the report. This report should be a major factor in the chair’s charge to the department and its committees for the academic year.

3. Assessment Timeline and Personnel

Assessment Timeline

  1. Specific assessments should occur as described in the tables above:
  2. Every semester: course-based, senior survey, MFT
  3. Fall: Leadership Team review of results
  1. Spring and summer: faculty survey, alumni survey, IAB meeting
  1. Every three years: review of course-based results, course outcomes, CO/SO mapping, 3-year cycle, and Assessment Plan
  1. Reports on assessment results should be completed soon after the end of the spring semester.
  1. The Assessment Summary should be completed soon after the assessment result reports.
  1. The Leadership Team Recommendations should be completed early in the fall semester.

Assessment Personnel

The following individuals and groups participate in the Assessment Plan:

  • Chair - Chair of the Department of Information Technology
  • Assessment Committee (AC) – This committee, which is formed each fall by the Department, has primary responsibility for assessment activities for the department and the department.
  • Leadership Team (LT) –The primary function of this committee is to provide advice to the Dean, Chair, and Directors. Members of the committee include the department chair, the Program Directors, and past Department Chairs.
  • Course Coordinator – Each course offered by the department has a designated coordinator who is responsible for the course.
  • Curriculum Committee

4. Program Educational Objectives

At the time of the writing of this document, the current PEOs for the computer science and technology and information science and systems programs are as follows:

Within three to five years of graduation, alumni of the program are expected to:

  1. obtain positions where they analyze, design, develop, maintain, test, manage, or support real world computing systems.
  1. be valued employees as demonstrated by advancement in responsibility, position, or compensation, either within their company or at another firm.
  1. gain admission to graduate programs and complete advanced degrees, for those who wish to pursue graduate education.
  1. adapt and learn in response to advances in Information Technology.
  1. deal professionally with ethical and cultural situations encountered in the workplace.
  1. make significant contributions when working independently and when working with a team.

The last revision of the PEOs was approved by the department March 3, 2010 and most recently reaffirmed by department leadership team May 2015.

A listing of the PEOs is available from the department’s assessment web page:

The following table shows the PEOs that are assessed by each assessment mechanism. The Senior Survey is conducted every semester. The Alumni Survey is conducted annually. The Employer Survey is conducted every three years. Each mechanism is to assess the specifiedPEOseach time it is used.

Program Educational Objectives
Assessment Mechanism / 1 / 2 / 3 / 4 / 5 / 6
Alumni Survey / X / X / X / X / X / X
Employer Survey / X / X / X
Senior Survey / X / X

5.Student Outcomes

At the time of writing this document, the Student Outcomes (SO)used by the department’s programs are those provided by ABET for computer science and information systems programs, approved by the department September 17, 2008, and reaffirmed by the LT in May 2015. Student outcomes for certificates in Information Security and Web Development were developed by the department.

The program enables students to achieve, by the time of graduation:

  1. An ability to apply knowledge of computing and mathematics appropriate to the program's student outcomes and to the discipline.
  2. An ability to analyze a problem, and identify and define the computing requirements appropriate to its solution.
  3. An ability to design, implement, and evaluate a computer-based system, process, component, or program to meet, desired needs.
  4. An ability to function effectively on teams to accomplish a common goal.
  5. An understanding of professional, ethical, legal, security, and social issues and responsibilities.
  6. An ability to communicate effectively with a range of audiences.
  7. An ability to analyze the local and global impact of computing on individuals, organizations, and society.
  8. Recognition of the need for, and an ability to engage in, continuing professional development.
  9. An ability to use current techniques, skills, and tools necessary for computing practices.

For computer science programs:

  1. An ability to apply mathematical foundations, algorithmic principles, and computer science theory in the modeling and design of computer-based systems in a way that demonstrates comprehension of the tradeoffs involved in design choices.
  2. An ability to apply design and development principles in the construction of software systems of varying complexity.

For information systems programs:

The program outcomes are consistent with those accepted by the information systems community.

The program enables students to achieve the following attributes by the time of graduation:

jAn understanding of and an ability to support the use, delivery, and management of information systems within an Information Systems environment.

A list of the SOs is available at the department assessment web page:

Assessment of the SOs is accomplished by assessing student work in specific courses over a three-year cycle. For each year of the cycle, the Assessment Committee has specified whichSOs are to be assessed in which courses. Over the three years of the cycle, each SO is scheduled to be assessed at least twice. AnSO is selected to be assessed in a courseif that course has strong support for the SO. In a few cases where anSO has only a small number of courses that strongly support it, the SO is also assessed in a course that has only medium support for that SO. The table below shows the strength of support each course has for each SO at the time of this writing. If the strength of support table is updated in the future, the updated table will be found in the database available via the department assessment web page.

In the table below, a course has strong support (designated “S”) for a SO if it has one or more Course Outcomes (CO) that strongly supports that SO. A course has medium (or weak) support (“M” or “W”) if the strongest support that any of its CO has for that SO is medium (or weak). In a few cases a course is designated as having strong support for anSObecause it has five or more COs that have a medium mapping for that SO and none that have a strong mapping.

Course / Student Outcome
a / b / c / d / e / f / g / h / I / j-cs / j-is / k-cs
ITEC 110 / M / W / W / M / M / W / W / W / M / W
ITEC 120 / S / S / S / W / M / W / M / W / W / S
ITEC 122 / S / S / S / S / S
ITEC 220 / S / S / S / S / M / W / W / S / S / S
ITEC 225 / M / S / S / M / W / W / S / M / S / S
ITEC 226 / M / S / S / S
ITEC 227 / W / S / W / S / S
ITEC 281 / S / M / S / S
ITEC 310 / S / M / M / M
ITEC 315 / M / S / S / M / M / S / M
ITEC 320 / S / M / S / W / S / S / S
ITEC 324 / M / M / S / S / M / S
ITEC 325 / W / S / S / S / W / W / W / S / S / M
ITEC 335 / M / M / M / W / W / M / W / S / M
ITEC 340 / M / S / S / S / S / S / S / S / S
ITEC 345 / M / S / M / S / M / M / M / M
ITEC 350 / M / M / M / S / M / M / M / W / S / M / W
ITEC 352 / S / S / S / M / S
ITEC 360 / S / S / S / S / S / S
ITEC 370 / S / S / S / S / M / S / S / M / S / S / S
ITEC 371 / S / M / M / M / M
ITEC 380 / S / S / S / W / S / S / S
ITEC 410 / S / S / S / M / S / S / M / M
ITEC 420 / S / M / M / S
ITEC 425 / S / S / S / S / M / S / W / S / S
ITEC 430 / S / S / S / M / S / S / S / M / S
ITEC 441 / S / M / S / S / M
ITEC 442 / M / S / S / S / S / S / S / M
ITEC 445 / M / M / S / W / M
ITEC 451 / S / S / S / S / S / S
ITEC 452 / S / S / S / S / S / S / S / S
ITEC 455 / S / M / S
ITEC 460 / S / M / S / S / S / S
ITEC 471 / S / S / S / S / S / W / S / M / S
ITEC 472 / S / S / S / S / W / W / S / S / S
ITEC 480 / S / M / M / M / M / M
ITEC 485 / S / W / S / S
ITEC 490 / S / S / M / S
ITEC 495 / W / W / S / S / S

The assessments of particular student outcomes in particular courses are spread over a three yearcycle. The table below shows which outcomes are assessed in which courses in each year of the cycle. In the table, R and O designate a required or optional course, in that order, in a program.

Course / CS / IS / WD / Year 1 (2014-15) / Year 2
(2015-16) / Year 3
(2016-17) / Summary of
All Three Years
ITEC 110 / R / R / R / d / d
ITEC 120 / R / R / R / c,kcs / a / B / a,b,c,kcs
ITEC 122 / R / c,k-cs / b,a,jcs / a, b, c, jcs , kcs
ITEC 220 / R / R / R / i / a,c,e,kcs jcs / B / a,b,c,e,i,jcs,kcs
ITEC 225 / R / R / R / c,kcs, i / b,jis / b,c,i,kcs, jis
ITEC 226 / R / c, i / c, i
ITEC 227 / R / f, i / f, i
ITEC 281 / R / i / a, c / a, c, i
ITEC310** / O / O
ITEC 315 / O / O / O / i / c / B / b, c, i
ITEC 320 / R / O / O / a, jcs, i / c, kcs / a,c,i,jcs,kcs
ITEC 324 / R / O / O / i, c,kcs / c,i,kcs
ITEC 325 / O / R / d,i,jis / b,c / b,c,d,i,jis
ITEC 335 / O / O / i, jis / i, jis
ITEC 340 / O / R / R / g / a,jcs, c,kcs / e, i, jis / a,c,e,g,i,jcs,jis,kcs
ITEC 345 / R / R / R / b / e / H / b, e, h
ITEC 350 / O / O / O / d / i / d,i
ITEC 352 / R / O / O / a, jcs / b,c / a,b,c,jcs
ITEC 360 / R / O / O / b c,kcs / a,jcs,i / a,b,c,i,jcs,kcs
ITEC 370 / O / R / R / a,jcs,c,kcs / b,d,f,g,i / a,b,c,d,f,g,i,jcs,kcs
ITEC 371 / R / O / O / a,c / a,c
ITEC 380 / R / O / O / b / i / a,jcs,c,kcs / a,b,c,i,jcs,kcs
ITEC 410* / O / O / O
ITEC 420 / R / O / O / a,jcs / a,jcs
ITEC 425 / O / R / e,i / c,kcs / a,b,jis / a,b,c,e,i,jis,kcs
ITEC 430* / O / O / O
ITEC 441** / O / O
ITEC 442** / O / O
ITEC 445 / O / O / I / I
ITEC 451** / O / O
ITEC 452** / O / O
ITEC 455 / O / O / ai / Ai
ITEC 460* / O / O / O
ITEC 471** / O / O
ITEC 472** / O / O
ITEC 480* / O / O / O
ITEC 485 / R / R / e / g,jis / e,g,jis
ITEC 490 / R / R / R / h / e,g / F / e,f,g,h
ITEC 495 / R / R / jis / e,g / e,g,jis

* Not taught on a regular schedule. ** Not in assessment cycle.

Actual semester by semester assignments are posted on the department’s assessment web pages. In addition, a spreadsheet in the assessment archives will contain the current version of the cycle. Changes can be made to the spreadsheet to reflect needs such as new courses and mistakes that are made and discovered.

To assess a particular SO in a specified course, the instructor(s) and/or the coordinator for that course do the following:

  1. Select as an assessment instrument a course assignment that supports the SO.
  2. Seek approval of the assessment from an Assessment Committee representative
  3. Select a sample of students from all of the sections in the course (which could be all of the students in the course) to assess.
  4. Usethe provided heuristic to rate the quality of each student’s work on that assignment on a scale of 1 to 5. Ratings indicate a high (level 5), medium (level 3), or low (level 1) level achievement of a given student outcome. If a student does not clearly fall into one of those three categories, a rating between those levels is used. A heuristicfor each SO is available via the department assessment web page.
  5. Submitto the Assessment Committee representative results from the assessment on the form which is found on the assessment web site.

Assessment assignments for each course for each semester will be published in the department’s assessment web pages.

6. Review of Course Outcomes and CO to SO Mappings

Course outcomes and CO to SO mappings will be reviewed on a three year cycle. The review will be conducted by the curriculum committee, in conjunction with course coordinators and common instructors. Recommendations for proposed changes will be presented to the assessment committee and department for review and comment and approved by the department chair. Approved changes will be provided to the assessment committee for entry into the assessment database and for integration into the plans for assessment of the SOs.

CO to SO mappings will be reviewed on at least every three years by the Assessment Committee. The review will be evaluated by the Leadership team. See Assessment Timeline below.

6.1 Course Outcomes for New Courses

All proposals for new courses must be include course outcomes and CO to SO mappings. After a course is approved, its COs and CO to SO mappings will be provided to the assessment committee for entry into the assessment database and for integration into the plans for assessment of the SOs. See Assessment Timeline below.

7. Review of Program Educational Objectives andStudent Outcomes

Program educational objectives and student outcomes will be reviewed by the leadership team at least every three years. However, changes to PEOs can occur between reviews if assessment feedback initiates modifications. A change to the university mission will also trigger a review of the PEOs. Data for this review will be provided by the assessment committee and will include

  • an analysis of SO to PEO mappings to verify that each PEO has adequate support from SOs
  • CO to SO mappings to verify that each SO has adequate support from COs
  • results from annual surveys of seniors, alumni, employers, and faculty, particularly results from questions involving future direction of department and suggestions for improvement.

Proposed changes will be presented to the department for discussion and approval. During that discussion, continued alignment with the university mission should be verified. Approved changes will be provided to the assessment committee for entry into the assessment database and/or assessment web pages and for integration into the plans for assessment of the SOs. See Assessment Timeline below.