Stevens Institute of Technology
Howe School of Technology Management
AACSB
ASSURANCE OF LEARNING
Undergraduate BS in Business
LEARNING GOAL # 1
Students can communicate effectively in written and oral communications.
Responsibility: Zvi Aronson & Andrew Stein
December 2016
Table of Contents
1. INTRODUCTION: COMMUNICATION SKILLS EDUCATION FOR B&T UNDREGRADUATES 4
2. LEARNING OBJECTIVES AND TRAITS 5
3. RUBRICS 6
4. ASSESSMENT PROCESS 9
5. RESULTS OF LEARNING GOAL ASSESSMENT - INTRODUCTION 10
6. RESULTS OF ASSESSMENT: SPRING 2007 11
7. RESULTS OF ASSESSMENT: SPRING 2008 14
8. SPECIFIC STEPS TAKEN IN FALL 2008 16
9. RESULTS OF ASSESSMENT: SPRING 2009 17
10. SPECIFIC STEPS TAKEN IN FALL 2009 20
11. RESULTS OF ASSESSMENT: SPRING 2010 21
12. SPECIFIC STEPS TAKEN IN FALL 2010 23
13. RESULTS OF ASSESSMENT: SPRING 2011 24
14. RESULTS OF ASSESSMENT: FALL 2013 26
15. RESULTS OF ASSESSMENT: FALL 2014 29
16. RESULTS OF ASSESSMENT: FALL 2015 32
17. RESULTS OF ASSESSMENT: FALL 2016 35
18. OUTCOMES: BSB LEARNING GOAL # 1 AFTER ROUNDS OF ASSESSMENT 38
19. CLOSE LOOP PROCESS – CONTINUOUS IMPROVEMENT RECORD 40
In subsequent years in addition to the above, this assessment process will also be conducted earlier in the 4-year program during either sophomore or junior year in order to provide ample time opportunity for remedial training prior to graduation. 40
Sampling: Rubrics are completed for a sample of at least 30 students for the course. 40
Closing the Loop: Actions taken on specific objectives 40
APPENDIX A 48
ASSESSMENT EXERCISE 48
APPENDIX B 50
ASSESSMENT DATA REVIEW 50
APPENDIX C. PREVIOUS PRESENTATION RUBRIC 52
LEARNING GOAL #1: Students can communicate effectively in written and oral communications.
Students will be able to write effectively.
Students will be able to deliver presentations effectively.
1. INTRODUCTION: COMMUNICATION SKILLS EDUCATION FOR B&T UNDREGRADUATES
Humanities Courses (100 level courses): All students are required to take 4 lower level humanities courses as part of their overall requirements within the program. These courses require 6 writing assignments per semester (2-3pages), plus one oral communication assignment. Each of these exercises is evaluated assessing student performance in written and oral communications. These courses are conducted by the College of Arts and Letters, and are dependent upon individual student course schedules, so it is difficult to comment on specific performance results other than to say that is being accomplished in that College for our students.
Notes:
1. Above also appears in Section 8 of the Learning Goal write-up.
2. “Additionally, throughout all of the Business and Technology Program courses, as indicated on the B&T Program curriculum alignment map, numerous written reports and oral presentations are developed and evaluated in fulfillment of both the specific course requirements and learning goal assessment process.
Specifically, students are assessed both as to their written and oral presentations skills in BT402 Business Plan VIII: Plan Perfection and Presentation, the capstone course that brings together everything they have learned as to core business knowledge and personal self-development.”
3. Next Action Steps:
May, 2009 Major assessment of learning goal #1 conducted in BT402 course.
Both as to written and oral communications skills
Section 8 of this Learning Goal write-up to be modified.
2. LEARNING OBJECTIVES AND TRAITS
Learning Goal:B&T – 1 / Students can communicate effectively in writing and oral presentations.
Learning Objectives
Objective 1: / Students will be able to write effectively
Traits
Trait 1: / Logical flow
Trait 2: / Grammar and sentence structure
Trait 3: / Spelling and word choice
Trait 4: / Development of ideas
Objective 2: / Students will be able to deliver presentations effectively
Traits
Trait 1: / Organization and logic
Trait 2: / Voice Quality
Trait 3: / Physical Presence
Trait 4: / Use of slides to enhance communications
Trait 5: / Transitions/Time Management/Q&A
21
3. RUBRICS
B&T Program Goal 1: RUBRIC #1B&T – 1 / Students can communicate effectively in writing and oral presentations
Objective 1: Students will be able to write effectively
Trait / Poor / Good / Excellent
Value / 0 / 5 / 10
Trait 1: / Logical flow / Unclear introduction or conclusion. Does not use a sequence of material to lead reader through the paper. Draws illogical conclusions / Develops ideas through effective use of paragraphs, transitions, opening and concluding statements. Generally well structured to suggest connection between sub-topics. / Maintains clear focus, uses structure to build the paper's conclusions. Presents analysis using sequence of ideas, clarity of flow and continuous voice or point of view.
Trait 2: / Grammar and sentence structure / Frequently uses inappropriate grammar and incomplete or poorly structured sentences which interfere with comprehension. / Generally complies with standard English and grammar and sentence usage. / Sophisticated use of English language, using varied sentence structured, phrasing and cadence. Grammar is error-free
Trait 3: / Spelling and word choice / Frequent misspellings. Poor or limited choice of words for expression ideas. / Has proofread or checked spelling, and uses vocabulary correctly. Minor errors. / Demonstrates good use of words to support written expression of topic. Spelling is error-free.
Trait 4: / Development of ideas / Many unsupported statements offered. Uses flawed or unclear reasoning. / Most statements supported, ideas explained with examples and written with sufficient explanation. / Shows thoughtful reasoning and explores alternatives. Uses existing, supported ideas to develop well-formed, readable output.
Criterion: / Does not meet expectations: 0 – 15; Meets: 16-30 ; Exceeds: 31-40
Objective 2 / Students will be able to deliver presentations effectively
Trait / Poor / Good / Excellent
Value / 0 / 5 / 10
Trait 1: / Organization and logic / Fails to introduce topic; no evidence of or poor logical flow of topic. / Prepares listeners for sequence and flow of topic. Loses place occasionally but flow and structure are still clear. / Engages listeners with overview, guides listeners through connections between sections, and alerts audience to key details and concepts.
Trait 2: / Voice Quality / Cannot be heard or understood well due to volume, mumbling, speed, monotone delivery, and/or heavily accented English. / Clear delivery with well-modulated voice. Displays some confidence and enthusiasm, but may also contain flatter periods or sound overly rehearsed. / Exemplary delivery, with a voice that sounds fully engaged, conveys enthusiasm and confidence, and relates to the audience well.
Trait 3: / Physical Presence / Turns away from audience or uses distracting gestures, such as pacing or tugging clothing. Speaker seems stiff, awkward or uncomfortable. Little eye contact. / Speaker is relaxed in front of the room and keeps distracting movements and gestures to a minimum. Generally faces audience and makes eye contact. / Speaker’s body language is superb and fully engages the room. Strong, consistent eye contact to the entire audience. Uses confident gestures to underscore key verbal points.
Trait 4: / Use of slides to enhance communications / Misspelled, too busy, too much text, too many slides for allotted time, and/or poor use of graphics like charts. / Slides are readable, containing a reasonable amount of material per slide. Good use of graphics or illustrations. / Slides are well written/designed, engaging to the audience, and used as support to verbal content presentation.
Trait 5: / Transitions
Time Management
Q&A / Transitions are awkward or non-existent. Speakers go over time limits. Answers are disorganized or non-responsive. / Transitions are smooth. Speakers generally stay within time limits. Speakers respond to questions well and provide sufficient response. / Transitions are professional and very smooth. Speakers respond convincingly and address all aspects of question.
Criterion: / Does not meet expectations: 0 – 19; Meets: 20-35 ; Exceeds: 36-50
21
4. ASSESSMENT PROCESS
Where and when measured? / How measured? / CriterionInitially, assessed at the end of Term 8 (2nd semester, senior year) in Course BT402
In subsequent years in addition to the above, this assessment process will also be conducted earlier in the 4-year program during either sophomore or junior year in order to provide ample time opportunity for remedial training prior to graduation. / Student presentations are graded for both oral PowerPoint presentations and for written reports, consistent with the communications evaluation standards established for the course.
Student evaluations will be completed by a panel of business faculty members.
Sampling: Rubrics are completed for a sample of at least 30 students for the course. / For both the oral and written evaluations for presentation skills, 80% of students must receive a grade of “A” or “B”. Students receiving “C” or “D” grades will be given a complete evaluation summary for areas that need improvement.
5. RESULTS OF LEARNING GOAL ASSESSMENT - INTRODUCTION
The results of the initial learning goal assessments carried out to date are included below.
Explanation
Each learning goal has a number of learning objectives and performance on each objective is measured using a rubric that in turn contains a number of desired “traits”. Students are scored individually on each trait.
The grading sheets for each student are used to develop a Summary Results Sheet for each learning goal objective. A selection of these Summaries is included below.
The first table in the Summary Results Sheet for a learning objective and trait gives the counts of students falling in each of the three categories:
- Does not meet expectations
- Meets expectations
- Exceeds expectations
The right-hand column in the table is used to record the average score of the students on each trait. This table provides an indication of the relative performance of students on each trait.
The second table on each sheet provides the counts of students who fall in each of the above three categories for the overall learning objective.
The person doing the assessment provides explanatory comments and recommendations on the bottom of the Results Summary Sheet. The recommendations improve content or pedagogy changes for the next time the course is given.
6. RESULTS OF ASSESSMENT: SPRING 2007
LEARNING GOAL #: 1 Students can communicate effectively in writing and oral communications.
LEARNING OBJECTIVE # 1: Students will be able to write effectively.
ASSESSMENT DATE: May 1-15, 2007 ASSESSOR: Prof. Skown
NO. OF STUDENTS TESTED: 14/26 with all data available.
Learning Goal Traits / Not Meet Expectations / MeetExpectations / Exceed Expectations / Avg. Grade on Trait
1: Logical Flow / 0 / 8 / 6 / 7.1
2: Grammar and sentence structure / 0 / 3 / 11 / 8.9
3: Spelling and word choice / 0 / 2 / 12 / 9.28
4: Development of ideas / 0 / 7 / 7 / 7.5
Average Grade (Maximum 10) / 8.19
Not meet Expectations / Meets Expectations / Exceeds Expectations
Total Students by Category
(Based on Average score across all traits) / 0 / 5 / 9
Students meeting or exceeding expectations: / 100%
COMMENTS:
We feel that our students are doing an acceptable job in putting together professional, written business reports. But there still is room for improvement. One area that needs improvement is better organization and logical flow to written reports. Too often, the student jumps from one topic to another, without clear transitions and stepping stones to the next topic. For the most part, logical conclusions are reached. Grammar, sentence structure, and spelling is acceptable and good in most papers. As to development of business ideas, supporting material and data could be stronger.
REMEDIAL ACTIONS:
We will continue to emphasize these weaknesses and work with students, particularly in the earlier years. Throughout the spine courses, there are amble opportunities to stress these traits in student reports and presentations. Accordingly in all the business planning courses, we are adding to the curriculum more time spent on these subjects. This will take place in courses BT101, BT102, BT302, and BT401.
RESULTS OF ASSESSMENT: SPRING 2007 (CONTINUED)
LEARNING OBJECTIVE # 2: Students will be able to deliver presentations effectively.
ASSESSMENT DATE: May 1-15, 2007 ASSESSOR: Prof. Skown
NO. OF STUDENTS TESTED: 26 with all data available.
Learning Goal Traits / Not Meet Expectations / MeetExpectations / Exceed Expectations / Avg. Grade on Trait
1: Logic and Organization / 0 / 20 / 6 / 6.15
2: Voice and body language / 0 / 18 / 8 / 6.53
3: Use of slides to enhance communication / 0 / 17 / 9 / 6.73
4: Ability to answer questions / 2 / 13 / 11 / 6.73
5: Content / 2 / 15 / 9 / 6.34
Average Grade (Maximum 10) / 6.49
Not meet Expectations / Meets Expectations / Exceeds Expectations
Total Students by Category
(Based on Average score across all traits) / 1 / 17 / 8
Students meeting or exceeding expectations: / 96%
COMMENTS:
Faculty believes that the B&T students do a creditable job in making effective business presentations. However, there is always room for improvement. One area of concern that needs additional reinforcement is a good organization and flow to the presentation, particularly in a good opening to the presentation. Also, while students are able to answer questions effectively, we need to work on preparing students to think about the questions they might be asked. Some of this might be due to language barriers, as a few are still struggling a bit in basic English understanding and speaking ability.
REMEDIAL ACTIONS:
One area that we are placing greater emphasis on in the course instruction will be in the area of presentation organization and introducing the presentation subject to your audience. During April, a month prior to final senior presentations, we will be reviewing with students these weaknesses and working on improving student performance.
7. RESULTS OF ASSESSMENT: SPRING 2008
LEARNING GOAL #: 1 Students can communicate effectively in writing and oral communications.
LEARNING OBJECTIVE # 1: Students will be able to write effectively.
ASSESSMENT DATE: Spring 2008 ASSESSOR: Prof. Skown
NO. OF STUDENTS TESTED: 14/26 with all data available.
Learning Goal Traits / Not Meet Expectations / Meet Expectations / Exceed Expectations / Avg. Grade on Trait1: Logical Flow / 2 / 26 / 12 / 6.67
2: Grammar and sentence structure / 0 / 10 / 30 / 8.65
3: Spelling and word choice / 0 / 11 / 29 / 8.55
4: Development of ideas / 3 / 21 / 16 / 6.50
Average Grade (Maximum 10) / 7.60
Not meet Expectations / Meets Expectations / Exceeds Expectations
Total Students by Category
(Based on Average score across all traits) / 0 / 21 / 19
Students meeting or exceeding expectations: / 100%
Comments. As first measured at the end of the 2007 academic year, we continue to feel that students overall are doing an acceptable job in preparing professional business reports. The same student weaknesses continue to be noticed one year later, i.e., the end of the 2008 academic year. That is, the student’s report organization and logic can be improved, as students too often jump from one topic to another, without clear transitions and stepping stones to the next topic. For the most part, logical conclusions are reached. In some cases, grammar and sentence structure remains weak. Spelling is reasonably good. Overall report content in many cases could be stronger.