STUDENT LEARNING ASSESSMENT PROGRAM

SUMMARY FORM AY 2006-2007

Degree and

Program Name:

Submitted By:

PART ONE

What are the learning objectives? / How, where, and when are they assessed? / What are the expectations? / What are the results? / Committee/ person responsible? How are results shared?
1. Demonstrate a working knowledge of the basic laws of physics in the areas of mechanics, electrodynamics, thermodynamics, and modern physics. / Indirect measures:
1a. Exit interview questionnaire given to physics majors. / 1a. On the scale 5=strongly agree, 4=agree, 3=neutral, 2=disagree, 1=strongly disagree, the expectation would be an average score greater than 3.
12 responses in 2005
8 responses in 2006
8 responses in 2007 / 1a. Question: The Physics program at EIU has given me an appropriate background in:
Mechanics
2005 Ave=4.1
2006 Ave=3.9
2007 Ave=4.0
E&M
2005 Ave=3.9
2006 Ave=4.0
2007 Ave=4.0
Thermodynamics
2005 Ave=3.8
2006 Ave=3.5
2007 Ave=3.3
Quantum Mechanics
2005 Ave=3.7
2006 Ave=3.4
2007 Ave=3.7 / The Assessment Committee is responsible for analyzing and updating assessment tools and for analyzing data and reporting the results to the Physics Department and to the Curriculum Committee for possible revision of the curriculum.
Data is also solicited from the Physics 4000/4010 Seminar instructor and the Physics 4710 Advanced Lab instructor.
1. continued. / Direct measures:
1b. ETS Major Field Exam given by the Chair of the Physics Department. / 1b. The expectation is that the average Eastern student score would be greater than 50% of other schools. / Only 4 majors took the exam so ETS does not provide a topical breakdown and institutional average, but the individual results are as follows:
Intro Physics sub scores:
Student Score (National % of students below)
34 (15%)
44 (40%)
29 (5%)
49 (50%)
Advanced Physics sub scores:
Student Score (National % of students below)
55 (60%)
25 (1%)
49 (50%)
49 (50%)
Last year’s data:
Percentage of schools at or below Eastern’s average:
Classical Mechanics & Relativity =57%
E&M = 81%
Optics, Waves, & Thermo = 90%
QM & Atomic = 80%
Differences between last year and this year may be attributable to:
Last year’s exam was taken in class. This year’s exam was taken at the Chair’s request, so students may not have been as serious about it. The student receiving the lowest score left early. Last year, students were solving and presenting problem solutions in seminar which may have better prepared them.
On a positive note, EIU students did much better in the advanced physics topics than the intro subjects, perhaps because the material was more recently studied.
1. continued. / 1c. Force Concept Inventory Exam / The expectation is that the score would increase as students take more Mechanics. An ambitious goal would be to have a gain of 30 to 40%. / Fall 2006, the FCI was given as a pre and post exam to students in one section of PHY 1352, the lab for the introductory mechanics course for majors. The average pre-score was 38%, while the average post-score was 46%. The average gain on the FCI, defined as (post score – pre score) divided by (the number incorrect on the pre-test), was 14% +/- 6%. This compares to a typical range of 18% to 28% for traditional lecture delivery, versus 34% to 69% for interactive instruction techniques. However, the sample size here was only 7 students.
Last year’s FCI average scores from PHY 1000 were as follows:
For those having:
no mechanics = 23.5%
through 1351 = 52.7%
through 2390 = 73.7%
The apparent 1351 gain last year was 38%.
1d. Conceptual Survey of Electricity and Magnetism / 1d. This concept exam was given to PHY 1161 students who are pre-med, bio, industrial tech, and geology majors. No definite expectations have been determined, but one study saw an average gain of about 25% with an algebra based course. / The average gain:
(post – pre)/(pre incorrect) was 8%.
In the future, we hope to use the CSEM with our majors.
1e. For Teacher Certification majors: the ICTS exam / >80% should pass the exam with the average institution score on sub areas 5 & 6 greater than or equal to the state average.
(5:Motion,Forces,&Waves) (6:Heat,Elec.,Mag.&Modern) / SP06: 1 of 1 passed
(5) EIU=292, State=245
(6) EIU=300, State=236
1f. For the B.S. in Engineering Cooperative degree with UIUC:
Since our students spend the last two years on that campus, the rate at which our students graduate from that program should be an indication of how well they have learned the pre-requisite math and physics at EIU. / 1f. The expectation is that more than 80% of EIU students transferring into UIUC’s various engineering programs should complete the degree. / 1f. Of the 80 students in pre-engineering that transferred to UIUC from EIU since 1998, 90% have graduated or are in good standing in the program.
2. Students will be able to perform physics experiments using modern equipment and analyze them using appropriate mathematical techniques. / Indirect measures:
2a. Exit interview questionnaire. / 2a. On the scale 5=strongly agree, 4=agree, 3=neutral, 2=disagree, 1=strongly disagree, the expectation would be an average score greater than 3.
12 responses in 2005
8 responses in 2006
8 responses in 2007 / 2a. Question: The Physics program at EIU has given me the ability to:
calculate the experimental error in real physical situations.
2005 Ave=4.0
2006 Ave=3.6
2007 Ave=3.7
use basic experimental apparatus for studying physical phenomena.
2005 Ave=3.9
2006 Ave=3.8
2007 Ave=4.3
Direct measures:
2b. Analysis of Advanced Lab results. / 2b. The expectation is that at least 75% of students would perform at the >=80% level. / 2b. 60% of students reached that level.
2c. (For Science with Teacher Certification, Specialization in Physics) Analysis of a lab submitted during PHY 3500. / 2c. The expectation is that at least 75% of students would perform at the >=90% level. / 1 of 1 student performed at an acceptable level.
2d. For the B.S. in Engineering Cooperative degree with UIUC:
Since our students spend the last two years on that campus, the rate at which our students graduate from that program should be an indication of how well they have learned the pre-requisite math and physics at EIU. / 2d. Of the 70 students in pre-engineering that transferred to UIUC from EIU since 1998, 90% have graduated or are in good standing in the program. / 2d. Of the 80 students in pre-engineering that transferred to UIUC from EIU since 1998, 90% have graduated or are in good standing in the program.
3. Demonstrate a proficiency in mathematics up to and including differential equations. / Indirect measures:
3a. Exit interview questionnaire. / 3a. On the scale 5=strongly agree, 4=agree, 3=neutral, 2=disagree, 1=strongly disagree, the expectation would be an average score greater than 3.
12 responses in 2005
8 responses in 2006
8 responses in 2007 / 3a. Question: The Physics program at EIU has given me the ability to:
use mathematics in the solution of real physics problems.
2005 Ave=4.3
2006 Ave=4.0
2007 Ave=4.3
Direct measures:
3b. ETS Major Field Exam / 3c. The expectation is that the average Eastern student score on the Advanced section would be greater than or equal to 50% of other schools. / Four students took the ETS exam this year with the following results:
Student Score (National % of students below)
55 (60%)
25 (1%)
49 (50%)
49 (50%)
3c. For the B.S. in Engineering Cooperative degree with UIUC:
Since our students spend the last two years on that campus, the rate at which our students graduate from that program should be an indication of how well they have learned the pre-requisite math and physics at EIU. / 3c. Of the 70 students in pre-engineering that transferred to UIUC from EIU since 1998, 90% have graduated or are in good standing in the program. / 3c. Of the 80 students in pre-engineering that transferred to UIUC from EIU since 1998, 90% have graduated or are in good standing in the program.
3d. For the B.S. in Engineering Cooperative degree with UIUC:
Mathematics Component of Final Exam in PHY 2400 / 3d. The expectation would be that at least 33% of students would demonstrate a proficiency in mathematics through differential equations at the 80% correct level, which is an appropriate level for entry into UIUC’s engineering program. / 5 of 15 students reached the >= 80% level.
4. Students will demonstrate the ability to communicate technical material effectively in speaking and writing. / Indirect measures:
4a. Exit interview questionnaire. / 4a. On the scale 5=strongly agree, 4=agree, 3=neutral, 2=disagree, 1=strongly disagree, the expectation would be an average score greater than 3.
12 responses in 2005
8 responses in 2006
8 responses in 2007 / 4a. Question: The Physics program at EIU has given me the ability to communicate effectively, both verbally and in writing.
2005 Ave=3.8
2006 Ave=3.8
2007 Ave=4.7
Direct measures:
4c. Analysis of presentations given in Physics seminar. / 4c. The expectation is that at least 75% of students would demonstrate the ability to communicate technical material effectively in speaking. / 3 of 6 students performed at an acceptable level.
Direct measures:
4d. Analysis of Advanced Lab results. / 4d. The expectation is that at least 75% of students would perform at the >=80% level. / 4d. 60% of students reached that level.
4e. (For Science with Teacher Certification, Specialization in Physics) Analysis of a presentation given in PHY 3500.
4f. (For Science with Teacher Certification, Concentration in Physics) Analysis of a lab submitted during PHY 3500. / 4e,f. The expectation is that at least 75% of students would demonstrate the ability to communicate technical material effectively in speaking and writing. / A presentation rubric was used to evaluate the performance of the one PHY 3500 student. (rubric attached.)
1 of 1 students demonstrated the ability to orally communicate technical material effectively.
1 of 1 students demonstrated the ability to communicate effectively in writing.

(Continue objectives as needed. Cells will expand to accommodate your text.)

PART TWO

Describe what your program’s assessment accomplishments since your last report was submitted. Discuss ways in which you have responded to the CASA Director’s comments on last year’s report or simply describe what assessment work was initiated, continued, or completed.

Since last year’s report, a grading rubric was tried in one section PHY 1352, an introductory lab course for majors. The rubric proved useful in increasing student awareness of the elements that are required in a lab report, but no general trend in scores was observed. The rubric is attached to this report. The use of the rubric was time consuming. Some version of the rubric will be tried again in the fall. An error analysis post quiz was given in this section of PHY 1352, as well. The average score, 58%, was disappointing; however, the high score of 83% was respectable.

Two items were added to the exit survey:

Should PHY 1351, 1361, 1371 have a one-hour of discussion each week (to discuss homework, etc.) to go along with the three hours of lecture and three hours of lab?

86% of students in the survey responded in the affirmative.

Should the department implement a freshman orientation course to provide information about how to study physics, jobs in the major, research opportunities with faculty, etc.?

71% of students responded in the affirmative.

The survey is attached.

The assessment committee has also worked on an oral presentation rubric that could be used in seminar or in student research presentations.

PART THREE

Summarize changes and improvements in curriculum, instruction, and learning that have resulted from the implementation of your assessment program. How have you used the data? What have you learned? In light of what you have learned through your assessment efforts this year and in past years, what are your plans for the future?

The curriculum committee will be asked to consider the results of the two questions posed on the survey regarding the possibility of discussion sessions or some other change that would make classes more interactive, and the possible addition of a freshman seminar course. The department has also purchased two sets of TurningPoint response systems that several faculty have shown interest in using. These may help to make existing lecture classes more interactive.

The ETS Major Field Exam remains a convenient way to assess student problem solving ability in various areas of physics; however, motivating students to perform at their best is a problem. The solution to this problem is still undetermined.


PHY 3500 Practicum Rubric v.1

Pre-lab Presentation / 3 / 2 / 1 / 0
Purpose / Clearly stated purpose. Stated purpose is relevant to the lab being done. / Briefly stated purpose. Stated purpose is relevant. / Purpose stated, but not clearly. The implied purpose is relevant. / Purpose not stated or stated purpose is irrelevant.
Theory / The Physics and Mathematics related to the lab are clearly, completely and accurately developed. / The Physics and Mathematics related to the lab are briefly, but accurately developed. / The Physics and Mathematics related to the lab are not clearly developed. / The Physics and Mathematics related to the lab are not developed or are developed inaccurately.
Procedure / a. Steps for completing the lab are clearly and orderly explained. b. Procedures are demonstrated using best techniques. c. Possible blunders are pointed out. / One of a, b, or c are deficient. / Two of a, b or c are deficient. / a, b and c are all deficient.
Error Analysis / The mathematics related to error analysis is clearly, completely and accurately developed. / The mathematics related to error analysis is briefly, but accurately developed. / The mathematics related to error analysis is not clearly developed. / The mathematics related to error analysis is not developed or is developed inaccurately..
Checking for Understanding / Frequently & consistently asked questions and or observed skill of students / Occasionally asked questions and/or observed skill of students. / Asked vague or nonspecific questions (“Do you understand?”); minimal observation of skill / Did not check for understanding during lesson through questions or observation