Project 2- Item Analysis

This project is based on a knowledge assessment included in the dataset for projects 1 and 2. Again, this data was collected on 240 undergraduates in the psychology 101 pool and consists of numerous variables thought to influence performance on a complex radar tracking task. One of the variables dealt with knowledge acquired through the study of a training program and manual. Twenty-five questions were used to assess task knowledge and these items are reproduced in the Appendix. These items are labeled dk1-dk25 in your data set and they have already been scored for you (0=incorrect; 1=correct). There is no criterion available for evaluating adequate vs. inadequate task knowledge. Using this data, answer the following questions.

  1. Compute the item difficulties. Evaluate the pattern of observed item difficulties. If you observe a problem, how would you deal with the issue?
  1. Compute the item discrimination indices based on the difference between high and low performance groups and the item-total correlation.
  1. Construct a table for the 25 items presenting the difficulty and discrimination indices.
  1. Do you observe any problems in the item discrimination indices? If you observe a problem, how would you deal with the issue? How similar is the information provided by these two discrimination indices?
  1. Do you see any relationship in this data between item difficulty and item discrimination?
  1. To interpret the scores, it would be useful to have measurement norms.
  2. Construct norms for this knowledge test using Z-score and T-scores.
  3. Interpret the 75th participant’s knowledge score using the T-score norms.
  4. Interpret the 200th participant’s knowledge score using the T-score norms.

Appendix

  1. Which of the following should be done after clearing the command center of the Victor Attack Pattern?
  2. Clear the rest in the same manner *
  3. Completely process remaining targets
  4. Always intercept and declare hostile
  5. Ignore them, the formation is no longer threatening
  1. What does a triangle-shaped target on your radar indicate?
  2. A similar-looking target will appear on the screen for only a few moments *
  3. The Victor Attack Pattern will come from the top left
  4. Two other similar targets will appear deep in the perimeter
  5. The Ulysses Attack Pattern will appear outside the perimeter
  1. What do red diamond targets signify?
  2. Team targets
  3. Individual targets *
  4. Spy planes
  5. High-priority targets
  1. What marks the defensive perimeter?
  2. A single circle at the center of the radar
  3. A ring of circles in the middle of the radar screen*
  4. It is invisible at 256nm
  5. It is invisible at 1024nm
  1. Under the CLSS menu, what does a Communication Time of “801” suggest?
  2. Military
  3. Ambiguous
  4. Rebel *
  5. Civilian
  1. Under the CLSS menu, what does “radar” Electronic Warfare suggest?
  2. Rebel
  3. Civilian
  4. Military *
  5. Ambiguous
  1. Under the CLSS menu, what category is a target with a Communication Time of “400”, “none” on Electronic Warfare and “platform” for “Intelligence”?
  2. Rebel
  3. Civilian
  4. Military*
  5. Ambiguous
  1. Under the TYPE menu, what does “weak” Single Strength suggest?
  2. Ambiguous
  3. An airplane *
  4. A helicopter
  5. A spy plane
  1. Under the TYPE menu, what does an Altitude of “4000” and a “Code Delta” Maneuvering Pattern suggest?
  2. Airplane
  3. Helicopter *
  4. Not enough information to make an assumption
  5. Spy plane
  1. Under the TYPE menu, what category is a target with “moderate” Signal Strength, “Fox Trot” maneuvering pattern, and an altitude of “5050”?
  2. An airplane *
  3. A helicopter
  4. Not enough information to make a decision
  5. A spy plane
  1. What is the best strategy to take when making a CLSS (Class) decision for an individual target?
  2. Verify that all three cues agree
  3. Verify that two cues agree *
  4. Verify only the Intelligence of the target
  5. Look at the Target Class option under this menu
  1. What should be done with a normal-priority target in the X-ray formation?
  2. Process completely
  3. Clear it *
  4. Intercept as hostile
  5. Intercept as friendly
  1. What should be done with a normal-priority target in the Ghost Attack?
  2. Process completely *
  3. Clear it
  4. Intercept as hostile
  5. Intercept as friendly
  1. What should be done with targets in the Ulysses Attack Pattern?
  2. Process completely *
  3. Clear them
  4. Clear only if high-priority
  5. Intercept only if high-priority
  1. What is the optimal zoom position on the radar screen?
  2. Zoom out five clicks from where the simulation begins (1024nm) *
  3. Position it is in when the simulation begins (32nm)
  4. Zoom in two clicks from where the simulation begins (8nm)
  5. It is relative to the where the target formation appears
  1. When making the final engage decision, targets identified as friendly should be ______and targets indicated by your sensors to be hostile should be ______.
  2. Cleared, Intercepted *
  3. Cleared, Engaged
  4. Engaged, Intercepted
  5. Intercepted, Cleared
  1. Where is the command center located in the Victor Attack Pattern?
  2. The center target
  3. Two targets away from (clockwise) the center target
  4. Directly next to (counter-clockwise) the center target *
  5. Directly next to (clockwise) the center target
  1. How do you process the command center of the Victor Attack Pattern?
  2. Clear only if high-priority
  3. Clear it
  4. Intercept only if high-priority
  5. Process completely *
  1. How do you process triangle-shaped targets?
  2. Clear only if it’s a team target
  3. Clear it
  4. Process completely as team targets*
  5. Intercept only if it’s an individual target
  1. How many high-priority targets are in the X-Ray Formation?
  2. It depends on the specific formation
  3. 2
  4. 3
  5. 1 *
  1. How do you process a high-priority target in the X-Ray Formation?
  2. Process completely *
  3. Clear it
  4. Intercept as hostile
  5. Intercept as friendly
  1. How many low-priority targets are in the Ghost Attack?
  2. It depends on the specific formation
  3. 2
  4. 3 *
  5. 1
  1. What should be done with a low-priority target in the Ghost Attack?
  2. Process completely
  3. Clear it *
  4. Intercept as hostile
  5. Intercept as friendly
  1. What is most important about processing targets in the Ulysses Attack Pattern?
  2. Speed
  3. Cuing the priority of the target
  4. Speed and accuracy *
  5. Accuracy and cuing the priority of the target
  1. What should be done with the command center in the Ulysses Attack Pattern?
  2. There is no command center in the Ulysses Attack Pattern *
  3. Clear it
  4. Intercept only if high-priority
  5. Process completely