Bond and Fox, Chapter 3

The pathway

A graphic that presents information on the following . . . .

Person ability

Item difficulty

Precision

Item and person inconsistency

As we’ll see – having all the information in the pathway will give us much more control over measurement than, for example, just having the scalogram. P 38

Characteristics of items and scales

Unidimensionality – very important in Rasch modeling

Reliability –

Person reliability index – the expected correlation of person ordering we could expect if this sample of persons were given another parallel set of items measuring the same construct

Item reliability index – the expected correlation of item placements along the pathway if these same items were given to another sample of persons of the same size that behaved in the same way.

Basic quantities

Person ability – vertical positions of person symbols

Item difficulty – vertical positions of item symbols

Precision – sizes of person and item symbols – smaller means more precise

Fit – horizontal position of symbols

Ch 4 – Developing a test

The Bond Logical Operations Test (BLOT).

A test of childhood cognitive development suitable for administration to a whole class of students.

Items were taken one by one from Chapter 17 of Inhelder and Piaget (1958), The Growth of Logical Thinking.

The 35 items on the BLOT are identified as follows

01 Negation (to negate identity) ; Item labels courtesy of Trevor Bond

02 Reciprocal (to negate identity)

03 Implication

04 Incompatibility

05 Multiplicative compensation

06 Correlations

07 Correlations

08 Correlations

09 Conjunction

10 Disjunction

11 Conjunctive negation

12 Affirmation of p

13 Reciprocal exclusion

14 Probability

15 Reciprocal implication

16 Reciprocal (to negate identity)

17 Identity (to negate reciprocal)

18 Negation (to negate correlative)

19 Reciprocal (to cause disequilibrium)

20 Negation (to cause disequilibrium)

21 Correlative + negation > equilibrium

22 Reciprocal + negation > disequilibrium

23 Correlative + identity > disequilibrium

24 Coordination of two systems of reference

25 Complete negation

26 Complete affirmation

27 Negation of p

28 Non-implication

29 Affirmation of q

30 Equivalence

31 Negation of q

32 Negation of reciprocal implication

33 Probability

34 Coordination of two systems of reference

35 Coordination of two systems of reference

The test was administered to a group of 158 children. The results of the administration are below.

PSY 5950 L13 - 1

001 11111111110110101101011111111011111

002 11111111111111111111111111101111111

003 11010111111111011111011111101011111

004 11111111111111111111101111111111111

005 11111111111101111111011111111111111

006 11111111111110111101011111111111111

007 11111111111101111111011111111111111

008 11111111111111111111111111101011111

009 11111111111111111111111101111111111

010 11111111111111111111111111111001111

011 11111110111111111111111111111111111

012 11011111011111011111011111000110111

013 11111110111111111111011011111101111

014 11111110111111111111111111101001111

015 11111111111111011111010111101111111

016 11111111111101111101111111111111111

017 11111111111101111101111111111111111

018 11111111111101111111011111101110111

019 11111110111111111111111111111111111

020 11111111111111111110011111111110111

021 11101110111111111111111111101110111

022 11001111011101010111011111111111111

023 11111111111111111111111111111111111

024 11111111111111111111111111111101111

025 11111111111111111111011111111111111

026 11111111011111011111111110110010111

027 11111111111111111111111111111111111

028 11101111111111111111001101111011111

029 11111111111101111111110110111010111

030 11111111111111111101010110101111111

031 11111111111101111111011111111111111

032 00101110111111110111011111101101111

033 11111111011111011111011011111110111

034 11111111111111111111111111101111111

035 11111111111111111111111111111101111

036 01111111111101010101011010111001101

037 11011111111101111111011111111011111

038 11111111111110111111011111111011111

039 10011111101111011011011111111111111

040 11111111110111111111111111101011011

041 11111111110111001111011111101001111

042 11011111111111101111111111111101100

043 11111111111111111011111101101110111

044 11011111000110000111101011101100111

045 00111111111111111111010100111010111

046 11111111111111111111111111111111111

047 11111111011111110111110101111111111

048 11111110011101011111111111101100011

049 01110110110101111111011110110111111

050 11111111111101111111011101111111111

051 10010110110101101111110111111110011

052 11001101101101011111010101111011111

054 11111111110111011111011111111110111

055 11111111111101011111111111111111100

056 11111111110111111101011111101111111

057 11011110111101111110111111001011111

058 11001110111111011111011111111011111

059 11111111111111111111011111111111111

060 11111111111111011101011101110010111

061 11001110010111110111011111101110111

062 11110110111101111011110110101001111

063 11011110110110111111011111111110111

064 11111111111111011111011111111001111

065 11111111111111110011010111111111111

066 11111111111111111111011011111011011

067 11011111101110011111011011101011100

068 11111111111111011110011001111010100

070 11101101111101001001010101101111100

072 11011111010101111111011110111011111

073 11011111100101101111011101101111111

074 11101111111111111111011111101110111

075 10111111111111010001111100111011000

076 11011111001100111110010111111011111

077 11111111111101101111010111111011111

078 11000100111111011111011100101001111

079 00111111110111011111011100101111011

080 11111110101111010101110011111111100

081 11111111011111100111111111111111111

082 11011111111111111111111101111111111

083 11111111111111111111111111101111111

084 11111111111111111111011111111111111

086 00011111011101011110011110100011111

087 11011100111111011111111000101110100

088 11111111010110111111111111101111111

090 11111110111101100101011110101010111

091 11111101100101111111001100101000111

092 11111111111111111111011111111111111

093 11111111101101010101011111100011111

094 11111111111111111101111110101110111

095 01111111011111010011010101110011100

096 11011110111111111001011110001100010

097 11111111010101011101011100101110111

098 01101010000100011110010000100100011

099 11011111111111000101010110100110011

100 11100111111111001111011001011011111

101 11111111111111111111111011111111111

102 11111111111101111111111111111111111

103 11101111111101101001000101001000111

104 11101101111101111001011111101001011

105 11011100110110100101011110101101110

106 11011110011101110011010110110110111

107 11010110101101100001010111111011111

108 11010111011101000011011001010100111

109 11101010011111111111011111111110111

110 11111100111111111101110000101110011

111 11111110100101111101011001101000000

112 11011100110101101010001100010110111

113 00011111111101010010011111111011011

114 11000110001111110011111110101111111

115 11101110111111111111100000100111111

116 01011111111101001100011110000010111

117 00001110100101010111011011101111000

118 10001100010010010000010000010000111

119 00000100000001010001000010000000000

120 11111111110111111101011011111111111

121 10100111010111000001011001101110011

122 11001100100111011111011010101100100

123 10101010011111000001010010101011111

124 11111111101110111111010100101110111

125 11111101111101011101011100101100111

126 10011110001111111000010100111110100

127 11111111101100110101011110111011011

128 11111110111111001111010110101011111

129 11111110010111010001110111101011010

130 00010110110011110101111000001000111

132 10111111010101111111110111111000011

133 11111101101111111111011000101100110

131 00111100011110011001110011100010101

134 11001100000111010000110000101100101

135 11010110001111010100000100100100011

136 11001111111100011011011000101010100

137 00011110011100110011110010101010011

138 11101110100001000000001010000010100

139 10111010101001010000001000100100100

140 11001110110101110111011000000010000

141 11101110001100010111110000110111110

143 01010110110001000000000010010010110

144 01000110000000010011010001001000100

145 11010100000101010101011001111001101

146 11001110111100000000010111101010111

147 01011100100110000000100000001100110

149 11111111111111101111110100100000101

150 11111111111111011111011111101111111

151 10000100000001000100100110000000010

152 00011101111101011011011010101010100

153 11111110010111111111010111001110111

154 11001000001001101111001100101010011

155 11011111111111111111111111101110111

156 01101110011100110101011001101100101

158 11001101111101110111100110101111111

PSY 5950 L13 - 1

Think for a moment about the complex process that begins with this 158x35 collection of numbers (5530 in all) and ends with the output, tables, graphs, and interpretations that will be created from them.

A Regular analysis of the data in SPSS.

data list fixed /id 1-3 q1 to q35 5-39.

begin data.

001 11111111110110101101011111111011111

002 11111111111111111111111111101111111

156 01101110011100110101011001101100101

158 11001101111101110111100110101111111

end data.

compute totalscore = sum (q1 to q35).<---- This is the usual person estimate.

fre var=totalscore /format=notable /histogram.

This is a classic “easy” test, with scores “piled up” near the top of the range of possible scores.

reliability variables = q1 to q35 /summary=total.

Reliability Statistics
Cronbach's Alpha / N of Items
.876 / 35
Item-Total Statistics
Scale Mean if Item Deleted / Scale Variance if Item Deleted / Corrected Item-Total Correlation / Cronbach's Alpha if Item Deleted
q1 / 25.47 / 38.277 / .380 / .873
q2 / 25.47 / 38.345 / .355 / .873
q3 / 25.68 / 37.226 / .436 / .871
q4 / 25.56 / 37.765 / .398 / .872
q5 / 25.45 / 38.517 / .349 / .873
q6 / 25.37 / 39.496 / .207 / .875
q7 / 25.48 / 38.117 / .400 / .872
q8 / 25.70 / 36.775 / .509 / .870
q9 / 25.59 / 37.935 / .348 / .873
q10 / 25.53 / 37.539 / .467 / .871
q11 / 25.59 / 37.734 / .386 / .873
q12 / 25.39 / 38.294 / .558 / .871
q13 / 25.73 / 37.958 / .298 / .875
q14 / 25.47 / 38.949 / .213 / .876
q15 / 25.73 / 37.032 / .457 / .871
q16 / 25.52 / 38.520 / .273 / .875
q17 / 25.62 / 36.868 / .530 / .869
q18 / 25.55 / 37.349 / .487 / .870
q19 / 25.63 / 37.496 / .406 / .872
q20 / 25.46 / 38.116 / .429 / .872
q21 / 25.97 / 38.711 / .176 / .878
q22 / 25.44 / 38.423 / .385 / .873
q23 / 25.61 / 37.782 / .363 / .873
q24 / 25.59 / 37.129 / .498 / .870
q25 / 25.64 / 37.843 / .341 / .874
q26 / 25.69 / 36.847 / .501 / .870
q27 / 25.45 / 38.008 / .468 / .871
q28 / 25.85 / 37.661 / .338 / .874
q29 / 25.50 / 37.876 / .430 / .872
q30 / 25.74 / 38.113 / .269 / .876
q31 / 25.59 / 37.976 / .340 / .874
q32 / 25.75 / 36.898 / .474 / .871
q33 / 25.49 / 38.534 / .291 / .874
q34 / 25.51 / 38.037 / .387 / .873
q35 / 25.52 / 37.674 / .452 / .871

This is about all we typically get from SPSS when we analyze a test.

The Rasch analysis

The Rasch control file

&INST ; initial line (can be omitted)

TITLE = "Bond & Fox BLOT data: Chapter 4"

PERSON = Person ; persons are ...

ITEM = Item ; items are ...

ITEM1 = 5 ; column of response to first item in data record

NI = 35 ; number of items

NAME1 = 1 ; column of first character of person label

NAMELEN = 3 ; length of person identifying label

XWIDE = 1 ; number of columns per item response

CODES = 10 ; valid codes in data file

UIMEAN = 0 ; item mean for local origin

USCALE = 1 ; user scaling for logits

UDECIM = 2 ; reported decimal places for user scaling

TOTAL = Yes ; show total raw scores

CHART = Yes ; produce across-pathway picture

MNSQ = No ; use Standardized fit statistics

CONVERGE= L ; Convergence decided by logit change

LCONVERGE=.00001 ; Set logit convergence tight because of anchoring

IAFILE = * ; Item anchor file to preset the difficulty of an item

4 0 ; Item 4 exactly at 0 logit point.

* ; End of anchor list

&END

01 Negation (to negate identity) ; Item labels courtesy of Trevor Bond

02 Reciprocal (to negate identity)

35 Coordination of two systems of reference

END NAMES

001 11111111110110101101011111111011111

002 11111111111111111111111111101111111

003 11010111111111011111011111101011111

Mike – demo the analysis here.
The Rasch Item Map

TABLE 12.2 Bond & Fox BLOT data: Chapter 4 ZOU032WS.TXT Mar 24 18:13 2012

INPUT: 150 Persons 35 Items MEASURED: 150 Persons 35 Items 2 CATS 1.0.0

------

Persons MAP OF Items

4 ######### +

#### |

3 +

.### |

.######### | 21 Correlative + negation > equilibrium

### |

2 +

.### |T

.## M| 28 Non-implication

.#### |

###### |

.##### | 30 Equivalence

32 Negation of reciprocal implication

1 +S 13 Reciprocal exclusion

15 Reciprocal implication

.## | 08 Correlations

#### | 03 Implication

26 Complete affirmation

.# |

.# | 19 Reciprocal (to cause disequilibrium)

25 Complete negation

#### | 17 Identity (to negate reciprocal)

23 Correlative + identity > disequilibrium

### S| 24 Coordination of two systems of reference

# | 09 Conjunction

11 Conjunctive negation

31 Negation of q

0 +M 04 Incompatibility

18 Negation (to negate correlative)

# |

. | 10 Disjunction

16 Reciprocal (to negate identitiy)

35 Coordination of two systems of reference

. | 34 Coordination of two systems of reference

. | 29 Affirmation of q

33 Probability

. | 07 Correlations

.# | 01 Negation (to negate identity)

02 Reciprocal (to negate identity)

14 Probability

. | 20 Negation (to cause disequilibrium)

27 Negation of p

-1 T+S 05 Multiplicative compensation

. | 22 Reciprocal + negation > disequilibrium

. |

|T 12 Affirmation of p

-2 +

. |

| 06 Correlations

-3 +

EACH '#' IS 2.

Item characteristics . . . Item STATISTICS ordered by Measure

TABLE 13.1 Bond & Fox BLOT data: Chapter 4 ZOU527WS.TXT Mar 25 11:43 2012

INPUT: 150 Persons 35 Items MEASURED: 150 Persons 35 Items 2 CATS 1.0.0

------

Person: REAL SEP.: 2.04 REL.: .81 ... Item: REAL SEP.: 3.79 REL.: .93

Item STATISTICS: MEASURE ORDER

+------+

|------+------+------+-----+------+------|

| 21 54 150 2.40 .20|1.27 2.6|1.75 3.7| .32| 69.4 74.8| 21 Correlative |

| 28 73 150 1.68 .19|1.12 1.4|1.23 1.7| .43| 70.1 71.9| 28 Non-implicat|

| 32 87 150 1.17 .19| .96 -.5| .85 -1.1| .53| 72.8 71.2| 32 Negation of |

| 30 89 150 1.10 .19|1.19 2.3|1.15 1.0| .38| 63.3 71.4| 30 Equivalence |

| 13 91 150 1.03 .19|1.16 2.0|1.32 2.0| .37| 66.7 71.7| 13 Reciprocal e|

| 15 91 150 1.03 .19| .97 -.4| .84 -1.1| .52| 73.5 71.7| 15 Reciprocal i|

| 8 95 150 .88 .19| .91 -1.1|1.00 .1| .52| 75.5 72.5| 08 Correlations|

| 26 97 150 .80 .20| .90 -1.3| .75 -1.6| .55| 72.8 72.9| 26 Complete aff|

| 3 98 150 .76 .20| .98 -.2| .90 -.5| .49| 73.5 73.1| 03 Implication |

| 25 104 150 .52 .20|1.07 .8|1.26 1.3| .40| 74.1 74.9| 25 Complete neg|

| 19 105 150 .48 .20|1.01 .2|1.05 .3| .44| 74.8 75.2| 19 Reciprocal (|

| 17 107 150 .40 .20| .87 -1.4| .75 -1.3| .54| 78.2 75.8| 17 Identity (to|

| 23 108 150 .36 .21|1.06 .7| .92 -.3| .42| 70.7 76.2| 23 Correlative |

| 24 111 150 .23 .21| .89 -1.1|1.03 .2| .49| 82.3 77.4| 24 Coordination|

| 9 112 150 .18 .21|1.07 .7| .97 .0| .40| 76.2 77.8| 09 Conjunction |

| 11 112 150 .18 .21|1.02 .3| .96 -.1| .42| 80.3 77.8| 11 Conjunctive |

| 31 112 150 .18 .21|1.07 .7|1.55 2.2| .36| 78.9 77.8| 31 Negation of |

| 4 116 150 .00A .22|1.00 .0| .88 -.4| .43| 80.3 79.8| 04 Incompatibil|

| 18 117 150 -.05 .22| .90 -.9| .74 -1.0| .49| 79.6 80.3| 18 Negation (to|

| 10 120 150 -.20 .23| .92 -.7| .68 -1.2| .47| 84.4 81.8| 10 Disjunction |

| 16 122 150 -.31 .23|1.13 1.0|1.03 .2| .33| 79.6 82.9| 16 Reciprocal (|

| 35 122 150 -.31 .23| .93 -.5| .73 -.9| .45| 83.7 82.9| 35 Coordination|

| 34 124 150 -.42 .24|1.00 .1| .79 -.6| .41| 81.6 83.9| 34 Coordination|

| 29 125 150 -.48 .24| .94 -.4| .71 -.9| .43| 86.4 84.5| 29 Affirmation |

| 33 126 150 -.53 .25|1.10 .7| .93 -.1| .33| 81.0 85.0| 33 Probability |

| 7 128 150 -.66 .25| .97 -.1| .65 -1.0| .41| 85.7 86.0| 07 Correlations|

| 2 129 150 -.72 .26|1.01 .1| .75 -.6| .37| 85.0 86.6| 02 Reciprocal (|

| 14 129 150 -.72 .26|1.15 1.0|1.32 .9| .25| 85.0 86.6| 14 Probability |

| 1 130 150 -.79 .26| .98 .0| .69 -.8| .39| 86.4 87.1| 01 Negation (to|

| 20 131 150 -.86 .27| .91 -.5| .81 -.4| .40| 87.1 87.7| 20 Negation (to|

| 27 132 150 -.94 .27| .85 -.8| .62 -.9| .43| 89.8 88.3| 27 Negation of |

| 5 133 150 -1.01 .28| .98 -.1| .76 -.5| .35| 90.5 88.9| 05 Multiplicati|

| 22 134 150 -1.09 .29| .91 -.4|1.69 1.4| .35| 90.5 89.5| 22 Reciprocal +|

| 12 141 150 -1.81 .36| .69 -1.1| .24 -1.5| .46| 94.6 94.0| 12 Affirmation |

| 6 145 150 -2.49 .47|1.06 .3| .83 .0| .20| 96.6 96.6| 06 Correlations|

|------+------+------+-----+------+------|

| MEAN 109.9 147.0 .00 .24|1.00 .1| .95 -.1| | 80.0 80.5| |

| S.D. 19.5 .0 .97 .05| .11 1.0| .31 1.2| | 7.8 6.9| |

+------+

The key quantities in the table are

1. Measure – the difficulty of the item.

2. S.E. – the standard error of the estimate of the item’s difficulty. Note that the SEs of the more difficult items are smaller than those of the easy items. This is because there were too few respondents of low ability, resulting in nearly everyone getting these items correct, so the proportions of the sample getting them correct were very close to 1. When proportions are close to 1, larger numbers of persons are required for those proportions to be stable, so the proportions near 1 are not as stable as the proportions closer to .5, making the standard errors larger.

3. Infit – the mean of squared residuals with extra weighting given to persons whose abilities were close to the item difficulties. According to text, Rasch analysts give this more weight.

4. Outfit – the mean of squared residuals with all residuals weighted equally.

5. Note that Item 4 was anchored at difficulty = 0.

Item Characteristics . . . Items FIT GRAPH ordered by Measure

Output Tables -> 13. Item: measure -> Scroll down to Table 13.2.

TABLE 13.2 Bond & Fox BLOT data: Chapter 4 ZOU527WS.TXT Mar 25 11:43 2012

INPUT: 150 Persons 35 Items MEASURED: 150 Persons 35 Items 2 CATS 1.0.0

------

Items FIT GRAPH: MEASURE ORDER

+------+

| NUMBER| - + |-3 -2 -1 0 1 2 3 |-3 -2 -1 0 1 2 3 | Items |

|------+------+------+------+------|

| 21| *| : . :* | : . : *| 21 Correlative + negation > equilibrium |

| 28| * | : . * : | : . *: | 28 Non-implication |

| 32| * | : *. : | : * . : | 32 Negation of reciprocal implication |

| 30| * | : . * | : . * : | 30 Equivalence |

| 13| * | : . *: | : . * | 13 Reciprocal exclusion |

| 15| * | : *. : | : * . : | 15 Reciprocal implication |

| 8| * | : * . : | : * : | 08 Correlations |

| 26| * | : * . : | : * . : | 26 Complete affirmation |

| 3| * | : * : | : *. : | 03 Implication |

| 25| * | : . * : | : . * : | 25 Complete negation |

| 19| * | : * : | : * : | 19 Reciprocal (to cause disequilibrium) |

| 17| * | : * . : | : * . : | 17 Identity (to negate reciprocal) |

| 23| * | : .* : | : * : | 23 Correlative + identity > disequilibrium |

| 24| * | : * . : | : * : | 24 Coordination of two systems of reference|

| 9| * | : .* : | : * : | 09 Conjunction |

| 11| * | : * : | : * : | 11 Conjunctive negation |

| 31| * | : . * : | : . * | 31 Negation of q |

| 4| A | : * : | : *. : | 04 Incompatibility |

| 18| * | : * . : | : * . : | 18 Negation (to negate correlative) |

| 10| * | : *. : | : * . : | 10 Disjunction |

| 16| * | : . * : | : * : | 16 Reciprocal (to negate identitiy) |

| 35| * | : *. : | : * . : | 35 Coordination of two systems of reference|

| 34| * | : * : | : *. : | 34 Coordination of two systems of reference|

| 29| * | : *. : | : * . : | 29 Affirmation of q |

| 33| * | : . * : | : * : | 33 Probability |

| 7| * | : * : | : * . : | 07 Correlations |

| 2| * | : * : | : *. : | 02 Reciprocal (to negate identity) |

| 14| * | : . * : | : . * : | 14 Probability |

| 1| * | : * : | : * . : | 01 Negation (to negate identity) |

| 20| * | : *. : | : *. : | 20 Negation (to cause disequilibrium) |

| 27| * | : * . : | : * . : | 27 Negation of p |

| 5| * | : * : | : *. : | 05 Multiplicative compensation |

| 22| * | : *. : | : . * : | 22 Reciprocal + negation > disequilibrium |

| 12| * | : * . : | : * . : | 12 Affirmation of p |

| 6|* | : * : | : * : | 06 Correlations |

+------+

Item Characteristics – the bubble plot . . .

This plot show the same information as above, but it’s prettier.

Plots -> Bubble Chart -> Items (columns in data) -> Entry number

For this plot, I chose Items only and used Entry Numbers to identify items.

A comparison of 3 ways of scoring the BLOT – Summated score, log odds, and Rasch.Start here on 3/27/13.

To make the comparison, I had to put the Rasch Person measures into an SPSS file.

Output Tables -> 17. Person: entry.

I then copied the 150 lines of the table with the person entries and pasted them into Word. I then Alt+Selected the measure column and pasted it into a column in SPSS.

In SPSS I created a “rough” Rasch score for each person, logoddsscore, using the following syntax. (Recall it’s ln(score/(total possible - score)).

compute logoddsscore = ln(totalscore/(35-totalscore)).

Correlations of totalscore, logoddsscore, and raschscore

The correlations of the three measures are quite high. In fact, the correlation of the raschscore and logoddsscore is 1.000 to three decimal places. Its actual value to 6 decimal places is .999664.

Of course, this tells us that for some datasets we don’t need the program to compute person measures. For those datasets we can simply compute the ln(score/(total possible-score)) and use it.

The disadvantage of this is that we don’t get the other stuff that the BF program gives us. One advantage of using the program is that it will give us an estimate of the Rasch value for persons whose total score is perfect or 0, something the log method cannot do.
Dot plots of the three measures . . .

Note that for the summated score, totalscore, the best performers were just a little better than the crowd but were considerably better than the crowd for both logoddsscore and raschscore. This is in keeping with the notion that Rasch measurement lengthens the tails of distributions, spreading out the best and worst performers. This spreading was most noticeable among the best performers for these data.

Scatterplots of the three measures. The plots involving logoddsscore do not include the 3 persons whose totalscore was 35.

The Wonderlic Personnel Test Form II given to UTC students.

The Wonderlic Personnel Test (WPT) Form II considered here is a 50-item paper and pencil test designed to measure overall cognitive ability (g). It is a timed test. For the paper and pencil version respondents are given 12 minutes to complete as many items as possible.

From the Wonderlic manual,

“The WPT and SLE (Scholastic Level Exam) are short form tests of general cognitive ability. Often referred to as general intelligence, or “g”, cognitive ability is a term that is used to describe the level at which an individual learns, understands instructions and solves problems.” – p 5

“The score is the total number of correct answers.” P. 9

If you’re interested, median scores for populations given in the manual printed in 2002 are

1992 Total Applicant Population N=118549 / Adult Working Population / HS Grads
Age 16-30 / Coll Grads
Ages 20-30
3rd Quartile / 26 / 27 / 25 / 34
Median / 21 / 22 / 21 / 30
1st Quartile / 16 / 17 / 16 / 25
SD / 7.12 / 7.6 / 6.8 / 6.3

For our mostly Frosh research sample . . ., Mean = 22.02 and SD = 5.401.

The dataset is balancedscale_120428.sav.

Some questions we might ask

1. Are all of the WPT items appropriate?

a. Does it correlate with the total score?

b. Is the pattern of correctness / incorrectness appropriate across persons?

2. Are the persons and items matched for our data? (A question we’d never have asked before Rasch.)

a. Are there sufficient numbers of difficult items, so differences between high scorers can be measured?

b. Are there sufficient numbers of easy items, so differences between low scorers can be measured?

c. are there sufficient numbers of items at all difficulty levels?

3. Are the person ability (WPT scores) that we’re using appropriate, or should we use Rasch scores?

The SPSS Analysis

Reliability Statistics
Cronbach's Alpha / N of Items
.762 / 50
Item-Total Statistics
Scale Mean if Item Deleted / Scale Variance if Item Deleted / Corrected Item-Total Correlation / Cronbach's Alpha if Item Deleted
wpt1 / 21.26 / 27.911 / .237 / .758
wpt2 / 21.11 / 28.851 / .076 / .763
wpt3 / 21.39 / 26.970 / .391 / .750
wpt4 / 21.14 / 28.404 / .189 / .759
wpt5 / 21.07 / 28.912 / .091 / .762
wpt6 / 21.42 / 28.000 / .179 / .761
wpt7 / 21.34 / 27.886 / .215 / .759
wpt8 / 21.15 / 28.106 / .270 / .757
wpt9 / 21.30 / 28.163 / .168 / .761
wpt10 / 21.62 / 28.734 / .037 / .767
wpt11 / 21.31 / 28.898 / .013 / .767
wpt12 / 21.10 / 28.541 / .188 / .760
wpt13 / 21.55 / 27.117 / .346 / .752
wpt14 / 21.08 / 28.603 / .195 / .760
wpt15 / 21.20 / 28.228 / .194 / .759
wpt16 / 21.14 / 28.414 / .186 / .759
wpt17 / 21.36 / 28.106 / .166 / .761
wpt18 / 21.65 / 27.741 / .235 / .758
wpt19 / 21.75 / 27.995 / .207 / .759
wpt20 / 21.30 / 27.887 / .228 / .758
wpt21 / 21.28 / 28.720 / .054 / .765
wpt22 / 21.90 / 28.127 / .275 / .757
wpt23 / 21.17 / 27.966 / .284 / .756
wpt24 / 21.45 / 27.546 / .265 / .756
wpt25 / 21.64 / 26.924 / .398 / .750
wpt26 / 21.59 / 26.936 / .386 / .750
wpt27 / 21.64 / 28.251 / .132 / .763
wpt28 / 21.53 / 27.958 / .181 / .760
wpt29 / 21.45 / 27.663 / .242 / .758
wpt30 / 21.25 / 27.280 / .390 / .751
wpt31 / 21.59 / 26.800 / .413 / .749
wpt32 / 21.96 / 28.383 / .280 / .757
wpt33 / 21.82 / 28.639 / .085 / .764
wpt34 / 21.71 / 28.042 / .186 / .760
wpt35 / 21.91 / 27.631 / .443 / .752
wpt36 / 21.74 / 26.865 / .453 / .748
wpt37 / 21.96 / 28.120 / .383 / .755
wpt38 / 21.67 / 27.385 / .310 / .754
wpt39 / 21.95 / 28.759 / .129 / .761
wpt40 / 21.94 / 28.401 / .236 / .758
wpt41 / 21.96 / 28.637 / .182 / .760
wpt42 / 21.86 / 28.225 / .211 / .759
wpt43 / 22.01 / 29.063 / .092 / .762
wpt44 / 21.80 / 28.121 / .200 / .759
wpt45 / 21.99 / 28.702 / .243 / .759
wpt46 / 22.01 / 28.980 / .247 / .761
wpt47 / 21.88 / 28.455 / .163 / .760
wpt48 / 22.02 / 29.170 / .000 / .763
wpt49 / 22.02 / 29.170 / .000 / .763
wpt50 / 22.02 / 29.170 / .000 / .763

Data files -> BondFoxChapterBSWPT.txt

The Rasch Control file

&INST

Title = “BalancedScaleQuestionnaireWPTdata”

ITEM1 = 5

NI = 50

NAME1 = 1

NAMELEN = 4

XWIDE = 1

CODES = 10

Total = Yes

&End

ENDLABELS

500111111101011111110101001011011100001101001000000000

206 data lines follow.

The Rasch Analyses – Item Map

TABLE 12.2 BalancedScaleQuestionnaireWPTdata ZOU839WS.TXT Mar 25 15:21 2012

INPUT: 206 PERSONS 50 ITEMS MEASURED: 206 PERSONS 50 ITEMS 2 CATS 1.0.0

------

PERSONS MAP OF ITEMS

6 + I0048 I0049 I0050

| I0046

5 +

| I0043

4 +

. | I0045

3 +

| I0032 I0037 I0039 I0041

. | I0040

2 + I0022 I0035

. |

| I0047

. | I0042

## T|

. | I0033 I0044

1 .## +

# | I0019 I0036

## |

.##### S| I0034

###### | I0018 I0038

########## | I0010 I0025 I0027

0 ######## +M I0026 I0031

######## | I0013

######## M| I0028

.##### |

##### | I0024 I0029

######## | I0006

-1 ############ + I0003 I0017

.##### S| I0007

#### | I0009 I0011

.## | I0020 I0021

.### | I0001 I0030

. |

-2 # T+ I0015

| I0023

| I0004 I0008 I0016

| I0002

-3 + I0012

| I0014

| I0005

-4 +

EACH '#' IS 2.

Item Information . . . Item STATISTICS ordered by Measure

TABLE 13.1 BalancedScaleQuestionnaireWPTdata ZOU839WS.TXT Mar 25 15:21 2012

INPUT: 206 PERSONS 50 ITEMS MEASURED: 206 PERSONS 50 ITEMS 2 CATS 1.0.0

------

PERSON: REAL SEP.: 1.75 REL.: .75 ... ITEM: REAL SEP.: 7.45 REL.: .98

ITEM STATISTICS: MEASURE ORDER

+------+

|------+------+------+-----+------+------|

| 48 0 206 6.66 1.83| MAXIMUM ESTIMATED MEASURE | | I0048|

| 49 0 206 6.66 1.83| MAXIMUM ESTIMATED MEASURE | | I0049|

| 50 0 206 6.66 1.83| MAXIMUM ESTIMATED MEASURE | | I0050|

| 46 1 206 5.44 1.01| .79 .1| .04 -1.8| .32| 99.5 99.5| I0046|

| 43 2 206 4.72 .72|1.05 .3| .65 -.2| .10| 99.0 99.0| I0043|

| 45 6 206 3.55 .43| .91 -.1| .61 -.6| .29| 97.1 97.1| I0045|

| 32 13 206 2.68 .30| .92 -.2| .68 -.9| .34| 94.2 93.9| I0032|

| 37 13 206 2.68 .30| .84 -.6| .54 -1.5| .44| 94.2 93.9| I0037|

| 41 13 206 2.68 .30| .96 -.1|1.20 .7| .24| 94.2 93.9| I0041|

| 39 14 206 2.59 .29|1.12 .6| .98 .0| .17| 92.7 93.4| I0039|

| 40 17 206 2.37 .26| .99 .0| .97 .0| .28| 91.7 92.0| I0040|

| 35 22 206 2.05 .24| .83 -1.0| .56 -2.0| .50| 90.3 89.7| I0035|

| 22 24 206 1.94 .23| .95 -.3| .88 -.5| .34| 89.3 88.7| I0022|

| 47 28 206 1.75 .21|1.04 .3|1.11 .6| .24| 86.9 86.9| I0047|

| 42 32 206 1.58 .20|1.02 .2|1.04 .3| .28| 85.0 85.0| I0042|

| 33 42 206 1.20 .18|1.15 1.4|1.26 1.7| .15| 77.2 80.6| I0033|

| 44 45 206 1.11 .18|1.03 .4|1.08 .6| .28| 77.7 79.4| I0044|

| 19 56 206 .78 .17|1.04 .6|1.07 .6| .29| 73.8 75.0| I0019|

| 36 57 206 .75 .17| .85 -2.0| .79 -2.0| .52| 80.1 74.6| I0036|

| 34 64 206 .56 .16|1.06 .9|1.15 1.5| .26| 70.9 71.9| I0034|

| 38 73 206 .34 .16| .96 -.6| .97 -.3| .39| 71.8 69.2| I0038|

| 18 76 206 .27 .15|1.03 .5|1.03 .4| .32| 66.5 68.4| I0018|

| 25 78 206 .22 .15| .90 -1.7| .86 -1.8| .47| 73.3 68.0| I0025|

| 27 78 206 .22 .15|1.11 1.9|1.15 1.9| .22| 62.6 68.0| I0027|

| 10 82 206 .13 .15|1.18 3.2|1.28 3.4| .13| 61.7 67.0| I0010|

| 26 89 206 -.03 .15| .91 -1.8| .88 -1.7| .46| 69.9 65.7| I0026|

| 31 89 206 -.03 .15| .89 -2.2| .86 -2.0| .48| 69.9 65.7| I0031|

| 13 96 206 -.19 .15| .94 -1.3| .92 -1.2| .42| 69.4 65.0| I0013|

| 28 101 206 -.30 .15|1.07 1.5|1.08 1.2| .27| 61.2 64.8| I0028|

| 24 118 206 -.68 .15|1.00 .0| .99 -.1| .35| 66.0 65.6| I0024|

| 29 118 206 -.68 .15|1.00 .1|1.06 .8| .32| 68.0 65.6| I0029|

| 6 124 206 -.82 .15|1.07 1.3|1.04 .6| .27| 63.1 66.4| I0006|

| 3 130 206 -.96 .15| .89 -2.0| .85 -1.7| .46| 70.9 67.5| I0003|

| 17 135 206 -1.08 .16|1.06 1.1|1.07 .7| .25| 65.5 68.7| I0017|

| 7 139 206 -1.18 .16|1.02 .3|1.01 .2| .30| 69.4 69.9| I0007|

| 11 146 206 -1.35 .16|1.15 2.1|1.41 3.1| .10| 68.4 72.2| I0011|

| 9 148 206 -1.41 .16|1.04 .5|1.23 1.7| .24| 72.3 72.9| I0009|

| 20 149 206 -1.43 .16|1.00 .0| .98 -.1| .30| 71.8 73.3| I0020|

| 21 152 206 -1.52 .17|1.13 1.6|1.23 1.6| .14| 72.3 74.5| I0021|

| 1 156 206 -1.63 .17| .99 -.1| .96 -.3| .31| 75.7 76.1| I0001|

| 30 159 206 -1.72 .17| .87 -1.5| .77 -1.6| .44| 77.7 77.4| I0030|

| 15 169 206 -2.04 .19|1.00 .0| .98 -.1| .26| 82.0 82.0| I0015|

| 23 175 206 -2.27 .20| .94 -.5| .78 -1.0| .34| 85.0 84.9| I0023|

| 8 180 206 -2.49 .22| .93 -.4| .78 -.9| .32| 87.4 87.4| I0008|

| 4 181 206 -2.54 .22|1.00 .0| .88 -.5| .24| 87.9 87.9| I0004|

| 16 181 206 -2.54 .22|1.00 .0| .93 -.2| .24| 87.9 87.9| I0016|

| 2 187 206 -2.86 .25|1.05 .3|1.10 .4| .13| 90.8 90.8| I0002|

| 12 189 206 -2.99 .26| .96 -.1| .92 -.2| .23| 91.7 91.7| I0012|

| 14 193 206 -3.29 .29| .97 .0| .73 -.7| .23| 93.7 93.7| I0014|

| 5 196 206 -3.58 .33|1.02 .2| .90 -.1| .13| 95.1 95.1| I0005|

|------+------+------+-----+------+------|

| MEAN 90.7 206.0 .40 .33| .99 .1| .94 .0| | 79.6 79.7| |

| S.D. 65.0 .0 2.57 .41| .09 1.1| .23 1.3| | 11.5 11.2| |

+------+

The very difficult items were not even responded to by most of the participants, so their SEs are huge. Luckily, since no participants even got to them, that probably won’t be a problem for our sample.

Item Information . . . Bubble chart

The span of difficulty values is much greater for these items (-5 to +7) than for the BLOT items in the previous example, for which it was -3 to +3.

I would say that if I were to “look” at this test for ways to improve it, I would begin with items 3, 31, 36, 10, and 11, for all of which there was inconsistency in the proportions of low and high ability persons who got them correct.

This analysis skirts the issue of whether an item should be included in the analysis if it was not responded to. That is, because the test it timed, most people do not respond to all items. In fact, not many people got to items 31 and 36, so most of the “incorrect” responses were counted as such because persons didn’t get to them. This is something that the BF program does not know about – all it knows is that these items were counted as being “incorrect”.

Comparison of the 3 ways of scoring – Summated vs. log odds vs Rasch.

(These graphs were created after I copied and pasted the 206 Rasch measures into the SPSS file.)

Log odds vs Total Score (compute wptlogoddsscore = ln(wpttotscore/(50-wpttotscore)).

Rasch vs. Total Score

Rasch vs. Log odds

PSY 5950 L13 - 1