Selected Nonparametric Statistics
Categorical Data Analysis
Packet CD05
Dale Berger,
Claremont Graduate University
()
Statistics website:
2Counting Rules
5Binomial Distribution
8D11: Wilcoxon Ws and Mann-Whitney U
10D12: Comparing two groupswithSPSS (t, WilcoxonWs, Mann-Whitney U, Median)
18D13: Wilcoxon T for paired data
22D14a: SPSS CROSSTABS Statistics for 2x2 Contingency Tables
28D14b: SPSS CROSSTABS analyses for larger contingency tables
33D15: McNemar’s test of related proportions
35D16: Spearman r and SPSS
39Table D (Binomial) and Table P (Spearman r) from Siegel (1956) Nonparametric Statistics
40Critical values for Spearman r from Ramsey (1989) Journal of Educational Statistics
41Mann-Whitney U Table from Kirk (1978) Introductory Statistics
42Wilcoxon T Table from Kirk (1978) Introductory Statistics
43Table F (Runs tests – too few or too many)from Siegel (1956) Nonparametric Statistics
Berger, CGUCounting Rules
Rule 1:If any one of k mutually exclusive and exhaustive events can occur on each of n trials, then there are kn different sequences that may result from a set of trials.
Example: Toss a coin 4 times. How many possible outcomes are there? One possible outcome may be represented by HTTH. The total number of possible outcomes can be illustrated by a branching diagram, as shown below. There are two possibilities for the first coin. For each of these possibilities there are 2 possibilities for the second coin, giving a total of 4 distinct two-coin sequences (HH, HT, TH, and TT). For each of these two-coin sequences, there are 2 possible outcomes for the third coin, giving 4x2 or 8 possible three coin outcomes. Similarly, for each of the 8 three-coin sequences, there are 2 possible outcomes for the fourth coin, giving 8x2 = 16 distinct four-coin sequences. If we apply Rule 1 with k=2 (i.e., heads or tails) and n=4 (four coin tosses) we obtain kn = 24 = 2 x 2 x 2 x 2 = 16.
Suppose each outcome is equally likely to occur. Then the probability of any particular sequence is 1/kn. What is the probability of four heads on four coin tosses? (1/kn = 1/24 = 1/16 = .0625.)
Example: How many distinct ways are there to answer a 10-item multiple choice test with four alternatives on each item? (Answer: k=4 and n=10, kn = 410 = 1,048,576.)
What is the probability that someone who is purely guessing will score all 10 correct on this test? (Only one sequence is totally correct, so the probability is 1/1048576 = .00000095367.)
Rule 2:If any we have n trials where the number of different events which can occur on trials 1, 2, 3, …, n are k1, k2, k3 ….kn respectively, then the number of distinct outcomes from the n events is (k1)(k2)(k3)…(kn).
Example: Suppose we have a task with a sequence of three choice points. At the first point we have two choices, at the second point we have three choices, and at the third point we have four choices. How many different ways might we complete the task? [Answer: (2)(3)(4) = 24.]
What is the probability of any one specific sequence if all choices are equally likely at each step?
[Answer: 1/24 = .0417.]
Note that Rule 1 is a special case of Rule 2, where k1 = k2 = k3 = …. = kn = k.
Rule 3:The number of different ways that n distinct objects may be arranged in order is (n)(n-1)(n-2)…(3)(2)(1). This product is called n-factorial, symbolized by n!. 0! is defined to be equal to 1. Any particular arrangement of n objects is called a permutation. Thus, the total number of permutations for n objects is n!.
Example: You are the judge for a pie baking contest, and it is your task to rank the three finalists: Apple, Banana, and Cream. How many distinct orders are possible? Applying the formula, n! = 3! = 3x2x1 = 6. The six possible orders are ABC, ACB, BAC, BCA, CAB, and CBA. There are three possible ways to fill the first place. After first place is assigned, there are two pies left, or two ways to fill the second place. Thus, for each of the three possible ways to fill first place, there are two ways to fill second place, giving 3x2 ways to fill the first two places. For each choice of first and second place, there is only one pie left for third place; the total number of ways to rank the three pies is 3x2x1 = 3! = 6.
Rule 4:The number of ways of selecting and arranging r objects from N distinct objects is NPr = N! / (N-r)! = “Permutations of N objects r at a time.”
Example: Given a set of 5 different cards, how many ways can you and I each choose one card? The first card chosen can be any one of the 5 cards. After I have chosen my card, there are 4 cards left for you to choose from, so there is a total of 5x4 = 20 ways in which we can choose two cards. We have selected and arranged r=2 objects from a set of N=5 objects, so we can calculate NPr = 5! / (5-2)! = 5! / 3! = 5x4x3x2x1 / 3x2x1 = 5x4 = 20.
Example: How many ways can we select 3 students from a class of 10 to fill the offices of President, Vice-President, and Secretary? The first office can be filled by any one of 10 people. After this office is filled, we must choose from among the remaining 9 people to fill the second office. Finally, the third office can be filled by any one of 8 people. This gives a total of 10x9x8 = 720 ways to fill the three offices. Notice that the descending product is the first part of 10!, with the last part (7!) missing. The expression 10x9x8 can be written as 10! / 7!, which is 10P3.
Rule 5:The number of ways of selecting a sample of r objects from a set of N distinct objects is NCr = N! / [r!(N-r)!] = “Combinations of N objects r at a time.”
Example: How many ways can you select two cards from a deck of 5 distinct cards, with no concern for order? Let us look again at the situation described in Rule 4 where you and I each selected a card from a deck of 5 distinct cards. There are 5!/3! = 20 ordered pairs. If the Ace and King were drawn, we counted AK and KA as two separate outcomes because we were concerned with order. Thus, each pair of cards was counted twice. If we want only the number of possible pairs with no concern for order, we must divide the number of ordered pairs by 2!, the number of different orders for the two cards. This gives us 20/2 = 10 distinct pairs. If we apply Rule 5 we get 5C2 = 5! / [(2!)(3!)] = 120 / [(2)(6)] = 10.
Example: Given a group of 10 people, how many ways can we choose 3 to form a committee with no regard for order of selection? We have already found the number of ordered groups of 3 people. By applying Rule 4 we found 10!/7! = 720 ordered groups of 3. The number of unordered committees must be less than this number because for every unordered group of 3 people, there are 3! = 6 orders. In general, because a group of r objects can be ordered in r! ways, there are r! times as many ordered groupings as there are different groups not considering order. If we divide the number of ordered groups given to us by Rule 4 by r!, we have Rule 5, the number of groups of size r not considering order, N!/[(N-r)!r!].
Note that the number of groups of r objects selected from N objects is equal to the number of groups of N-r objects. This is because for each group of size r, the remaining objects form a group of size N-r. Thus, 100C3 = 100C97 = 100! / [97! x 3!].
Rule 6:The number of distinct orders of N objects consisting of k groups of N1, N2, …, Nk indistinguishable objects is N! / [N1! N2! … Nk!].
Example: How many ways can the letters A,B,B,C,C,C be arranged if we can’t distinguish among like letters? If all of the 6 letters were distinct, there would be 6! distinct arrangements. But for any specific arrangement of the letters, such as ACBCBC, the Bs can be arranged in 2! ways and the Cs can be arranged in 3! ways. Each of the now indistinguishable orders is counted when we use 6!, so we must divide by both 2! and 3!, giving 6!/[2!3!] = 720/[2x6] = 60.
Example: How many distinct ways can you rearrange the letters in MISSISSIPPI?
N=11; NM=1; NI=4; NS=4; NP=2 11!/[1!4!4!2!] = 34,650
Example: How many distinct ways could you have 4 items correct on a 10-item test? That is, how many distinct arrangements are there of 4 Cs and 6 Ws?
Dale Berger, CGUBinomial Distribution
When we have a dichotomous event (two mutually exclusive possibilities) such as male or female, success or failure, Republican or Democrat, open or closed mind, etc. and we have multiple independent observations, we may be able to make use of the Binomial Distribution
(bi = two; nominal = names).
Consider a multiple-choice examination where each item has 4 choices, only one of which is correct. If all choices are equally likely, then we would expect a student who knows absolutely nothing about the subject to be correct on about one item in four simply by guessing. In general, we let p be the probability of a success and 1-p = q be the probability of a failure. If the test has four choices on each item, is in Russian, and a student taking the test knows no Russian, we would expect p=1/4 and q=3/4.
(1)For a test with a single item (n=1), there are two possible outcomes:
He is correct (C) with probability p (here p = 1/4)
He is wrong (W) with probability q (here q = 3/4)
(2)If each item is independent of all others, then for a test with n=2 items, we have four possible outcomes:
Items
12
C C with probability p x p = p2 =1/4 x 1/4 = 1/16
C W " p x q = pq =1/4 x 3/4 = 3/16
W C " q x p = pq =1/4 x 3/4 = 3/16
W W " q x q = q2 =3/4 x 3/4 = 9/16
Because these four outcomes are mutually exclusive and exhaustive, the sum of their probabilities is equal to 1.000.
p2 + 2pq + q2 = (p + q)2 = 1 because (p + q) = 1.
Also, 1/16 + 3/16 + 3/16 + 9/16 = 16/16 = 1.00
Suppose we are concerned with the number of items a student might have correct on this two-item test. Let's call this number X. Then X is a random variable which may take on the values 0, 1, or 2 and there are probabilities associated with each of these values. We could construct a sampling distribution for X as follows:
Number of Successes
x P(X=x) This example
0 q2 9/16 = .5625
1 2pq 6/16 = .3750
2 p2 1/16 = .0625
Question:What is the probability that a person who is guessing on this two-item test will have exactly one item correct?
Answer:p(X=1) = 2pq = 6/16 = .3750 in this example. Note that this probability comes from summing the probabilities associated with the outcomes CW and WC, the two ways of having exactly one success.
Let's look at the 8 possible outcomes from a three-item test (n=3).
Items Number Number This
1 2 3 Sequence probability correct of ways Prob. example
C C C p x p x p = p3 3 1 p3 .0156
C C W p x p x q = p2q
C W C p x q x p = p2q 2 3 3p2q .1406
W C C q x p x p = p2q
C W W p x q x q = p q2
W C W q x p x q = p q2 1 3 3pq2 .4219
W W C q x q x p = p q2
W W W q x q x q = q3 0 1 q3 .4219
The probabilities for these mutually exclusive events can be added to show that they sum to 1:
p3 + 3p2q + 3pq2 + q3 = (p + q)3 = 1 because (p + q) = 1.
The first term in the above sum gives the probability of observing three successes. The second term has two parts; the p2q is the probability of observing any particular sequence with two successes and one failure, and the coefficient of 3 represents the number of different sequences consisting of two successes and one failure. This coefficient could also be obtained by calculating the number of ways in which we could choose x=2 items to be correct out of the n=3 items. This can be expressed as the number of combinations of 2 chosen from a group of 3, or 3C2 = 3!/2!1! = 6/(2x1) = 3. Similarly, the coefficient for the third term in the sum represents the number of ways 1 item from a group of 3 can be correct, or 3!/1!2! = 3. To continue the pattern, the coefficient for the last term is 1 which is the number of ways of getting 0 correct on a 3 item test, or 3!/0!3! = 6/(1x6) = 1. Thus, each term in the sum gives the probability of getting x successes and can be expressed in the general form 3Cx pxq3-x.
We could thus write our sum:
3C0 p0q3 + 3C1 p1q2 + 3C2 p2q1 + 3C3 p3q0 = q3 + 3pq2 + 3p2q + p3.
In general, the exact probability associated with exactly x successes out of n trials in any situation where trials are independent and p remains the same on each trial can be calculated with the following expression:
p(x=X) = nCx pxqn-x [** This is the very useful Binomial Formula]
nCx is n!/[x!(n-x)!] “combinations of n objects taken x at a time” (Counting Rule 5)
Example: Find the probability that a person guesses correctly on every item in a five-item test where there are four choices on each item.
With n=5, x=5, and p=1/4, we find that
nCx pxqn-x = 5C5 (1/4)5(3/4)0 = (1/4)5 = 1/1024 = .0010
Example: Find the probability that our person is correct exactly four times and wrong only once. Applying the formula with n=5, x=4, and p=1/4:
nCx pxqn-x = 5C4 (1/4)4(3/4)1 = 5 (1/4)4(3/4)1 = 15/1024 = .0146
Note that for any one particular sequence of 4 correct and 1 wrong, the probability is p4q1, and there are 5 ways to get such a sequence: (CCCCW, CCCWC, CCWCC, CWCCC, WCCCC).
If we let X represent the number correct on the five-item test, then X is a random variable. We have just calculated the probabilities associated with two of the possible values of X, p(X=5) = .0010 and p(X=4) = .0146. To complete the sampling distribution for X when n=5, we must calculate the probabilities that X = 3, 2, 1, or 0.
Number of Number Probability
Successes Probability of Binomial Dist. if
x of Sequence Ways p(X=x) p=1/4, q=3/4
5 p5 1 p5 .0010
4 p4q1 5 5p4q .0146
3 p3q2 10 10p3q2 .0879
2 p2q3 10 10p2q3 .2637
1 p1q4 5 5p q4 .3955
0 q5 1 q5 .2373
1.0000
This table can be used to answer questions regarding the likelihood of getting any particular number of items right by chance. For example, what is the probability that a person will get exactly three items correct on this test if she is guessing? Answer: p(X=3) = .0879.
This table can also be helpful in making inferences about performance on the test. For example, suppose a job applicant says she can read Russian. You give her this 5-item Russian test, and she gets all five correct. Do you think she can read Russian? (How likely is it that she would get such a good score if she were guessing?) [Answer: The probability that someone would get all five items correct by chance is only .0010. It seems likely that she can read Russian.]
Dale Berger, CGUWilcoxon Ws and Mann-Whitney U D11
Sometimes we wish to compare performance of two groups but our data do not satisfy the normality assumptions of the parametric t-test, and we have a small sample size so the sampling distribution of means may not be close to normal. A nonparametric test may do the job for us.
Suppose we wish to compare an Experimental group (E) with a Control group (C) on the number of downloads from a research site in the past week. We have three randomly sampled observations from
E(7, 12, 86) and four randomly selected observations from C (0, 4, 6, 10). Can we conduct a t-test for independent groups? Sure, the computer won’t care. SPSS gives us the mean for sample E=35.0 with
SD=44.2, and the mean for sample C=5.0 with SD=4.16, and t(5)=1.395, p=.222.
Have we satisfied the mathematical assumptions of the t-test?If we use the SPSS estimate for t with variances not assumed equal we get t(df=2.027) = 1.171, p=.361. Is this test valid?The plot below may be helpful to assess assumptions. Do we believe that the sampling distribution of the difference between means is reasonably normal? Obviously, that is not likely because of the outlier and small sample sizes.
______E____E______E__
01020304050607080______
C C C C
The Mann-Whitney U test is based on the rank order of the observations, not on the scale values. The null hypothesis is that if a score is randomly chosen from each population, p(EC)=1/2. That is, the two randomly chosen scores are as likely to be ordered EC as CE.
We observed the order C C C E C E E. What is the probability that the seven scores would be ordered with E having as much or more of an advantage over C than this if the null hypothesis is true,
i.e., p(EC)=1/2? Let’s calculate this by hand.
How many distinct ways can we order three Es and four Cs? Counting Rule 6, described earlier in this handout, shows why this is (N1 + N2)! / (N1!*N2!) = 7!/(3!4!) = (7*6*5*4!)/(3!*4!) = 35.
What is the probability that the seven scores would be ordered C C C C E E E ? 1/35 = .0286
What is the probability that the seven scores would be ordered C C C E C E E ? 1/35 = .0286
This gives us a one-tailed probability of .0286+.0286 = .0572 and two-tailed p=.1144.
To compute the Mann-Whitney U statistic, we count the number of C scores that exceed each E score, and total them. Here we find U=1. This can also be seen as the number of reversals of adjacent pairs needed to reach perfect separation, CCCCEEE. Alternatively, if we counted the number of E values that exceed each C, we would find UE=3+3+3+2=11. UC=(N1*N2) – UE= (3*4)-11 = 12-11=1. Only one C exceeds one E. Mann-Whitney U is the smaller of UE and UC. There are tables for U in many books. For example, when N1=3 and N2=4, Siegel (1957) gives p=.057 for U=1, and p=.028 for U=0. These are one-tailed p values.
A descriptive effect size is ‘Probability of Superiority’ = 1 – U/(N1N2). PS = 1 – 1/(3*4) = 1 - 1/12 = .917.
Wilcoxon Ws is an alternate and equivalent statistical test based on ranks. We simply order all of the scores and find the sum of ranks for scores in the smaller group. A source of confusion is that ranking can be done from either end, with the largest value given the rank of 1 or the smallest value given the rank of 1. Ws is defined as the smaller of these two values. If we made the wrong choice we can apply a formula to our sum to find Ws. Important note: SPSS doesn’t necessarily report the smallest Wilcoxon Ws value but it reports Mann Whitney U correctly. SPSS gives a wrong Ws when the sum of ranks is smaller for the larger group.
Score / Group / Rank1 / R1C / R1E / Rank2 / R2C / R2E0 / C / 1 / 1 / 7 / 7
4 / C / 2 / 2 / 6 / 6
6 / C / 3 / 3 / 5 / 5
7 / E / 4 / 4 / 4 / 4
10 / C / 5 / 5 / 3 / 3
12 / E / 6 / 6 / 2 / 2
86 / E / 7 / 7 / 1 / 1
Sum of ranks: / 28 / 11 / 17 / 28 / 21 / 7
We have four possible values for the sum of ranks: 11, 17, 21, and 7. We define Ws as the smallest sum, so Ws = 7. This will be the smaller of the two possible sums of ranks for the smaller of the two groups.
SPSS erroneously reports Ws=11, computed as the smaller sum of ranks for the two groups (R1C), where the smallest number is ranked 1 (see Rank1, with corresponding R1C and R1E). If the smaller group has larger ranks, as we have here, we can find the correct Ws value by reversing the ranking, assigning a rank of 1 to the largest number (see Rank2). Then the smaller sum of ranks is R2E = 7, the correct value for Ws.
Conventionally, we call the smaller sample size N1 and the larger sample size N2. Here N1=3 and N2=4, and the sum of ranks for the smaller group = R1E = 17 = Ws'. We can convert Ws' to the correct Ws value with the formula Ws = - Ws', where = N1*(N1+N2+1).