Chapter 9: Linear Correlation

Chapter 9: Linear Correlation

1. Describing Linear Relationships

When a linear (or straight-line) relationship exists between two variables, knowing a subject’s value on one variable makes it easier to predict their value on the other variable. For example, imagine you see some adorable guy/girl across the room at an off-campus party and you want to know this person’s age. If you just glance at him/her from across the room, you would likely assume he/she must be somewhere between the ages of 18 and 22, a typical college age range. However, they could be someone’s younger sibling or older cousin, so really, you don’t have much information to go on, and the range could reasonably be closer to 16 to 26. But if someone told you the person was a junior at your college, you now have information that helps you more easily predict the person’s actual age (probably 19 to 21), since there is a high linear correlation between one’s academic year in college and one’s age. Don’t forget: there are exceptions to every rule (some people don’t start college until they are 24, and Doogie Howser was probably done with college by age 14), so the correlation certainly won’t be perfect, and there will be plenty of errors. However, as long as there is some consistent relationship between the two variables, one variable will help us predict the other. But how can we measure the consistency of the relationship? As an example of an interesting and roughly linear relationship, let’s look at some data relating the amount of weekly alcoholic drinking and academic performance.

In graphical terms, the relationship between GPA and average amount of alcohol consumption would look a lot like a straight line:

Keep in mind that these values are averages. If the data for ALL of the individual participants had been included in the graph, there would be dots scattered all around the line—but the squared distances of the dots above the line would balance out the squared distances of the dots below. [That is why the regression line is known as the best-fit or least-squares line, as will be explained in the next chapter.]

Conveniently, when the z scores are similar for both variables, they are highly positively correlated; when the z scores are similar in magnitude, but opposite in sign (i.e., one z score is positive, while the other is negative), the variables are highly negatively correlated. So, to determine the strength of a linear relationship, we can begin by converting all our raw scores to z scores. As a reminder, here’s the z-score formula, which you would use separately for both variables (generally, we label the one we are predicting from zx and the one being predicted as zy).

2. Calculating Pearson’s r from z Scores

Once you have converted your raw scores to zxs and zys, Pearson’s r can be computed by plugging your z scores into the following formula (where N = number of pairs):

The values for r will range from –1.0 (perfect negative correlation) up to +1.0 (perfect positive correlation). While the sign in front of r tells you whether there is a positive or negative relationship between your variables, the absolute value of r (i.e., its magnitude) tells you about the strength of the linear relationship between the two variables. Of course, an r of zero indicates no linear relationship at all—kind of like trying to relate the number of miles someone drives on an average day to the length of her right foot. There’s not likely to be much of a relationship, linear or otherwise!

Try this example:

1. For the paired z scores in the following table, compute the value for Pearson’s r using the z score formula, and then answer the questions that follow. [Note that these sets of scores look like z scores, but each set does not actually have a mean of zero and SD of 1.0. Nonetheless, this exercise will give you practice with the z score formula, and the result is realistic.]

zx / zy
+1.3 / +1.4
–2.6 / +2.9
+0.2 / –1.6
+1.5 / +1.9
–2.3 / –2.3
+0.3 / +1.5
–3.2 / +2.8
–1.2 / –1.7
+2.0 / +2.0

a. Do these two variables seem to have a strong linear relationship? Draw a graph, and describe what you see.

b. What value did you calculate for Pearson’s r? Given the scatter of the points in the graph that you drew for part a, does the overall Pearson’s r for these data come as a surprise to you? Why or why not?

3. Calculating Pearson’s r from Raw Scores

Although it is a useful way to explain Pearson’s r for linear correlation, it is a real pain to convert all your scores to z scores. Fortunately, if you have already calculated the means and biased standard deviations of your two variables, it is particularly easy to use the following formula:

The only new term you have to calculate, which, admittedly, is a pretty weird one, is the average of the cross-products of your raw scores. That is, for each case (e.g., participant), you have to multiply the value on X by the value of Y, then add up all of these cross-products and divide them by the total number of cross-products (i.e., the number of pairs of scores, N). Subtracting the cross-product of the two means from the mean of the cross-products gives you the numerator of the formula, which is called the covariance. It happens to be a biased estimate of the true covariance, so it works out just fine to divide it by the product of the biased standard deviations. However, it is more likely that you will have calculated, or for some other reason will be dealing with, the unbiased standard deviations, in which case you have to adjust the numerator of the preceding formula to remove its bias. The unbiased covariance has a somewhat messier formula, but if you have unbiased SDs handy, it make sense to use the following formula for Pearson’s r:

It is important to realize that the two different formulas for r just presented will always give exactly the same value for r (except for possible differences in how you round off for the means and SDs). Which one you should use depends simply on whether you have the biased or unbiased SDs handy.

Now try this example:

2. Students who have taken courses in both areas are asked to rate on a 10-point scale how much they like math and how much they like statistics. The ratings for 10 random students appear next.

Student # / Math / Statistics
1 / 7 / 5
2 / 8 / 4
3 / 2 / 0
4 / 9 / 3
5 / 3 / 1
6 / 5 / 6
7 / 6 / 7
8 / 0 / 1
9 / 4 / 3
10 / 8 / 2

a. Is there a tendency for those who like math to also like statistics? Determine the strength of the linear relationship by calculating the correlation coefficient between these two sets of ratings (use the unbiased standard deviations in your calculation).

b. Draw the scatterplot for the data in this problem with the math ratings on the horizontal (X) axis. Is Pearson’s r a good way to summarize the relation between these two variables? Explain.

Now let’s try an example without the raw data, but with the necessary summary statistics:

3. You are convinced that your friend goes on more dates than you do because he’s the most outgoing, gregarious individual you have ever met. To try to confirm this idea, you collect data to determine the correlation between the number of dates young men go on in a month and their scores on the well-known Eysenck Extraversion Scale. Having collected values on both variables for 14 young men, you calculate the means and unbiased SDs for both variables, as well as the sum of their cross-products. Using the results that follow, and the appropriate raw-score formula, compute Pearson’s r.

Mean Extraversion Score (X) = 14.21

Mean number of dates per month (Y) = 6.3

= 1,465.46

N = 14

sx = 5.1

sy = 4.1

a. What value did you find for Pearson’s r? Would you consider this correlation coefficient to be large or small, or maybe in between (i.e., moderate)?

b. What can you conclude from this study about the causal relationship between extraversion and dating?

4. Correlations in Graphical Terms

In graphical terms, the relationships for varying r values may look something like this:

Now you should try these examples by guesstimating the Pearson r in each case:

5. Testing the Significance of Pearson’s r

Now that you know how to calculate the Pearson r and how to describe its magnitude, you need to know how to test r for statistical significance. As usual, the null hypothesis will be that there is no effect—no linear relationship—in the population (we represent the population correlation coefficient by the Greek letter rho, ρ, so H0 is that ρ = 0). Because we generally don’t know the population standard deviation for Pearson’s r, we have to use our old friend, the t test formula:

Bear in mind that N is the number of pairs of scores, and that the df for looking up the critical t is N – 2. As an example, let’s test the significance of the correlation you were supposed to calculate for the dating-extraversion example:

The critical t is t.05(12) = 2.179. Because 4.31 > 2.179, the correlation from our most recent example is significant at the .05, two-tailed level; the null hypothesis, that ρ = 0, can be rejected.

Although it can be instructive to do it, you really do not have to perform a t test to determine the significance of a Pearson r; it is much easier to simply look up a critical value for r in a handy statistical table, like the one in your text. All you have to know to find the critical r is the number of degrees of freedom associated with your calculated r (df = N – 2) and the alpha and number of tails you want to use. Once you have looked up the appropriate cutoff value in the table, compare it to your calculated r. If your calculated r is larger in magnitude than the critical r, you can reject the null hypothesis (e.g., r = –.4 is larger in magnitude than a critical r of .33, so the null hypothesis would be rejected in that case). Following is an abbreviated version of the critical r table in your text, which we will use to illustrate how to look up a critical value for r.

Level of significance for two-tailed test

df / .10 / .05 / .02 / .01
1 / .988 / .997 / .9995 / .9999
2 / .900 / .950 / .980 / .990
3 / .805 / .878 / .934 / .959
4 / .729 / .811 / .882 / .917
5 / .669 / .754 / .833 / .874
6 / .622 / .707 / .789 / .834
7 / .582 / .666 / .750 / .798
8 / .549 / .632 / .716 / .765
9 / .521 / .602 / .685 / .735
10 / .497 / .576 / .658 / .708
11 / .476 / .553 / .634 / .684
12 / .458 / .532 / .612 / .661

As you can see, for df = 12 and α = .05 (two-tailed), your r would need to be at least .532 to be significant. In the latest example, the calculated r is .78, which easily beats this critical r and allows you to declare statistical significance at the .05 (or even .01) level. Of course, this statistical decision is consistent with the conclusion we reached based on the t test we performed.

Now try these examples:

4. Test the correlation you calculated for Example #2 to determine whether it is significant at the .05 level. Can you reject the null hypothesis that the two ratings have a zero correlation in the population?

5. By using the t test formula:

(a) Would you attain significance with r = .37, when N = 89?

(b) What if your N, for the same r, was only 19?

6. (a) If you have 13 pairs of scores with a correlation of –.48, at what level for a two-tailed test do you have significance? Use a table of critical rs to answer this question.

(b) How many more pairs would you need to just barely attain significance at the .05 level?

6. Spearman Correlation (rs) for Ranked Data

Pearson’s r is rather easily thrown off by bivariate outliers in your data, and if the relationship between the two variables is steep at first and then bends over to become nearly horizontal (even without actually curving downward), this can lower your Pearson’s r quite a bit. You can obtain a more robust correlation coefficient, one that gives you a high value as long as one variable increases whenever the other one does, regardless of the actual amount, by converting the scores of both of your variables to ranks and then obtaining a Pearson r for the ranks. The r for the ranks is called the Spearman correlation for ranked data, and it is symbolized as rs. Also, if either or both of your variables has already been measured on an ordinal scale or in terms of relative ranks, rs is the correlation coefficient you should be using. We will illustrate the calculation of rs by using the following example. Imagine that you want to know whether the amount of time it takes someone to solve a spatial problem is related to how long it takes the same person to solve a verbal problem. Each participant solves both a spatial and a verbal problem (in a counterbalanced order), and the number of seconds it takes them to solve each problem is recorded. Here are the data for eight participants: