Sociology 128

Jess Tollette

Section 5: Sampling and Surveys

Sampling:

Professor Waters wants to survey students in Sociology 128 about their experience in the course. She does not have the time to survey all 72 students, so she decided to sample just 20% of the class.

  • If she wanted to select a simple random sample, how would she do this?

-Assign a number to every person enrolled in her class. Then use some sort of random number generator or computer software to select students at random.

  • What are the advantages of a simple random sample?

-It is unbiased – every student has the same chance of being chosen during the sampling process.

-Should be representative.

-It is fairly easy to do. You don’t need a lot of information about your population; you just need your sampling frame.

  • What are the disadvantages?

-There are only 72 people in the class and therefore the sample to begin with is small. You might find that taking a subsection of the class (14 people) causes sampling error.

-For larger populations it can be quite laborious and time consuming.

  • What is a systematic sample? When is it most convenient to use?

-Start with a sampling frame and choose every nth person on the list.

  • How the list is arranged matters

-Every element has the same probability of being selected but not every combination can be selected

  • To ensure that everyone has the same chance of being selected, the starting point has to be random

-Use when drawing a simple random sample is too difficult or when the list of sampling is long and not computerized

  • Imagine Professor Waters is most interested in finding out what students think of section. What sampling strategy would you recommend? Why?

-Stratified sampling. Stratify by section (one group for Asad, one group for Beth and one for me); each of the sections is a stratum, so you use stratified sampling so you don’t accidentally end up with a disproportionate amount from one TF’s sections

-This ensures that the sample resembles the population of 128 students as opposed to random sample or systematic sampling, which may divide students unevenly between the threeTFs.

  • Professor Waters is considering standing in the back of the lecture hall at the end of class and speaking to the first 14 students who exit the room. What are the issues with this approach?

-This might create selection bias where the first 14 people to leave are not representative of the larger class (maybe they are all in Asad’s section)

  • Imagine Harvard commissions Professor Waters to evaluate students’ satisfaction with university life across campus using face-to-face surveys and a limited budget. How might you advise Professor Waters here?

-Cluster by discipline/house/ some sort of metric in which the clusters are heterogeneous in terms of who is represented but similar in terms of the metric that you are using

-stratify by gender/race (or whatever other metric you might think is important)

-You use clusters for convenience because you can’t survey everyone, so this helps to keep costs down. (but errors increase) Let’s say Harvard has enough resources to interview 400 people

-Cluster by discipline (humanities, social sciences, natural sciences, applied sciences) because you want to survey people face-to-face, so you randomly select people from whatever concentration within each of these clusters and you survey them wherever they are.

SURVEY DESIGN: Do’s and Don’ts

  • Avoid double-barreled questions(i.e. asking 2 different things in one question)
  • Try deal with social desirability bias by asking questions in a way that seems more acceptable
  • Include reliability checks to make sure that people aren’t being lazy with their survey responses
  • Avoid negative terms
  • Avoid bias introduced by identifying an attitude with a prestigious person or agency
  • Ask boring questions first in face-to-face interviews and last in questionnaires
  • Keep the language simple and unambiguous
  • Ask sensitive questions at the end (e.g. income, sexual activity)
  • Use cards or other aides for sensitive questions
  • Use scales and indexes for multiple measures of complex higher order concepts

______

Sample Questions:

1. Do you like to shop at Trader Joe’s and Whole Foods?

Problem: double barreled question

Solution: separate it into two separate questions

2. What brand of computer do you own?

a. IBM PC

b. Apple Mac

Problem: It’s a forced choice question AND (it isn’t mutually exclusive)

Solution: add in a C which allows you to write in OR add a C category other OR expand to include other brands, Dell, etc.

3. What kind of music do you like?

a. rap

b. pop

c. country

d. other

Problem: not mutually exclusive

Solution: Maybe rephrase the question to ask what your favorite type of music is.

4. Are you in favor of Proposition 12?

Problem: It is not clear what proposition 12 is based on your question.

Solution: Rephrase the question to indicate to the survey taker what Proposition 12 is

5. Are you concerned about high crime rates in your city?

a. yes

b. no

Problem:introduces social desirability bias. Everyone will say yes, of course.

Solution:Which of the following is most concerning to you? Or, rank the following in the order of concern to you. Or, of the following things, rate how much you worry about them on a daily basis on a scale of (low) 1-5 (high).

6. Do you agree with how Governor Christie dealt with Hurricane Sandy?

a. Agree

b. Disagree

c. Don’t know

Problem: There is a bias here (invoking of authority).

Solution: Avoid identification of an attitude with a prestigious person or agency.

7. What do you think of the food in Harvard’s dining halls?

a. poor

b. average

c. good

d. very good

e. excellent

Problem: unbalanced

Solution: very poor – poor – average – good -- excellent

8. Because they are better at managing the home, should women be prevented from entering the labor force?

a. strongly agree

b. agree

c. disagree

d. strongly disagree

Problem: avoid loaded or leading questions

Solution: just ask the question without assuming anything about women