What Is the Difference Between an Observational Study and an Experiment?

NOTES 4.2 – 8/14/15 – EXPERIMENTS

What is the difference between an Observational Study and an Experiment?

-Observational Study – OBSERVES individuals and measures variables of interest but does not attempt to influence the results.

-Experiment – deliberately imposes some treatment on individuals to measure their responses.

Example of observational study: Wanting to know what type of bands are most represented on a t-shirt….I go to the mall and sit on a bench and make a list of people that walk by me and what band (if any) they have on their shirt.

Example of an experiment: 0in%20America/Rock-n-Roll/mice.htm

We will come back to this in a few.

Explanatory variables: explain or predict changes in response variable (may be called factors)

Response variable: Measures an outcome of a study

Confounding variables: two variables are associated in such a way that their effects on a response variable cannot be distinguished from one another OR two variables are confounded if it is impossible to determine which of the two variables is causing a change in the response variable.

CYU on Page 237

When doing an experiment, you will have a treatment, experimental units and subjects.

TREATMENT – Specific condition applied to the individuals in an experiment

EXPERIMENTAL UNITS – the smallest collection of individuals to which treatments are applied

SUBJECTS – when experimental units are humans

EXAMPLE: A study published in the New England Journal of Medicine compared two medicines to treat head lice: an oral medication called ivermectin and a topical lotion containing malathion. Researchers studied 812 people in 376 households in seven areas around the world. Of the 185 households randomly assigned ivermectin, 171 were free from head lice after 2 weeks compared with only 151 of 191 households randomly assigned to malathion. What is the experimental unit, explanatory variable, response variable and treatment?

Example: Does adding fertilizer affect the productivity of tomato plants? How about the amount of water given to the plants? To answer these questions, a gardener plants 24 similar tomato plants in identical pots in his greenhouse. He will add fertilizer to the soil in ½ of the pots. Also, he will water 8 of the plants with.5 gallon of water per day, 8 of the plants with 1 gallon of water per day and the remaning 8 plants with 1.5 gallons of water per day. At the end of 3 months, he will record the total weight of tomatoes produced on each plant.

Experimental units or subjects?

Explanatory variables?

Response?

Treatments – NAME ALL OF THEM

(Ted Talk if time)

BAD EXPERIMENT 

A high school regularly offers a review course to prepare for the SAT. Budget cuts will allow the school to offer only an online version of the course. Supposed the group of students who take the online course earn an average increase of 45 points in their math scores from a pretest to the actual SAT. Can we conclude that the online course is effective?

Who are the subjects?

What is the treatment?

What is the outcome?

Why is this not a good experiment?

Are there confounding variables?

The way to actually fix the confounding is to do a comparative experiment where some students are taught in the class and some online and all the students are similar. Most experiments that are well designed have more than one treatment.

We can’t however let students choose if they want to do class or online, they need to be assigned and told where to go – RANDOM ASSIGNMENT

Let’s do the experiment again….We can have the online SAT course and a classroom SAT course. 50 students have agreed to participate in the an experiment comparing the two methods.

-How can you randomly assign 25 students to each of the two methods?

When doing something like this we need to control some variables

EX – the pretest needs to be the same; take SAT on same day; same amount of study time

We also need replication – This has a different meaning in statistics….means use enough experimental units so that any differences in the effects of the treatments can be distinguished….what would happen if I had one kid in each group?

So to design a good experiment we need

Comparison (two or more treatments)

Random Assignment

Control

Replication

What is a placebo????

Read the example on 243….Who did the results of this experiment apply to?

Completely randomized design – means the experimental units are assigned to the treatments completely by chance. (flipping a coin

Can you make a completely randomized design?

Psychologists want to know how different types of television shows impact young children. They recruit 60 four year olds and have them watch 9 minutes of a fast paced children’s program, 9 minutes of a slow paced program, or draw for 9 minutes. After 9 minutes, each child will complete several tasks, including tests for mental ability and impulse control. Describe a completely randomized design for this experiment. We want 3 equal groups.

CYU page 247

Double Blind Experiment – The subject nor the person doing the experiment knows which treatment was received….KNOW THIS!!!

CYU 249

Are your results in an experiment statistically significant? If they are statistically significant, it is so large that it would rarely occur by chance alone.

Your assignment 

Page 259

45, 47, 48, 49, 51, 53, 55, 58, 59, 61, 63, 67, 68, 69, 71, 72, 73