Probability Judgment

1

Probability judgment

Behavioral Economics

SS200 Winter 04

Colin Camerer

How do people judge probabilities?

Events:

Hussein executed by end 2004

Bin Laden found alive by end 2004

Bush reelected in 2004

Health

Your death this year

What cause?

Mad cow deaths in US 2004

Law

Kobe Bryant convicted

Finance

S&P 500 ends 2004 above 1300 [currently 1100+ 1/27/04]

Entertainment

2004 domestic box office gross of “Jersey Girl” above $100 million

Sports

Lakers win 2004 NBA championship

I: Heuristics-and-biases approach: (Tversky-Kahneman 74 Science etc)

People probably use rule-of-thumb "heuristics"

Heuristics can be discovered by the "biases" (systematic departures from laws of probability) they produce

Representativeness

Availability

Anchoring

NOTE WELL: The emphasis on biases *does not* mean heuristics are bad. KT had in mind optical illusions as an analogy—focussing on illusions is just a way to understand the mechanisms, not an assertion that people can’t see.

Dual-system view (Kahneman-Frederick, 2000+)

Definition of rational choice: System 2 always overriding system 1!

Typical system 1 `mistake’:

Attribute substitution:

“...when an individual assesses a specified target attribute of a judgment object by substituting another property of that object—the heuristic attribute—which comes more readily to mind.” (Kahneman 03 p 1460)

E.g. prototype heuristics

Use “prototype” to compute category properties

Prototypes are accessible; sums are not (have to be computed)

--> neglects “extension” (set properties), can lead to dominance violation, (Alevy-List card study) etc

look how easily “average line length” springs to mind...but “total line length” does not.

Potential example in law:

Match defendants to “prototypes”

OJ doesn’t fit murderer prototype

Kobe doesn’t fit rapist prototype

Michael Jackson doesn’t fit child molester prototype

(fits “eccentric pop star” prototype *too well*)

Martha Stewart doesn’t fit inside trader prototype

even when evidence is strong, (system 1) prototype-match fights back

-->”Representativeness (pre-2000)

Leads to systematic violation of normative principles

(...unless system 2 checks)

1. “Law of small numbers”

Short sequences should be representative of the statistical process that generated them.

Systematic misperception of randomness

e.g. self-generated “coin flip sequences” are too close to 50% heads,

too few long runs, too many reversions

--> Truly random sequences seem to have "momentum"

So...misplaced faith in the "hot hand" in basketball, baseball.

Hot hand evidence here

Beetle Bailey cartoon

Models: Budescu and Rapoport (1997? Psych Rev)

People maintain window of k previous trials.

Predict next trial t+1 will “balance” subsequence of k+1

(i.e., make relative proportions=probabilities)

Model predicts *very* accurately

Rabin (2003 QJEcon):

People act as if sampling without replacement

Interesting new prediction:

Suppose there are good, neutral, bad forecasters

After observing a streak of success, people mistakenly

think forecaster is likely to be good (because “nobody could be that lucky”)--> “illusion of expertise”

Explains puzzle: Why are there 5000+ mutual funds?

Investors all think they can pick a winner

--> Rabin model predicts “churning”—too much hiring and firing of experts (sports coaches, money managers)

1. Prototype heuristic--> underweighting of base rates

Representativeness has no special role for base rates.

P(major|Tom W profile) is judged by similarity, ignores base rates of majors

(NB: When profile is “bland”, major base rates *are* used)

Other examples:

-- probability of obscure major

-- Is she from Tibet or China?

-- Is he/she an actor or a teacher?

Med school: "If you hear hoofbeats, it's a horse not a zebra"

4. Violations of extensionality and logic

Linda the feminist bank teller

P(Banker & Feminist) P(Feminist), P(Bank teller)

But Linda seems more representative of the class B & F

than of the class B.

--> Violations of conjunction rule

Caltech: 20 0 violations

22 1 violation (usually P(B&F) > P(B) )

3 2 violations

--> effect disappears when people are asked relative frequencies

“Out of 100 people like Linda, how many are bank tellers?”

--> question format makes checking for conjunction easy

--> General: Adding detail to a scenario may make it seem more likely

adding a less painful ending “improves” memory of a colonoscopy (above, Kahneman 03 from Kahneman-Redelmeier)

1. ignore statistical extremity--> surprised by regression to the mean

[note this may be a failure to condition on “hidden” selection]

P(extreme observation in t+1 | extreme observation in t)

seems likely

examples: movie sequel grosses

"sophomore slump" in record sales

Sports Illustrated curse

pilot performance in Israeli army

Extreme observations likely to be followed by less extreme observations

Availability heuristic

Judge probability of X by how easily instances of X can be retrieved.

--> memorable events will be overjudged

-- recency effect (earthquake insurance purchase rises

after major quake)

-- Is r more likely to occur as 1st or 3rd letter?

-- Fault tree bias: Probability of "all other causes" too low

(Mickey Rourke, Body Heat: 50 things you forget...)

-- Illusory correlation: Perceived correlations depend on

strength of associative bond.

Association may overwhelm statistical evidence

Gay men draw big muscles in Draw-A-Person test (false)

Paranoids draw big eyes (false)

Gay men draw human/animal hybrids (true)

Fueled by confirmation bias: Overweight +/+ cell

(Secret to “cold reading” by psychics)

-- Egocentric self-serving biases of credit/blame.

Married couples: What % of fights do you start?

What % of chores do you do?

Both add to 110%

Partition-dependence

Most numerical events are not naturally partitioned. Partition can influence perceived probability (Fox et al)

High-stakes example:
1/n heuristic

II. Optimism and overconfidence

Overconfidence about relative skill

People think they are above average relative to others in most categories

Dunning et al “scope” effect

-- if question is narrowly posed, OC shrinks

a. Overconfidence in probability judgments

Calibration literature

Binary questions (picture)

Compare relative frequency in category p with subj probability p

Experts:

Weather forecasters

Blackjack etc

80 year olds (“experts at life”)

Trust game experiments

Figure: Calibration curves for young (20s) and older (80s) subjects. Note that both “resolution” (more points at 100% confidence) and calibration is better for older subjects in their 80’s. From Kovalchik et al (2003, JEBO)

Figure: Calibration of players in repeated trust games. Probability is belief that a player will not repay trust. Four belief assessments (using quadratic scoring rule) are averaged together to get average belief on the x-axis. Note that calibration is nearly perfect. From Camerer, Ho and Chong (2002, in revision) “Strategic teaching...”

b. Confidence intervals for numerical quantities

Typical result: Way too narrow

90% CI’s contain true number only 50% of the time

searching for a dropped contact lens

errant golf shot

estimating project cost/completion time

Market level: Is there a demand for overconfidence? Yes (see Price and Stone, JBehavl Dec Making 2003). Subjects prefer people who make extreme forecasts.

“Portfolio” effect in CI estimation:

Give 10 90% CI’s.

Asked: How many contain the true number?

Actual answer (given overconfidence)= 5 of 10

Subject answer = 5 of 10

any one CI looks right...but people know they are wrong as a whole

Kahneman/Lovallo business management (Mgt Sci 1990+)

inside view/outside view

inside: rich, emotive, narrative, bias-prone

outside: bland, comparative, automatically debiased

E.g. corporate mergers

Marriages

c. Transparency biases

Hard to imagine that others do not have the same information you have

Children—“false beliefs” (show cartoon)

Illusion of transparency (Gilovich etc)

“the tapper” study

“Curse of knowledge”

formal: Difficult to recover coarse partition from fine-grained one

Piaget example: New PhD’s teaching

EA Poe, “telltale heart”

Computer manuals

Hindsight bias

Recollection of P_t(X) at t+1 biased by whether X occurred

“I should have known!”

“You should have known” (“ignored warning signs”)

--> juries in legal cases (securities cases)

implications for principal-agent relations?

III: Are brains Bayesian?

Bayes’ rule

Prior [from where?]

Posterior updated from prior

P(H_i|data)=P(data|H_1)* P(H_1)

P(H_2|data) P(data|H_2) P(H_2)

Key properties:

• Separability of likelihood evaluation from prior

no encoding/confirmation bias in evaluating evidence

cf. Rabin-Schrag model (evidence consistent with more likely hypothesis is encoded more accurately)

• Choice modelsNo wishful thinking

subjective probabilities should be independent of whether outcomes are favorable

wishful thinking may be motivating/evolutionarily adaptive

• Logically coherent
• Conjunction: P(bank teller *and* feminist)P(bank teller)
• Probabilities should add to one (mutually exclusive, exhaustive)

Osherson-Smith data: Prob judgments generated in left hemisphere, deduction in right hemisphere (a la systems 1-2)

Common claim: Evidence is consistent with Bayesian updating --> Bayes’ rule is the best available model

Conjecture: all evidence in favor of Bayesian updating is actually just monotonicity

i.e. when P(data|H_1)/P(data|H_2)> 1 then P(H_1|data)/P(H_2|data) rises

But many adjustment rules have this property!

IV: Experimental economics and the study of probability judgment

1. Abstract stimuli vs natural events??

pro: can precisely control information of individuals

can conpute a Bayesian prediction

con: maybe be fundamentally different mechanisms than for concrete events...

2. Do markets eliminate biases?

Yes: specialization

Market is a dollar-weighted average opinion

Uninformed traders follow informed ones

Bankruptcy

No: Short-selling constraints

Confidence (and trade size) uncorrelated with information

Camerer (1987)

Experience reduces pricing biases but *increases* allocation biases

Contingent claims markets:

Markets enforce correct prices..BUT probability judgment influences allocations and volume of trade (example: Iowa political markets)

DConv04 (1/28/04)
2004 Democratic National Convention Nomination Market

This graph is also available in flash format with the datapoints annotated, or in png format.

 Graph in Flash format (this image may take a few seconds to load)

 Graph in PGN format (depending on your browser, this image may appear clearer than the one above)