Choice and Preference

Chapter 9

Choice and Preference

Synopsis:

Behavior occurs in complex environments that provide the organism the opportunity to distribute behavior across multiple response options.The matching relation statesthat relative rates of response to alternatives or options are approximately equal to—match—the relative rate of reinforcement. Matching theory assumes that all behavior is choice behavior. Most choice behavior is explored with concurrent variable-interval (VI) schedules of reinforcement. However, some studies have used the Findley procedure that involves a changeover response. In general, studies of concurrent schedules work best when a changeover delay (COD) of at least two seconds is included. Response distributions are determined by the relative rate of reinforcement provided by available choice options. The matching relation has been demonstrated in humans and many other organisms. Matching describes the distribution of responses in human communication and self-injurious behavior. Mathematical equations describing choice behavior have been developed for single-response and concurrent-response options. Models of choice that include the relative rates of reinforcement, along with bias and sensitivity parameters, provide valuable insights into many aspects of choice behavior. The matching relation has been shown in the laboratory and has been applied to free-ranging wild pigeons and to examining human social behavior. An alternative approach to studying choice behavior developed as a merger between behavior analysis and economics: behavioral economics. Behavioral economics focuses on the relation between the value of available reinforcement and the cost (work) required to attain them. Applications of the findings from behavioral economics have led to advances in our understanding of substance abuse, impulsivity and self-control, and understanding of preference reversals in choice. Neuroscientists have begun to explore the underlying neural mechanisms associated with matching and choice behavior.

Study Questions:

How do behavior analysts define the following: (1) choiceand (2) preference?
What is a concurrent schedule of reinforcement?
Why are concurrent ratio schedules not typically used to study choice behavior?
How does steady-state performance on concurrent fixed-interval(FI) schedules differ from concurrent VI schedules?
Why is a COD an important and necessary feature of concurrent choice procedures?
What did Herrnstein’s proportional matching equation say about the distribution of responses across concurrent choices?
Write the matching equation for proportional response rates and identify the major components.
Write the matching equation for relative time spent on an alternative. Why might matching for relative time on a response alternative be more useful than basing matching on the relative response rates?
According to Myersonand Hale (1984), what schedule should be used when the goal is to increase a specific operant and the concurrent schedules are unknown? Why?
What is Herrnstein’s quantitative law of effect?
According to McDowell (2013), what do experimental findings regarding the constant k assumption suggest about the quantitative law of effect?
In Carr and McDowell’s (1980) study of matching and treatment of a 10-year-old boy, what were: (1) the target behavior and the hypothesized reinforcer, and (2) the general findings from the study?
What is the difference between maximizing and melioration?
Define the following behavioral economic concepts: (1) demand curve, (2) elastic, (3) inelastic, and (4) substitutability.
How have demand curves been used in the study of drug use?
What is the importance of the partial substitutability of sucrose for wheel running in the experiments by Belke and colleagues (2006)?
Write the hyperbolic discounting equation and identify each of the following variables: Vd, A,k,and d.
How do behavior analysts define self-control and impulsive behavior? How can a commitment response increase self-control?
Write the power law equation of matching and define each of the components. What values are associated with: (a) bias, (b) overmatching, and (c) undermatching.
What are the advantages of the log-linear generalized matching equation?
What two effects of rapid changes in relative reinforcement did Davison and Baum(2000) observe?
How does the administration of a D2 agonist (quinpirole) and a D1 agonist (SKF38393) influence observed behavior in rats? What affect does this have on the estimate of sensitivity in the generalized matching law?

Study Questions (Answers):

Choice is the distribution of operant behavior among alternative sources of reinforcement (options). Preference occurs when one alternative is selected more frequently than others.
Concurrent schedules involve two or more simultaneously available simple schedules that allow for the study of choice behavior.
Concurrent ratio schedules are seldom used to study choice behavior because they typically result in exclusive responding for the alternative associated with the highest rate of payoff (lowest ratio requirement).
On FI schedules, steady-state performance typically involves a majority of responses to the shorter FI schedule and then occasional shifts to the longer FI at approximately the time it is scheduled to payoff. On VI schedules, steady-state performance involves more of a distribution between the available response options.
The changeover delay, or COD, stipulates that responses have no effect immediately following a change from one schedule to another. The COD prevents frequent switching between alternatives and minimizes concurrent superstition—the inadvertent strengthening of the changeover response.
Herrnstein’s proportional matching equation stated that organisms match the relative rates of response (the main dependent variable) to the relative rates of reinforcement (the independent variable) across available response options.
The matching equation for proportional response rates is: Ba/(Ba + Bb) = Ra/(Ra + Rb), where B represents the rate of response for options a and b and R represents the rate of reinforcement programmed on options a and b.
Ta/(Ta + Tb) = Ra/(Ra + Rb). The extension of the matching law to continuous activities, such as standing in one place or looking at objects, is important as most behavior outside of the laboratory does not occur as discrete responses, and time allocation has been proposed as the more fundamental measure of choice and preference (Baum, 2015).
Myerson and Hale recommended the use of VI schedules to reinforce a specific operant when the concurrent schedules are unknown. VI schedules are best because they always redirect behavior to the desired alternative, although such a schedule may not eliminate inappropriate responding. Ratio schedules are less effective as they only work if they offer a rate of reinforcement greater than that available from the problem behavior.
The quantitative law of effect is an equation that describes the absolute rate of response on a single schedule of reinforcement and states that the absolute rate of response on a schedule of reinforcement is a hyperbolic function of the rate of reinforcement on the schedule relative to the total rate of reinforcement.
The constant k assumption that is required to derive the hyperbolic equation from proportional matching does not hold up to experimental tests. This finding suggests that the single-operant rate equation effect is untenable as a law relating reinforcement to response strength.
Carr and McDowell (1980) hypothesized that social attention, in the forms of reprimands, were the controlling stimuli for the young boy’s self-injurious behavior of scratching. They found that the hyperbolic single-rate equation accounted for nearly all of the variance in the data (99%). This finding demonstrates the relevance of matching theory to the natural ecology of human behavior.
Maximizing the overall rate of reinforcement refers to obtaining the highest overall rate of reinforcement from foraging among alternative patches. Melioration refers to doing the best at the moment and comes from Herrnstein’s idea that organisms are sensitive to fluctuations in the momentary rates of reinforcement rather than long-term changes in overall rates of reinforcement.
Demand curves show the relationship between the price of a commodity and consumption. A commodity is said to be elastic when the consumption of a commodity (reinforcer) changes with the price. Inelastic refers to the situation where the consumption of a commodity does not change with price. Substitutability occurs when consumption of one commodity decreases with an increase in price, and the consumption of another commodity increases.
Demand curves have been used to describe the reinforcing value of different drugs, to make comparisons of the demand for similar types of drugs and between drugs that are quite different, as well as to conceptualize drug treatments and offer new treatment strategies.
The partial substitutability of sucrose for wheel running reflects two energy-balance processes: the initiation and maintenance of travel or locomotion induced by loss of body weight and energy stores, and the termination of travel or locomotion as food supply increases.
The hyperbolic discounting equation is: Vd = A/(1+kd). In this equation, Vdis the discounted value of the outcome, A is the amount of the reinforcer (undiscounted), dis the delay to reinforcement, and k is the discounting rate, which determines how quickly the value of reinforcement decreases with delay (higher values of k are associated with greater discounting).
Self-control is defined as the choice of a larger-delayed reward while rejecting a smaller-immediate payoff. Adding a delay to both choice options increases preference for the larger-delayed option, resulting in a preference reversal from impulsive to self-control. A commitment response is some behavior emitted at a time prior to the choice point that eliminates or reduces the probability of impulsive behavior.
The power law equation of matching is: Ba/Bb = k(Ra/Rb)awhere B represents the response rates for options a and b and R represents the rates of reinforcement for options a and b. The parameter k represents bias, or preference caused by some unidentified factor. Bias is present if k differs from 1. The parameter a represents sensitivity. Values of a greater than 1 are overmatching, when changes in the response ratio are larger than the changes in the ratio of reinforcement. Values of a less than 1 represent undermatching, when the response ratio changes less than the reinforcement ratio.
The log-linear matching equation represents the matching law as a straightline that is easy to interpret. The value of the slope of the line measures sensitivity to the reinforcement schedules (overmatching and undermatching) and the intercept of the line reflects the amount of bias.
Davison and Baum (2000) observed: (1) preference pulses, a brief momentary increase in relative responding; and (2) lasting short-term effects on preference that influenced response ratios after at least the last six reinforcement presentations.
Administration of a D2 agonist (quinpirole) increased chewing and sniffing of the lever and food cup—behavior compatible with lever pressing—and produced overmatching. The D1 agonist (SKF38393) increased grooming and sniffing at some distance away from the lever—behavior incompatible with lever pressing and the reinforcement contingencies—andproduced undermatching.

Essay Questions (Student):

Describe the concurrent choice and Findley procedures that are used to examine choice behavior. What types of human social behavior do each of these procedures best model?

ANS: In the concurrent choice procedure, two or more simple schedules (i.e., FR, VR, FI, or VI) are simultaneously available on different response keys. Each key is programmed with a separate schedule of reinforcement, and the organism is free to distribute behavior between the alternative schedules. The Findley procedure involves a single response key that changes color. Each color is a stimulus that signals a particular schedule of reinforcement. The color and the programmed schedule may be changed by a response to a second key. This key is called the changeover key. Each response on the changeover key changes the stimulus color and schedule of reinforcement on the response key. The advantage of the Findley procedure is that the response of changing from one alternative to another is explicitly defined and measured.

Sunahara and Pierce (1982) suggested that the two-key procedure provides a model for social interaction. In a group discussion, for example, a person may distribute talk and attention to several group members. The Findley procedure may model role taking, in which an individual differentially responds to the social role of another person.

Discuss the results from the following studies on the preference for choice: (1) Leotti and Delgado (2011) and (2) Karsina, Thompson, and Rodriguez (2011). What was the major claim of the researchers based on their results? Overall, what do studies on preference for choice say about human choice behavior?

ANS: LeottiandDelgado (2011) found that even when the levels and probability of monetary reward were the same for both the choice and no-choice arrangements, participants selected the black key leading to a choice 64% of the time. Leotti and Delgado argued that making a choice is inherently rewarding.

Karsina, Thompson, and Rodriguez (2011) arranged for participants to have the choice between self-selected options (choice) or computer-generated options (no choice). Differential reinforcement established a preference for choice in six of seven participants, a preference that lasted even when the probability of points was again equated for the free-choice and restricted-choice options.

An overall conclusion is that preference for choice is a reliable finding and a history of reinforcement may establish and maintain this preference. Choice involves a history of selection between good versus better, or sooner versus later, but rarely between exactly equal alternatives. The simple option to “choose” suggests an opportunity to improve your situation, and is usually preferable.

Essay Questions (Instructor):

Explain how the studies of (1) Conger and Killeen (1974), (2) Borrero and colleagues (2007), and (3) McDowell and Caron (2010) demonstrate the applicability of matching to human behavior.

ANS: All three of these studies demonstrate matching in human communication. Conger and Killeen (1974) demonstrated matching in a group discussion situation with three experimenters (confederates) and one participant. The participant was not aware that the other group members were confederates in the experiment, and was asked to discuss attitudes toward drug abuse. Each listener reinforced the participant’s talk with brief positive words or phrases when a hidden cue light came on. The cue lights were scheduled so that the listeners gave different rates of reinforcement to the speaker. Results indicated that relative time spent talking to the listener matched relative rate of agreement (reinforcement) from the listener.

In Borrero and colleagues (2007), college students engaged in a 20-min discussion of juvenile delinquency in which they received agreement from confederates as alternative sources of reinforcement. The generalized matching law described the pooled data for relative response rates better than relative time spent talking.

McDowell and Caron (2010) used the generalized matching law to describe the verbal behavior of boys at risk for delinquency, as they interacted with their friends. The boys’ verbal responses were coded as either “rule-break talk” or “normative talk,” and positive social responses from peers were recorded as the presumed sources of reinforcement for the two verbal response classes. The generalized matching law provided an excellent description of the boys’ allocation of verbal behavior, with some deviation from exact matching and bias toward normative talk. Deviation from matching became extreme and the bias toward normative talk lessened as the risk for delinquency of the child increased.

Write the following matching equations: (a) the matching equation for proportional response rates, (b) the power law equation of matching, and (c) the log-linear form of the generalized matching law. How are each of these formulas similar and how do they differ? Why are the generalized matching laws superior to the matching law in their ability to describe behavior?

ANS: The four matching equations are: (a) Ba/(Ba + Bb) = Ra/(Ra + Rb), (b) Ba/Bb = k(Ra/Rb)a, and (c) log(Ba/Bb) = log k + [a× log(Ra + Rb)].

All three equations are similar in that they relate relative rates of response to relative rates of reinforcement. The equations also are similar in that they express the rates of reinforcement on a given alternative relative to other sources of reinforcement in the environment, and rates of response on a given alternative relative to responding on other available options. The two generalized matching equations express relative rates of reinforcement and response as ratios, rather than proportions. Additionally, the generalized matching equations include two additional terms or parameters, k and a, that represent bias and sensitivity, two additional sources of influence on the choice behavior.

The proportion equation describes the distribution of behavior when alternatives differ only in rates of reinforcement. In complex environments, other factors also contribute to choice and preference. The ratio equations include terms for bias (k) and for sensitivity (a) which allows the model to provide a more complete representation of howalternative sources of reinforcement influence operant behavior.

Videos:

Howard Rachlin: Matching (SQAB tutorial)

William Baum: Dynamics of choice (SQAB tutorial)

Amy Odum: Delayed discounting: what, who, when, where, why, and how (SQAB tutorial)

Warren Bickel: Discounting and drug abuse

Definition of concurrent schedules

Additional Videos: See additional video material for this chapter compiled by Brady Phelps in the supplemental section (videos in the supplemental section have not been reviewed by the primary authors of the textbook).

Study Tips:

Behavior analysts define choice as the distribution of responses between concurrently available response options. Preference occurs when a majority of responses occurs to one of the response options, relative to the other options.
All behavior is choice behavior. Even in the simplest of situations, the organism has a choice between responding or not responding.
The basic paradigm for studying choice behavior includes: two or more concurrently available schedules of reinforcement, variable interval schedules, and a COD of at least 2s.
The matching relation has been shown to describe human communication behaviors, animal foraging and feeding, self-injurious behavior, and many other behaviors across many different species.
Herrnstein’s proportional matching relation states that the relative rates of response equal or match the relative rate of reinforcement. The generalized matching law,which models choice as a ratio and includes parameters for bias and sensitivity, is the most general statement of the matching relation.
Delay discountinginvolvesthe behavioral economics of valuein situations that arrange choice between small, immediate and large, delayed outcomes (e.g., $70,000 immediately versus $100,000 in a year). Adding delay to the larger reward rapidly discounts its value (hyperbolic discounting), reducing the reinforcement effectiveness of the stimulus. Delay discounting helps to explain impulsivity, self-control, preference reversals, and the power of commitment.