The Role of Call Qualityin Votermobilization

The Role of Call Qualityin VoterMobilization:

Implications for Electoral Outcomes & Experimental Design

Supplemental Materials

Christopher B. Mann

Louisiana State University

Department of Political Science

and

Manship School of Mass Communication

Journalism Building

Baton Rouge, LA 70803

(202) 295-7834 cell

Casey A. Klofstad

University of Miami

Department of Political Science

1300 Campo Sano Avenue

Coral Gables, FL 33416

We thank our partner organization for the support that made this research possible. We thank the editors, anonymous reviewers, Stephen Ansolabehere, Donald Green, John Love, Frank Sansom, Brian Shaffner, and the participants in the Political Science Faculty Colloquium at the University of Miami for their helpful comments. An earlier version of this paper was presented at the 2012 Midwest Political Science Association Conference. This research was conducted under University of Miami Human Subjects Research Office Protocol #20110124. Replication data will available at sites.google.com/site/christopherbmann. All errors are the responsibility of the authors.

1. Appendix - Call Script

[Instruction to callers: This script is intended as a guide to a “chatty” conversation with the voter. Please stick to it as closely as possible, but keep the interaction as natural as possible.]

Hello, is ______there?

Hi! This is ______calling from the [partner organization’s name]. We’re not asking for money today and we’re not campaigning for or against any candidate.

We’re just calling to thank you because our records show you voted in recent elections. Since you’re the kind of person who cares about your community and who votes, we know we can we count on you to join the thousands of people like you who will vote on Tuesday.

1)On Tuesday, are you planning to vote in the morning, at lunchtime, in the afternoon, or in the evening?

[If unsure of time/no response, prompt]: It’s important to plan ahead so you don’t forget to vote on Election Day. When do you think will be most convenient for you to vote on Tuesday, November 2nd?

Record:

1 – if response is a time of day (morning, lunch, afternoon, evening, any specific time)

2 – if response is already voted early or by mail -(do not read) – That’s great. Thanks for voting already. Sorry to bother you. Goodbye.

3 – if response is Not sure/Don’t know

4 – Refused

2)Do you plan to vote while you are on your way to [from] work or out running errands, or do you have to make a special trip to go to vote?

[Record]:

1 – To/From work or errands

2 – Special trip

3 – Not sure of polling place location

4 – [Refuse]

[If #1 or #2 from Question 2] Great!

[If No/Not Sure from Question 2] That’s okay. You can look up your polling place on the web at Vote411 dot org. You can also call your local election office to find out where to vote.

3)There are a lot candidates and issues on the ballot this year, and each of them is important for our future. We’re asking people to pledge to fill out their entire ballot, can we count on you to try to fill out the entire ballot?

[Record]:

1 – Yes

2 – No

3 – Maybe/Don’t Know

4 – [Refuse]

[If No/Maybe/Don’t know] I know there are a lot of things on the ballot but each of them is important to our future. Please cast your vote on as many as you can because there is so much at stake at the local, state and national level this year.

4)Many people are more likely to remember to vote if they get a reminder. I’d like to send you a reminder about voting on Tuesday. Can you please give me an email address where I can send you a reminder? We won’t give or sell your email to anyone else.

Email: ______@______

[If email provided] Thanks!

[If email not provided] That’s okay.

[Close - ALL] I know life can get hectic, but it’s important to remember to vote this Tuesday even if things come up. It looks like a lot of people will be voting this year, so thank you for being a good citizen who votes and for your promise to vote on Tuesday!

Thanks for your time. Goodbye.

1

2. Description of the Call Centers used for each Treatment

• Treatment A: Calls conducted by a firm that operates its own caller center for policy advocacy and election-related communication. Callers are hired, trained, and supervised by the firm’s staff. The firm reported that most callers working on the project had worked for the firm for at least several months through the election season. Some additional callers were hired to handle the late election season work, and therefore had only a few weeks of experience.
• Treatment B: Calls conducted by a firm that normally operates call centers for telephone fundraising solicitations for non-profit organizations. The callers are highly trained, had worked for the firm for an average of eighteen months, and are retained because they are successful at soliciting fundraising pledges. In many ways, these callers are the highest quality telemarketing professionals available for voter mobilization calls.
• Treatment C1: Call conducted by a firm that supervises the calling program, but out-sources/subcontracts the actual calling to independent commercial call centers. The calls were conducted by what was described as the firm’s preferred commercial call center. Firm staff reported being on-site for training callers on this program, but the callers were not hired or directly supervised by firm staff. The commercial call center used for this project conducts non-political telemarketing calls as well as election-related calls.
• Treatment C2: Call conducted by a firm that supervises the calling program, but out-sources/subcontracts the actual calling to independent commercial call centers. The calls were conducted by a commercial call center selected by the firm. Firm staff reported supervising training by phone. The callers are not hired or directly supervised by firm staff. The commercial call center used for this project conducts non-political telemarketing calls as well as election related calls.

3. Selection of Experimental Population

Our partner organization selected the experimental population based on the following criteria (see Figure S1):

• Registered voters in eleven states with a “strong” match between phone listings and voter file records according to Catalist LLC, a consumer data firm specializing in information on registered voters.[1]
• Registered voters with a predicted probability of voting between 30 percent and 70 percent based on a predictive voter turnout model provided by Catalist LLC. This criterion was based on previous research that voter mobilization contacts have maximum impact for registered voters with a 50-50 chance of turning out (Green and Gerber 2008 p. 174; Arceneaux and Nickerson 2009; Niven 2004; Hillygus 2005; Parry et al. 2008).
• Registered voters expected to trust information about political issues from our partner organization based on a proprietary micro-targeting model.
• Multiple voter households were excluded from this experimental population, defined by multiple registered voters associated with the same phone number.
• Registered voters who had requested an absentee/mail ballot, or cast early in-person ballot prior to October 27, 2010, were excluded from the experiment.[2]

[Figure S1 about here]

4. Stratified Random Assignment

Random assignment was conducted separately in two blocks. This stratification of the random assignment was necessary because a subset of registered voters in four states (IL, MI, NY, and PA)had a different probability of assignment to the experimental conditions. This subset of voters wasrandomly assigned to the conditions for this experiment, or to other experiments using voter mobilization phone calls reported elsewhere (Arceneaux, Mann and Nickerson 2012; Mann andKlofstad 2012; Mann and Sinclair 2012).

[Table S1 about here]

We designated all records in this subset of registered votersas “Block A”. All remaining records are designated as “Block B”. Therefore, random assignment in each block was conducted separately (a stratified randomization process or blocked randomization process). Tables S1a and S1b demonstrate that the random assignment process produced control and treatment groups in each block that appear well balanced across observable covariates for age, sex, race, past voting, and geography. Our analysis uses a fixed effects estimator to account for the difference in probability of assignment to the two blocks.

Figure S1: Selection of universe for experiment

(by partner organization)

1

Table S1: Stratified Random Assignment and Covariate Means
Table S1a: Randomization Block A: IL, MI, NY, PA
Assignment / N / Age / Female / African American / Hispanic / Voted in 2008 General / Voted in 2006 General / Voted in 2004 General / IL / MI / NY / PA
Control Group / 80,073 / 43 / 65% / 11% / 6% / 91% / 25% / 61% / 27% / 18% / 30% / 25%
Treatment A / 46,755 / 43 / 65% / 11% / 6% / 91% / 26% / 61% / 27% / 18% / 30% / 25%
Treatment B / 40,290 / 43 / 65% / 11% / 6% / 90% / 26% / 61% / 27% / 17% / 30% / 25%
Treatment C1 / 43,164 / 43 / 65% / 11% / 6% / 91% / 25% / 61% / 27% / 17% / 30% / 25%
Treatment C2 / 43,742 / 43 / 66% / 11% / 6% / 91% / 25% / 61% / 27% / 17% / 30% / 25%
Total / 254,024 / 43 / 65% / 11% / 6% / 91% / 25% / 61% / 27% / 17% / 30% / 25%
Table S1b: Randomization Block B: Remaining States
Assignment / N / Age / Female / African American* / Hispanic* / Voted in 2008 General / Voted in 2006 General / Voted in 2004 General / FL / IA / ME / MN / NM / OH / SC
Control Group / 63,905 / 43 / 65% / 5% / 4% / 95% / 27% / 61% / 31% / 11% / 7% / 31% / 2% / 16% / 3%
Treatment A / 86,344 / 43 / 65% / 5% / 4% / 95% / 27% / 61% / 31% / 11% / 7% / 31% / 2% / 16% / 3%
Treatment B / 73,621 / 43 / 65% / 5% / 4% / 95% / 27% / 61% / 31% / 11% / 7% / 31% / 2% / 16% / 3%
Treatment C1 / 79,913 / 43 / 65% / 5% / 4% / 95% / 27% / 61% / 31% / 11% / 7% / 31% / 2% / 16% / 3%
Treatment C2 / 79,677 / 43 / 65% / 5% / 4% / 95% / 27% / 62% / 31% / 11% / 7% / 31% / 2% / 15% / 3%
Total / 383,460 / 43 / 65% / 5% / 4% / 95% / 27% / 61% / 31% / 11% / 7% / 31% / 2% / 16% / 3%
Notes: African American and Hispanic indicated by commercial coding of ethnicity from Catalist LLC, a firm specializing in individual voter data.

1

5. ProbitRegression Estimation of Average Treatment Effects

Table S2:Average Treatment Effects on Voter Turnout Using Probit Regression

Model 1 - No Covariates / Model 2 - Incl. Covariates
Β
(probits) / Robust Std Error / β
(probits) / Robust Std Error
Treatment A / 0.0109⁺⁺ / (0.0049) / 0.0111⁺⁺ / (0.0050)
Treatment B / 0.0181⁺⁺⁺ / (0.0051) / 0.0174⁺⁺⁺ / (0.0052)
Treatment C1 / 0.0037 / (0.0050) / 0.0024 / (0.0051)
Treatment C2 / 0.0037 / (0.0050) / 0.0028 / (0.0051)
Randomization Block A / -0.1027*** / (0.0033)
Age / 0.0069*** / (0.0001)
Female / -0.0790*** / (0.0034)
African American^ / 0.1516*** / (0.0062)
Latino^ / -0.0729*** / (0.0080)
Voted in 2008 General / 0.3731*** / (0.0068)
Voted in 2006 General / 0.3795*** / (0.0038)
Voted in 2004 General / 0.0679*** / (0.0034)
IA / 0.4081*** / (0.0074)
IL / 0.3714*** / (0.0062)
ME / 0.3945*** / (0.0089)
MI / -0.3313*** / (0.0079)
MN / 0.4891*** / (0.0054)
NM / 0.4155*** / (0.0150)
NY / 0.2241*** / (0.0061)
OH / 0.1854*** / (0.0066)
PA / 0.2811*** / (0.0064)
SC / 0.5572*** / (0.0126)
Constant / -0.2454*** / (0.0038) / -1.2762*** / (0.0105)
Observations / 637,484 / 637,484

Notes: Standard errors in parentheses. Florida is the omitted state in Model 2. ^ African American and Hispanic indicated by commercial coding of ethnicity from Catalist LLC, a firm specializing in individual voter data. One tailed test: ⁺p < 0.05, ⁺⁺p < 0.01, ⁺⁺⁺p < 0.001; Two tailed test: *p < 0.05, **p < 0.01, ***p < 0.001.

6. Instrumental Variable Estimation of Complier Average Causal Effects

Table S3: Complier Average Causal Effects on Voter Turnout Using Two-Stage Least Squares Regression

Model 1 - No Covariates / Model 2 - Incl. Covariates
β
(pct points) / Robust Std Error / β
(pct points) / Robust Std Error
Treatment A: Contacted / 0.0087⁺⁺ / (0.0039) / 0.0084⁺⁺ / (0.0038)
Treatment B: Contacted / 0.0144⁺⁺⁺ / (0.0041) / 0.0132⁺⁺⁺ / (0.0040)
Treatment C1: Contacted / 0.0018 / (0.0025) / 0.0012 / (0.0025)
Treatment C2: Contacted / 0.0020 / (0.0027) / 0.0014 / (0.0026)
Randomization Block A / -0.0393*** / (0.0013)
Age / 0.0025*** / (0.0000)
Female / -0.0292*** / (0.0013)
African American^ / 0.0571*** / (0.0024)
Latino^ / -0.0253*** / (0.0028)
Voted in 2008 General / 0.1329*** / (0.0023)
Voted in 2006 General / 0.1426*** / (0.0014)
Voted in 2004 General / 0.0244*** / (0.0012)
IA / 0.1516*** / (0.0028)
IL / 0.1371*** / (0.0023)
ME / 0.1460*** / (0.0033)
MI / -0.1066*** / (0.0024)
MN / 0.1833*** / (0.0020)
NM / 0.1554*** / (0.0058)
NY / 0.0815*** / (0.0022)
OH / 0.0671*** / (0.0024)
PA / 0.1028*** / (0.0024)
SC / 0.2084*** / (0.0049)
Constant / 0.4031*** / (0.0015) / 0.0295*** / (0.0036)
Observations / 637,484 / 637,484

Notes: Standard errors in parentheses. Florida is the omitted state in Model 2. In the 2SLS regression to estimate the CACE, random assignment to each treatment used as instrument for successfully delivering the script to the targeted voter. ^ African American and Hispanic indicated by commercial coding of ethnicity from Catalist LLC, a firm specializing in individual voter data. One tailed test: ⁺p < 0.05, ⁺⁺p < 0.01, ⁺⁺⁺p < 0.001; Two tailed test: *p < 0.05, **p < 0.01, ***p < 0.001.

References for Supplemental Materials

Arceneaux, Kevin, and David W. Nickerson. 2009b. “Who is Mobilized to Vote? A Re-Analysis of Eleven Field Experiments.” American Journal of Political Science 53 (1): 1-16.

Arceneaux, Kevin, Christopher B. Mann and David W. Nickerson 2012. “Motivating Collective Action by Appealing to Potential Losses Rather Than Potential Gains: Evidence from Field Experiments”. Annual Meeting of the Midwest Political Science Association, Chicago, IL.

Green, Donald P., and Alan S. Gerber. 2008. Get Out the Vote! (second edition). Washington, D.C.: Brookings.

Hillygus, D. Sunshine. 2005. “Campaign Effects and the Dynamics of Turnout Intention in Election 2000.” Journal of Politics 67 (1): 50.

Mann, Christopher B. and Casey A. Klofstad, 2012. “Voter Mobilization through Friends and Family: Social Priming of Political Participation”. Annual Meeting of the Midwest Political Science Association, Chicago, IL.

Mann , Christopher B. and Betsy Sinclair, 2013. “Social Cues in Behavior and Opinion”.Annual Meeting of the SouthernPolitical Science Association, Chicago, IL.

Niven, David. 2004. “The Mobilization Solution? Face-to-Face Contact and Voter Turnout in a Municipal Election.” Journal of Politics 66 (3): 868.

Parry, Janine, Jay Barth, Martha Kropf, and E. Terrence Jones. 2008. “Mobilizing the Seldom Voter: Campaign Contact and Effects in High-Profile Elections.” Political Behavior 30 (1): 97-113.

1

[1] In practice, the strong match usually means a match of address and full name. Medium and weak match phone numbers include records that match only on address and last name, address only, etcetera. The standard practice of our partner organization, based on extensive experience with voter contact phone calls, was to use only strong match phone numbers.

[2] The exclusions were based on data obtained from local election officials by Catalist LLC.