Evaluating Active (Opt-In)and Passive (Opt-Out)Consent Biasinthe Transfer of Federal Contact Datato a Third-Party Survey Agency
Joseph W. Sakshaug (corresponding author)
Department of Social Statistics, University of Manchester
Humanities Bridgeford Street-G12
Manchester M13 9PL
Phone: +44 161 275 0271
Fax: +44 161 275 4722
Email:
Alexandra Schmucker
Research Data Center, German Institute for Employment Research
104 Regensburger Str.
Nuremberg 90461
Phone: +49 911 179 1762
Email:
Frauke Kreuter
Department of Sociology, University of Mannheim
Department of Statistical Methods, German Institute for Employment Research
Joint Program in Survey Methodology, University of Maryland
1218Q LeFrak Hall
College Park, MD 20742
Phone: +1 301 405 0935
Email:
Mick P. Couper
Joint Program in Survey Methodology, University of Maryland
Institute for Social Research, University of Michigan
PO Box 1248
Ann Arbor, MI 48106
Phone: +1 734 647 3577
Email:
Eleanor Singer
Institute for Social Research, University of Michigan
PO Box 1248
Ann Arbor, MI 48106
Phone: +1 734 647 4599
Email:
Tables: 8
Figures: 0
References: 42
Word count:6,737
ABSTRACT
Obtaining informed consent from individuals to participate in voluntary research studies iswidely considered to be an ethical research practice. However, there is considerable debate overhowconsent should be obtained from subjects. Many researchers argue that active (opt-in) consent is the only type of consent that accurately reflects the true wishes of the subject, and is closer to the informed consent ideal than passive (opt-out) consent procedures. Opponents of active consent procedures argue that such procedures harmstudy participation ratesand increase the risk of self-selection bias to a greater extent than passive consent procedures.Empiricalevaluations ofthese claims are rare given the lack of studies that experimentally assign subjects to different consent procedures and utilize acontrol group (in whichno consent is sought) to facilitate comparison. We report on an experiment that overcomes these issues in an application of consent to transfer contact data from a federal register to a third-party data collector for purposes of carrying out a telephone survey. Specifically, we evaluatethe impact of requiring consent on survey participation rates, self-selection bias, and the resulting survey estimates. We find that the passive consent procedure does a better job of minimizing self-selection bias and maximizing the validity of the survey estimates (relative to the control group) compared to the active consent procedure. However, neither procedure is ideal: both consent procedures increase the totalself-selection bias and reduce the sample size.We conclude with a general discussion of the main findings and their practical implications.
Key words:self-selectionbias, nonresponse, informed consent, data privacy, total survey error
- Introduction
Obtaining informed consent from individuals to participate in voluntary researchis a central tenet of ethical research practice. Legal frameworks and regulations oftenrequire that researchers obtain informed consent from human subjects prior to their participation in voluntary research studies. For example, theU.S. Privacy Act of 1974 and the U.S. Department of Health & Human Services put forth regulations mandating that research subjects be properly informed about the risks and benefits associated with their participation, and outlined several required elements of informed consent, including an explanation of the study procedures and their purpose, a description of reasonably foreseeable risks and discomforts, an explanation of who to contact for answers to pertinent questions, and a statement that participation is voluntary and can be withheld at any time, among other stipulations (U.S. Privacy Act of 1974; U.S. Department of Health & Human Services).While theintended purpose of the informed consent processis clear, there is a long-standing concern among researchers that obtaininginformed consent reduces study participation and undermines the validity and generalizability of the study results (Singer, 1978; Kearney, Hopkins, Mauss, and Weisheit, 1983). Hence, a central question for researchersis to what extent the informed consent processaffectsthe accuracy of inferences drawn from sample surveys,and what can be done to minimizethis source of self-selection error.
Consent to participate in voluntary surveys has declined steeply over many years, especially for telephone surveys (Curtin, Presser, and Singer, 2005), raising concerns about the quality of the resulting survey estimates. Sometimesfurther consent requests are administered to survey participants after they have already consented to take part in the survey. For example, consent may be needed to linkrespondents’ survey answers with administrativedata sources, collect physical or biological measurements, and collect process-oriented information (or paradata), among others (e.g., GPS, accelerometry measures). Studies investigating the impact of obtainingconsent to these requests often find that individuals who consent are systematically different from those who do not, suggesting that survey estimates can be affected by different sources of consent bias (Sakshaug, Couper, and Ofstedal, 2010; Couper and Singer, 2013; Sala, Knies, and Burton, 2014; Skender, Schrotz-King, Böhm, Abbenhardt, Gigic et al., 2015).
Another survey-relatedfunction, one in which informed consent may be needed, is the transfer of contact informationforstudy subjects from government organizationsto a third-party data collection agency. This type of data transfer is often performed because government organizationsthat sponsor data collection activities are usually the only ones withaccess to databases containing relevant contact information suitable for sample selection and recruitment. Privacy laws generally prohibit government agencies from disclosing information about individuals contained in these databases. However, there are some legalexceptions. For example, in the United States, federal agencies may disclose record information to the U.S. Census Bureau without consent from the individual towhom the record pertains “for purposes of planning or carrying out a census or survey or related activity pursuant to the provisions of Title 13” (U.S. Privacy Act of 1974, PL 93-579, December 31, 1974; 5 U.S.C.-552a).An analogous situation is the transfer of contact information from medical records without patient authorizationwhich is generally prohibited under the HIPAA Privacy Rule unless(contingent on IRB or Privacy Board approval) the information will be used to conductresearch that is “designed to develop or contribute to generalizable knowledge” (The HIPAA Privacy Act, 45 C.F.R. 164.501).
In Germany, similar legislation exists that allows, under limited circumstances, the transfer of name, address, and telephone information to a third-party agency without the consent of the individual for purposes of conducting interviews for scientific research (Section 75, Tenth Book of the German Social Code).This legislation is frequently drawn on at the Institute for Employment Research (German abbreviation: IAB) in Nuremberg, the independent research arm of the Federal Employment Agency (BA). The IAB commissions several household and establishment surveyson behalf of the BA to conduct labor market research and evaluate active labor market programs.These studies are used to supplement the vast amounts of administrative employment information collected by the BA. One source of this contact information is the BA administrative register, which covers over 80 million individuals in Germany. The primary purpose of the register is to collectnotification processes for the German social security systemand public service processes administered by the BA(see vom Berge, Burghardt, and Trenkle, 2013, for more details about the BA register). As of December 2012, the register contained information on about 89 percent[1] of the German civilian labor force between the ages of 15 and 64.Given its broad population coverage, the BA register is commonly used as a sampling frame for surveys commissioned by the IAB, including the Panel Study of “Labour Market and Social Security” (PASS) and the “Further Training as Part of Lifelong Learning” (WeLL), as well as for surveys that are commissioned by other research institutions. The IAB does not maintain its own data collection staff and thus contracts with a third-party vendor to carry out all data collection activities. Names, addresses, and telephone numbers (when available)corresponding to individuals sampled from the BA register are then forwarded to the contractor immediately prior to the start of the data collection period.
As previously mentioned, thesharing of BA contact data for survey purposesis exempt from data protection laws that normally require the consent of the individual whose information is being shared. However, recently this exemption came under increased scrutiny by representatives of the German Federal Data Protection Agency,which is charged with ensuring that strict data privacy standards are met across Germany’s federal statistical system. A key point raised was that an individual’s contact data contained within the BA register is confidential information and should not be revealed to any third-party without the consent of the individual. The counterargument expressed was that requiring consent wouldlikelylower participation rates and increase self-selection bias in IAB surveys, which already suffer from low response rates.
This discussion prompted the IAB to experiment with alternative procedures for obtaining consent from individuals regarding the transfer of their contact data, andassess the potential impact of this consent step on the sample size and resulting self-selectionbias in the subsequent survey. Two consent procedures weretested: active and passive consent. Under the active consent (or opt-in) condition, subjects were asked to sign and return a consent form documenting their authorization to the transfer of their contact data from the IAB to an unnamed third-party data collection agency. Under the passive consent (or opt-out) condition, authorization was granted if no action was taken, i.e., a signed refusal form was not returned. A control sample reflecting the status quo (no consent requirement)wasalso utilized for comparison purposes.
Further details of the experiment are presented in subsequent sections of thispaper, which is structured as follows. In Section 2 we review the advantages and disadvantages of active and passive consent procedures by drawing on the empirical literature. In Section 3 we describe the methodology used in designing and deploying the experiment. In Section 4 we summarize the main findings of the experiment. Specifically, we examine the extent of sample loss and consent bias under each consent procedure and determine what effect (if any) the consent procedures have on the resulting survey estimates. In Section 5 we expand on the study’s main findings anddiscuss their practical implications.
- Background
The choice between active and passive consent procedures has generated considerable debateamong social scientistswho conduct voluntary observational studies that pose low risks to subjects relative to clinical research. Advocates of active consent argue that the procedure, specifically, the deliberate action of documenting one’s consent with a signature, fulfillsthe legal requirements of obtaining informed consent and provides additional protection to researchers should any legal repercussionsarise due to the subject’s participation in the study.They criticize passive consent procedures on ethical grounds, arguing that subjects who are actually opposed to participation may be included simply because they did not read or return the opt-out form, or because they never received the recruitment materials (Ellickson, 1989; Ellickson and Hawes, 1989; Hollman and McNamara, 1999). Writing in the context of research with children, for which parental consent is sought, Range, Embry, and MacLeod (2001; p. 28) argue that, for these reasons, “active consent is closer to informedconsent than passive consent.”
On the other side of the debate, opponentsof active consent argue that the legal and ethical benefits of this procedure are outweighed by lowparticipation rates, increased risk of bias in the study variables, and costly implementation relative to passive consent procedures. It has also been argued that there are many other opportunities to withdraw from survey research (unlike clinical trials or experiments), as evidenced by breakoff rates in surveys and panel attrition. Furthermore, active consent opponents argue that a failure to return a signed consent formdoes not necessarily reflect a deliberate refusal to participate.Indeed, there is evidence that passive consent procedures yield higher participation rates, reduce the likelihood of self-selection bias, and decrease implementation costs relative to active consent procedures (Ellickson and Hawes, 1989; Anderman, Cheadle, Curry, Diehr, Shultz et al., 1995; Schuster, Bell, Berry, and Kanouse et al., 1998;Johnson, Bryant, Rockwell, Moore, Straub et al., 1999; Range, Embry, and MacLeod, 2001;Junghans, Feder, Hemingway, Timmis, and Jones, 2005). Even outside of survey research, passive consent procedures can have strong participation effects when consent is the default option. For example, the percentage of organ donors is considerably higher in countries that employ passive (as opposed to active) consent procedures for organ donor registration (Johnson and Goldstein, 2003).
A keyquestion on both sides of the debate is to what extent each procedure accurately captures the true wishes of the subject. Some empirical evidence sheds light on this issue. In a follow-up interview of parents who had passively consented to their child’s participation in a research study by not returning an opt-out form, Ellickson and Hawes (1989) found that the overwhelming majority (96 percent) of parents indeed intended to allow their child to participate. This is in contrast to the active consent group, in which about 87 percent of those who initially failed to mail back a signed form indicating consent eventually did so after further prompts. Thus, for the majority, failure to return the signed form did not appear to be a deliberate attempt to withhold consent. A similar result was found by Singer (1978, 2003), who noted that some respondents are willing to participate in research but are not willing to sign a consent form. The requirement for the documentation of informed consent may serve to increase the (perceived) risk of disclosure for subjects.
The aforementioned research indicates that active and passiveconsent procedures can differentially impact theconsent rate. However, what isalso important to investigators is the extent to whicheach procedureintroduces bias in the survey estimates. There are generally two strategies that researchers have adopted to assess consent bias. The first strategy is to randomize subjects to an active or passive consent conditionand evaluate whether systematic differences exist between consenters in each group. Usually the passive consent group is treated as the reference group against which the active consent group is compared. Thus, any systematic differences that result are attributed to increased bias in the active consent procedure. The second strategy for assessing consent bias, commonly used in observational studies employing a single consent procedure, is to simply compare consenting and non-consenting subjects on variables available for both groups (whether from the frame, or from survey questions asked prior to the consent request). Both of these bias evaluation strategies have been used in the context of consent to study participation (Kearney et al., 1983; Severson and Ary, 1983; Anderman et al., 1995; Dent, Sussman, and Stacy, 1997; Pokorney, Jason, Schoeny, Townsend, and Curie, 2001; Henry, Smith, and Hopkins, 2002; White, Hill, and Effendi, 2004; Junghans et al., 2005; Spence, White, Adamson, and Matthews, 2014), and record linkage consent (Jenkins, Cappellari, Lynn, Jäckle, and Sala, 2006; Sakshaug, Couper, Ofstedal, and Weir, 2012; Sala, Burton, and Knies, 2012; Knies and Burton, 2014).
However, a common (and understandable) limitation of consent bias studiesis the lack of a control group in which consent is not requested. This omission limits the analysis of bias in two important ways. First, in the context of assessing the biasing effects of multiple consent procedures, assuming that one procedure is bias-free relative to the other may be inappropriate, even when the “gold standard” reference procedureyields higher rates of consent. The survey participation literature hasshown that nonresponse rates and nonresponse biases are only weakly correlated, with high participation rates providing no guarantee of less bias (e.g., see Groves, 2006; Groves and Peytcheva, 2008) and, in some cases, increased bias results when efforts are undertaken to improve the response rate (e.g., Keeter, Miller, Kohut, Groves, and Presser, 2000). A second consequence of omitting a control group in consent studies is that it precludes any analysis of how the consent request may influence the answers that respondents give to the substantive questions asked in the study. That is, in order to fully capture the extent of biasattributed to each consent procedure it is necessary to consider not only differences between subjects who consent and those who do not, but also the effect that exposure to the consent procedurehas on their answersto the study questions.
There are virtually no studies that have examined the effects of nonconsent using a control group for comparison. One exception is the study by Das and Couper (2014). Using the Longitudinal Internet Studies for the Social Sciences in the Netherlands, a random selection of respondents were sentan advance letter detailing plans to link their survey information with administrative records maintained by Statistics Netherlands. A passive consent procedure was adopted which gave respondents the ability to “opt-out” if they objected to the linkage. The mode of delivery (e-mail versus postal mail) and the amount of text (short versus long) were experimentally manipulated in a 2x2 factorial design. Overall, the consent rate was about 95 percent and there was no strong evidence that the mode and content manipulations influenced the consent rate. A follow-up survey was then conducted with these respondents as well as a set of respondents who did not receive the advance letter (control group). The survey contained several attitude questions related to privacy, confidentiality, and trustinsurvey organizations, among other items. In general, Das and Couper found that concerns about privacy issues were largely unaffected by the consent statement; however, respondents who received the consent statement did report higher levels of trust in government agencies and agencies that study public opinion.
To sum up, studies examining the effect of bias due to active and passive consent procedures are sparse and generally limited to participation in school-based studies and record linkage among survey respondents. To our knowledge, no studies have simultaneously examined the biasing effects of these different consent procedures on individuals’ authorization to transfer their federal contactdata to a third-party survey agency prior to receiving an invitation to participate in a subsequent survey interview. We sought to address this knowledge gap by investigating 1) the extent to which active and passive consent procedures affect consent rates to the transfer of contact data; 2) the impact of the consent procedures on the bias due to nonconsent, nonresponse, and total self-selection into the survey; and 3) the extent to which each consent procedureaffects the resulting survey estimates relative to the control group.