Draft
Is it Time to Consider a New Approach for Reviewing and Updating the NAAQS?
Brock Faulkner1[SL1]*, Julie E. Goodman2, Sonja N. Sax2, Sabine S. Lange3, Michael Honeycutt3, Annette Sharp4
*Corresponding author, ...
1Texas A&M University…
2Gradient, 20 University Rd., Cambridge, MA
3Texas Commission on Environmental Quality, Park 35 Circle, Austin, TX
4Louisiana Department of Environmental Quality[SL2]
Author Information
Julie E. Goodman, Ph.D, DABT, ACE, ATS
Gradient
20 University Rd
Cambridge, MA 02138
617-395-5525
Sonja N. Sax, Sc.D.
Gradient
20 University Rd
Cambridge, MA 02138
617-395-5530
Sabine S. Lange, Ph.D.
Texas Commission on Environmental Quality
12100 Park 35 Circle
Austin, TX 78602
512-239-3108
Michael Honeycutt, Ph.D.
Texas Commission on Environmental Quality
12100 Park 35 Circle
Austin, TX 78602
512-239-1793
1
Draft
Abstract
Significant advances in the scientific understanding of air pollution and toxicology have been made since the CAA was last amended in 1990. These advances, coupled with the difficulty and cost of attaining ever-tightening standard levels, raise the question: is it time to consider a new approach for reviewing and updating the NAAQS? There are several aspects of the NAAQS process that should be reconsidered, including the causal framework, the exposure and risk assessments, and the regulatory impact analyses. In particular, causal evaluations should be more thorough, transparent, and scientifically sound, and should carefully consider study quality, including flaws and biases. They would be greatly enhanced by the use of new evidence integration frameworks and causality tests. The NAAQS risk assessments would benefit from the inclusion of thresholds based on known pollutant modes of action, reconsideration of the assumptions used in the exposure assessments, and the consideration of multi-pollutant models in concentration-response functions. In addition, the uncertainties and limitations of risk assessments should be better communicated to inform policy decisions. Although costs cannot be considered in the final decision of the standard, regulatory impact analyses are conducted for most NAAQS pollutant reviews and shouldaccurately reflect the total costs and benefits of a rule. The economic impacts of proposed air quality standards should include greater clarity and comprehensiveness, a more appropriate consideration of uncertainties in expected costs and benefits, and an assessment of the impacts of the pollutant in question, rather than “co-benefits” from reduction of PM2.5. Overall, improvements to the NAAQS review process would yield standards that are based on the best available science and scientific analysis.
1Introduction
In the US Environmental Protection Agency's (EPA's) 2014-2018 Strategic Plan, one of Administrator McCarthy's objectives under Goal 1, "Addressing Climate Change and Improving Air Quality," is to "[a]chieve and maintain health- and welfare-based air pollution standards and reduce risk from toxic air pollutants and indoor air contaminants" (US EPA, 2014a no acn). This objective is consistent with Sections 108 and 109 of the federal Clean Air Act (CAA), which specify the process for introducing new and reviewing existing National Ambient Air Quality Standards (NAAQS). The NAAQS process includes the following steps:
- Planning,
- Conducting an Integrated Science Assessment (ISA),
- Conducting a Risk and Exposure Assessment (REA),
- Conducting a Policy Assessment,
- Having the Clean Air Scientific Advisory Committee (CASAC) review and comment on each of these assessments,
- Conducting a formal rulemaking, which includes issuing a proposed rule for public comment, addressing public comments, and issuing a final rule, and
- Implementing the final rule.
Section 109(a) of the CAA requires EPA to set NAAQS for six "criteria" air pollutants (i.e., ozone, particulate matter [PM], lead, carbon monoxide, nitrogen dioxide, and sulfur dioxide) and review those standards every five years. The statute instructs the agency to set standards such that "the attainment and maintenance of which...are requisite to protect the public health" with "an adequate margin of safety" (42 US Code §7409; US Congress, 2011 211-6848). Regulatory Impact Analyses (RIAs), conducted to examine benefits, costs, and other economic impacts of regulations, are a standard part of the regulatory process. Even though EPA is not permitted to consider costs in setting the NAAQS, the CAA mandates EPA to conduct these analyses to demonstrate that each intended regulation is necessary and the potential benefits of implementation justify its costs (US EPA, 1999 214-9488).
Significant advances in the scientific understanding of air pollution and toxicology have been made since the CAA was promulgated in 1970 and the last major amendments were adopted in 1990. These advances, coupled with the difficulty and cost of attaining ever-tightening standard levels, raise the question: Is it time to consider a new approach for reviewing and updating the NAAQS? The following discussion uses examples primarily drawn from the reviews of the PM and ozone NAAQS, but most of the issues raised applies to the review of other criteria pollutants as well.
2The NAAQS Framework for Causation Assessment
EPA uses a weight-of-evidence (WoE) frameworkfor causal determination(hereafter, the "NAAQS causal framework")that includes methods for literature searches; study selection, evaluation andintegration; and causal judgments. The NAAQS causal framework has many valuable features, but it could be more explicit in some cases and some features are missing. This has caused it to be applied inconsistently and its use has sometimes led to conclusions that are not supported by the overall WoE. Goodman et al. (2013 213-7578) identified additions to the NAAQS causal framework that would help align it with best practices for systematic reviews and evidence integration. This includes guidelines for evaluating all of the evidence in a consistent manner using well-specified criteria and determining whether such evidence constitutes support for causation or an alternative hypothesis. The additions that Goodman et al. (2013 213-7578) identified should be adopted by EPA so that causal evaluations are more thorough,transparent, and scientifically sound, and such that they do not yield a causal determination that may not be warranted. This is particularly important because associations that are considered to be "causal" or "likely causal" are included in EPA's risk assessments and RIAs.
2.1Epidemiology Studies
Epidemiology evidence figures prominently in EPA's evaluation of causality for most criteria pollutants. Despite significant advances in epidemiology methodsthat have been made in the past 25 years, these studies still have major limitations that often preclude conclusions regarding causality. In general, EPA has focused on epidemiology studies that report very small, but statistically significant, associations between increasingly low levels of air pollutants and health effects, but it has not given the same weight to studies of similar quality that report no associations (e.g., see US EPA, 2013a 211-1526d, US EPA 2015 213-8712). Importantly, although EPA discusses some limitations associated with these studies, it is unclear how EPA considers them when judging the evidence. These limitations include exposure measurement error, confounding by co-pollutants and other factors, and uncertainty with the statistical models.
2.1.1Exposure Measurement Error
Most epidemiology studies rely on data from central ambient monitoring sites to provide community average ambient pollutant exposure concentrations (e.g., Neaset al., 1999 207-5124; Naeheret al., 1999 207-5122; Mortimer etal., 2002 205-3967; Gent et al., 2003204-0728; Katsouyanniet al., 2009209-6475; Stiebet al., 2009209-5706), and the interpretation of statistical associations is predicated on the assumption that these ambient measurements reflect actual personal exposures. That is, in these studies, individuals are assumed to be exposed to the concentration of each pollutant measured outdoors at the ambient monitor nearest to their home, 24 hours a day, seven days a week. There are several reasons why this assumption does not hold. One is that ambient monitors may be miles away from where individuals work and live, and therefore do not necessarily reflect local concentrations. Another is that being indoors affects a person's exposure to air pollutants. In the case of PM, there are many indoor sources that account for much higher individual exposures (e.g., Long et al., 2000206-7638). Exposure measurement error results when there is poor correlation between the measured exposures used in an epidemiology study and actual individual exposures of the study population.
During the last ozone review process, CASAC highlighted exposure measurement error as a key uncertainty affecting the ozone epidemiology literature, concluding that central-site community monitors that measure ozone in the ambient air are generally poor measures of individual exposures (Henderson, 2006206-4381). Henderson (2006206-4381) further reported that personal ozone exposures are typically much lower than ambient ozone levels and, more importantly, often show little or no correlation with concentrations measured at the central ambient sites. For example, for a Baltimore-based cohort of 56 subjects, Sarnatet al. (2001 201-5841) reported no correlation between ambient and personal ozone measurements for either winter or summertime sampling periods (correlation slopes of 0.00 and 0.01, respectively). In a similarly designed study conducted in Boston, Sarnatet al. (2005206-7663) reported comparable results, finding no correlation between ambient and personal ozone concentrations in winter (correlation slope of 0.04) and only a moderate correlation between ambient and personal ozone concentrations in summer (slope of 0.27).
EPA and others have asserted that exposure measurement error is likely to underestimate risk from air pollutant exposures. For example, in the REA for PM, EPA cited the intra-urban analysis conducted by Krewskiet al. (2009209-5135) as support for this assertion. As discussed in detail in Rhomberg et al. (2011a211-7617), exposure measurement error generally leads to an underestimation of risks at high exposures and an overestimation of risks at low exposures; the latter is particularly relevant for the NAAQS. This should be considered when assessing whether epidemiology studies support causal associations.
2.1.2Confounding
Co-pollutants can significantly impact risk estimates, whichis why multi-pollutant models are so important. In the most recent analysis of the American Cancer Society (ACS) cohort, Krewskiet al. (2009209-5135) reported associations between several pollutants and mortality in single-pollutant models, but they did not present results from multi-pollutant models. In this study, mortality risks reported for several pollutants (e.g., sulfur dioxide and summertime ozone) were of similar magnitude and statistical significance as fine PM (i.e., PM2.5)(Krewskiet al., 2009209-5135). In their earlier re-analysis of the ACS study, Krewskiet al. (2000210-5982) found that adjustment for co-pollutants generally decreased PM2.5 risk estimates. For example, the relative risk (RR) for all-cause mortality from PM2.5 in the ACS cohort was reduced from a statistically significant risk of 1.18 (95% confidence interval [CI]: 1.03 to 1.35) to risks that were not statistically significant when adjusted for sulfur dioxide (RR = 1.03, 95% CI: 0.95 to 1.13) or all four gaseous co-pollutants (RR = 1.06, 95% CI: 0.95 to 1.18).
Temperature and other environmental factors can also confound the relationship between pollutants and health effects. For example, the longitudinal study of ozone and children with asthma by Gent et al. (2003204-0728) only considered same-day maximum temperature, but meteorological variables such as relative humidity may have confounded associations with respiratory symptoms. Air conditioning use and exposure to tobacco smoke are also important potential confounders of causal associations with respiratory effects, yet they were not accounted for in some key studies (e.g., Mortimer et al., 2002205-3967; Stiebet al., 2009209-5706).
Although EPA evaluated confounders to an extent, EPA should interpret epidemiology study results with a full consideration of how co-pollutants or other environmental confounders (e.g., temperature) impact statistical associations, as well as how results are used in developing air quality standards. Furthermore, a multi-pollutant approach is essential in a risk assessment based on epidemiology studies to identify the true risks of pollutant exposure.
2.1.3Statistical Models and Model Specification
Results from air pollution epidemiology studies have been shown to vary depending on the statistical method or model specifications. For example, EPA has interpreted findings of the two prominent cohort studies underlying concentration-response (C-R) functions for PM2.5 (i.e., the ACS cohort evaluation by Krewskiet al.[2009 209-5135] and the National Mortality and Morbidity Air Pollution Study [NMMAPS] cohort evaluated by Dominiciet al. [2007 209-1011]) as supportive of a causal relationship. However, Cox (2012a212-0740a)described an analysis he conducted of the NMMAPS dataset, which includescensus data, daily mortality rates, daily PM2.5 estimates, and meteorological measurementsin more than 100 US cities. He showed that regression models yielded both positive and null associations between PM2.5 exposure and mortality, depending on the treatment of daily temperature, which is a strong confounder in the PM2.5-mortality associations. In addition, he applied Granger causality tests and found that less than 4% of associations between daily PM2.5exposure and all-cause, cardiovascular, and respiratory mortality were significant. He concluded that the results do not suggest a causal relationship.
While the need for causal interpretation of statistical associations has been acknowledged by many in the pollution health effects research field, and tests for assessing potential causation have been developed, these methods have not been generally applied to PM2.5 or ozone and mortality data (Cox, 2012b212-0735). Cox et al. (2013213-6467) suggested three general methodological steps to test for causality:
- Generate, test, and, if possible, refute plausible alternative (non-causal) explanations for positive associations (also known as a hypothesis-based weight-of-evidence evaluation; see Rhomberg et al., 2010210-8094);
- Show that the association cannot be explained by using alternative statistical models or other information; and
- If possible, test whether changes in responses follow (and can be successfully predicted from) changes in individual exposures.
Some of the statistical tests that could be applied to assess causality include conditional independence tests (Freedman, 2004 no acn; Friedman and Goldzsmidt, 1998 no acn), Granger causality tests (Eichler and Didelez, 2010 no acn), change point analysis (Gilmour et al., 2006 no acn; Helfenstein, 1991 no acn), causal network models of change propagation (Dash and Druzdzel, 2008; Hack et al., 2010 no acn), and tests with negative controls for exposures or for effects (Lipsitchet al., 2010 no acn). Such causal tests, rather than simple correlation analyses, should be applied to analyses of epidemiological studies in support of future NAAQS reviews.
2.2Experimental Studies
Traditional toxicology studies evaluate clinically relevant toxicity or disease, often at high exposure levels (and thus often with questionable relevance to humans; see Goodman et al., 2010 210-7786). Evidence based on mechanistic whole-animal toxicology studies, as well as in vitro studies of tissues, cells, and molecules (e.g., cell-membrane constituents, proteins, DNA), can help identifya mode of action. These studies build on our understanding of chemical toxicology and molecular biology and allow the plausibility of various causal pathways to be explored. The relationship of simple cellular or enzymatic results relative to the whole living organism may be complex, given that they are by their very nature simplifications that may not reflect actual exposure conditions (e.g., inhalation, reaction with extracellular lining fluid, penetration of extracellular matrix proteins and cell membranes) or account for complex whole-organism processes that could mitigate or amplify responses seen in isolated cells (e.g., immune responses). Also, in some cases, a particular biological change may be part of a homeostatic process and so may not be indicative of adverse effects. Despite these limitations, these types of studies can be very informative for understanding risks of criteria pollutants (Goodman et al., 2014a 214-0505).
Mode-of-action studies have shown that antioxidants present within airway lining fluid can prevent ozone-mediated cellular and tissue oxidation (Avissaret al., 2000211-4159; Ballinger et al., 2005211-4157; Cross et al., 1994211-4158; Mudwayet al., 1996200-5612; Sametet al., 2001205-3973), and only ozone exposure of a sufficient duration and concentration can overwhelm antioxidant defenses, allowing oxidative damage to occur in airway epithelial cells (Schelegleet al., 2007210-2194). Similarly, as reviewed by Cox (2012b 212-0735), low levels of PM exposure increase antioxidant generation in the lung, but higher levels induce the generation of reactive oxygen species that overwhelm the lung's homeostatic mechanisms. Together, these studies show that a threshold exists below which antioxidant defenses are sufficient to protect against adverse effects of ozone and PM2.5. It is notable that a mode of action has not emergedthat predicts adverse health impacts at ozone and PM2.5 exposure levels typical of the present-day ambient environment.
It is encouraging that EPA has cited more of this type of experimental evidence in its recent policy reviews. However, moving forward, it is critical that experimental studies, such as those noted above, be given the same weight as those showing a correlation with adverse health effects. This is because these types of studies may support a threshold below which more stringent pollutant regulations will not lead to improvements in public health or welfare.
2.3Evidence Integration
The NAAQS causal framework indicates one should look separately at epidemiology, controlled exposure, and animal toxicology evidence, first coming to a synthesizedjudgment for each and then integrating these separate judgments into an overall qualitative statement about causality (US EPA, 2013a211-1526d, US EPA, 2015 213-8712). As discussed by Goodman et al. (2013 213-7578), data evaluation should be integrated across all lines of evidence before coming to judgments based on each realm independently. In this way, interpretation of each line of evidence informs the interpretation of the others.
As an example of how a flawed approach to evidence integration can impact causality determinations, EPA (2013a211-1526d) stated that recent animal toxicology studies of ozone exposures provide stronger evidence for cardiovascular effects than epidemiology studies of ozone exposures and concluded that the evidence was indicative of a "likely" causal relationship. The key animal studies on which EPA relied were conducted at very high ozone exposure levels and have little relevance to ambient human ozone exposures (e.g. Chuang et al., 2009; Perepuet al., 2010). Also, EPA considered an increase in heart rate variability as the key indicator of effect in the animal studies, but the epidemiology evidence regarding heart rate variability is inconsistent and does not corroborate the animal data. In this instance, a more effective approach would be to consider the lack of consistency and coherence of evidence across different realms in making a causal determination (e.g., Goodman et al., 2014b214-0323).
Another example can be found in the PM Final Rule, in which EPA stated that "the findings of new toxicological and controlled human exposure studies greatly expand and provide stronger support for a number of potential biological mechanisms or pathways for cardiovascular and respiratory effects associated with long- and short-term PM exposures"(US EPA, 2013d 212-9800). A review of the PM ISA, however, suggests that the experimental evidence is inconsistent and not coherent with findings in epidemiology studies (US EPA, 2009). For example, the findings of mild and reversible effects in most experimental studies conducted at elevated exposures are not coherent with the more serious associations described in epidemiology studies (e.g., hospital admissions and mortality). Also, both animal and controlled human exposure studies have identified no-effect levels for acute and chronic exposure to PM and PM constituents at concentrations considerably above ambient levels(e.g., Gong et al., 2003209-1090; Holgate et al., 2003 no acn; 203-5116Schlesinger and Cassee2003 204-0922). Therefore, a better approach would be to consider experimental findings such as these in light of the high exposure levels and what the relevance may be for ambient exposures.