2003 LRTAP[1] and NEC AIR EMISSIONS REPORTING ROUND:

PILOT INVENTORY IMPROVEMENT REVIEW

OVERVIEW REPORT

Introduction

This draft overview report summarises the methods used and the initial results of a trial review of the annual emission inventories, submitted by Parties to UNECE/LRTAP/EMEP (due 15 Feb. 2003), and the NEC Directive inventories submitted by EU Member States to the European Commission (due 31 Dec. 2002). The report represents one of the outputs from the ETC-ACC work, funded by EEA, and was performed in close collaboration with EMEP/MSC-W and EMEP/CIAM – the Trial Inventory Expert Review Team (TIERT).

Use of emissions data and legal requirements for reporting

The United Nations Economic Commission for Europe (UNECE) Convention on Long-Range Transboundary Air Pollution (LRTAP) and the European Commission, require emission data for both policy development and checking compliance with international agreements and legal obligations. In addition EMEP/MSCW and EMEP/CIAM use emission information data for modelling activities in support of the Convention as well as the integrated assessment work within the context of the Clean Air for Europe (CAFE) programme. The EEA (and its ETC/ACC) use data for assessments, primarily in the form of emission indicators, in support of European Union environmental programmes. The legal requirements for emission reporting are summarised in Box 1.

Needs for trial review

In accordance with the work-plan for the implementation of the Convention (ECE/EB.AIR/2002/4, item 2.1), and at the invitation of the Government of Sweden, a Workshop on Validation and Evaluation of Air Emission Inventories was held in Gothenburg from 14-16 October 2002. The delegates critically examined the means of assuring the quality of air emission inventory data that form the basis for assessing the compliance with existing agreements and possible future revisions of international protocols, e.g. the 1999 Gothenburg Protocol and EC directive for transboundary air pollutants. It was recommended, among other things, that:

‘There was a need to undertake a review process, including a system of checking and gap filling and the preparation of national inventory reports. In addition, the workshop recommended that the TFEIP expert panel on verification and projections develop procedures for the review of inventories, taking account of the proposal by MSC-West; to carry out a pilot assessment of bias and data gaps and inconsistencies in early 2003, and to develop further these procedures in 2004. The workshop recommended, moreover, further harmonisation with the reporting and review process of UNFCCC’.

The Gothenburg meeting identified the need to assist countries with the submitting and checking of data in order to improve the timeliness of reporting. It further identified the need to address the inventory quality related properties of: completeness, comparability, consistency, transparency, and accuracy. It stressed that any additional measures must be cost effective and urged that relevant work carried out under other international programmes be adapted wherever feasible.

Objectives of the trial inventory review

The objective was to perform a trial review process to evaluate the quality of the national air emissions inventories submitted to the UNECE Convention on Long-range Transboundary Air Pollution (LRTAP/EMEP) and to the European Commission (DG Environment, with copies to EEA), under the EU National Emission Ceiling (NEC) Directive.

In particular, the specific objectives of the work, performed by ETC/ACC in collaboration with EMEP, were to:

  • develop and apply a set of inventory quality tests suitable for a future annual inventory review process;
  • to produce a summary of results in an overview report;
  • produce a list of country-specific questions which can be circulated to Parties for comments/response (note: the country-specific questions are not discussed in this overview report).

It is intended that the work will provide the EMEP Steering Body, the LRTAP Executive Body and the European Commission (DG Environment) with a consistent and transparent technical assessment for their various purposes. The results of the assessment can be used to assist further national inventory improvements through the UNECE TFEIP (Task Force on Emission Inventories and Projections) and the EU NEC Directive (Committee). Particularly for 2003, the work also provides support to the CAFE process, by assisting IIASA in the task of checking and comparing the RAINS model baseline scenario output with the data submitted by individual countries to NEC and LRTAP.

Scope

The inventory review covered all Parties who reported emissions in compliance with LRTAP agreements (Gothenburg protocol) and the NEC Directive . Where appropriate, for summary purposes data were aggregated into the following geographical subdivisions:

–EU-15:the 15 EU Member States;

–Candidate and accession candidate countries: here the AC-13 countries;

–Other:all remaining LRTAP Parties that reported in 2003.

Methodology

The TIERT work explored a variety of processes and tests to address these requirements. The TIERT work has examined a number of diagnostic data processing tools with the aim to assist countries optimise inventory quality checking routines. These include tests on:

Timeliness: Date of submission (compared with due date)

Completeness: Are all sources and pollutants that are included in the air emission reporting guidelines, reported? For this first review only the number of gaps in reported data for Tables IV 1A and 1B in Annex IV Reporting Forms of the Reporting Guidelines have been quantified;

Consistency: (1) do sub-sector air emission values add up to the reported aggregated sector totals?(note: results of this analysis are not included in this overview report). (2) the detection of discontinuities (dips and jumps) in the time series of data reported by a country for each sector and pollutant[2], and (3) the detection of repeated values (i.e. multiple entries of the same data) within the inventory[3]

- Comparability based on an analysis of implied emission factors for different sectors, countries and country groupings[4], and a comparison of reporting of similar (corresponding) sectors in inventories required by different agreements (LRTAP, UNFCCC, and the NEC Directive)[5].

Overview of initial results

  1. Timeliness: EMEP have introduced an electronic system (RepDab) for submitting data to both the CLRTAP and EU in a harmonized and timely fashion[6]. The timeliness of the data is assessed on the basis of the arrival date of official reports to the UNECE/EMEP Secretariat.

Twenty-nine Parties (of a total 49 Parties) reported emissions by the due date for LRTAP (15 Feb. 2003). As of 1 June 2003, the number of Parties for which emissions were available had increased to 36. Of those reports submitted by 1 June, 22 were in the new NFR format, with the remaining countries reporting using old formats or a mixture of the two[7].

Of the EU15 Member States that reported emissions data for NEC, the EEA received submissions from six countries by 31 Dec. 2002, the due date for reporting of country emissions to the Commission[8]. A further seven countries made subsequent submissions to EEA during February and March, but had submitted emissions data earlier to the Commission. Of the remaining two countries, one submitted emissions data to EEA in March, and as of 4 July 2003, one country has not yet provided emissions data to EEA.

  1. Completeness: Figure ES1 illustrates for the EU15 the level of completeness of reporting, in terms of the number of values or data flags that are required to be reported in the ‘new NFR’ format[9]. Some countries reported using the reporting guideline (Annex IV) formats fully or partially filled, while a number of countries did not use the new formats at all. The reasons for this mixed reporting will require further investigation to determine and address the underlying causes – administrative and/or technical. The current analysis, while not an unambiguous indicator of completeness, provides a useful overview of reporting levels that could help identify sectors where the level of reporting is high or low and guide future action.

Figure ES1. Completeness of LRTAP data for 2003 EU15 reporting.

Note:‘Total number of values reported’ includes both numerical data values and data codes (e.g. NO, IE, NA etc.) as specified in the reporting guidelines.

NFR = Nomenclature For Reporting

DNFR = Draft Nomenclature For Reporting

SNAP = Selected Nomenclature for Air Pollutants

  1. Consistency: Two specific tests were undertaken that relate to the consistency of the emission time series values reported for each sector; (1) identification of repeated values[10], and (2) assessment of sudden dips and jumps[11]. Such tests help to provide an overview across countries, pollutants and sectors regarding consistency of data. Figure ES2 illustrates the relative number of values reported by each country group, and shows the proportion of values identified as containing potential inconsistencies relative to the time series in which they belong (i.e. repeated values or dips and jumps). For the EU, 13% of the reported numerical emission values were identified as being potentially inconsistent. The majority (87%) of these identified inconsistencies were due to countries reporting repeated values i.e. reporting the same emission value for a specific sector for 2 or more consecutive years. This may indicate, for example, estimates have been carried over from previous years, with no new estimate having been made. The percentage of inconsistent values identified for the Accession countries and the other Convention Parties was 7% and 13%, respectively. As for the EU, in both these country groupings the majority of potential inconsistencies were due to countries using repeated values in their reported emissions.

Figure ES2. Consistency of LRTAP data for 2003 reporting

  1. Comparability: Figure ES3 shows an example of the results of the inventory comparison for NMVOC data submitted by countries to LRTAP and UNFCCC[12]. For a number of countries, the number of comparisons able to be made (as indicated by the solid bars) was less than the idealised number of comparisons that would have been possible had there not been missing data in either of the submissions (indicated by the dotted lines). Only three Parties provided sufficient data to allow a full number of comparisons to be performed. Data that has been reported but not in the new format (NFR) was flagged as missing in this preliminary analysis. Of 31 Parties for which the required data was available, 20 had apparent differences between data submitted to the two inventories. Differences occurred across all sectors, and also between national totals reported by Parties.

Figure ES3 Example of comparison between emissions data for NMVOC reported to LRTAP and UNFCCC. The solid bars show the number of comparisons that were able to be made, while the dotted lines indicate the number of comparisons that would have been possible had there not been missing data in either of the submissions.

Due to the limited numbers of countries reporting in new NFR format for year 2000, not many implied emission factors (EF’s) could be calculated. Figure ES4 shows an example of the results from the implied emission factor (IEF) analysis for NOx. Implied emission factors were calculated by dividing NFR sector emissions by the activity data.

The EF values were then compared across different countries, and potential inconsistencies identified if (1) the values fell outside a range of the average IEF +/- the standard deviation; or (2) where countries reported NE (not estimated) in sectors whilst other countries reported an emission number.

An important point to emphasise is that activity data being used in this analysis (from the Locator database and New Cronos) may be significantly different from the activity data actually used in the calculation of the emission estimate for the different Parties. The use of different types of activity data, and data from different sources, could lead to significant differences between implied emission factors, as tested on the basis of deviation from the average implied emission factor.

Figure ES4 Numbers of implied emission factors assessed for NOx, and number of potential inconsistencies identified by country.

Conclusions and Recommendations for future work

The LRTAP and EU countries have called for better reporting of emissions - provided that this can be achieved without undue allocation of resources at national level. The revised Reporting Guidelines and the EMEP reporting system (REPDAB) have demonstrated that more comprehensive reporting, in a timely manner, is possible.

The TIERT work has evaluated a variety of diagnostic data processing tools which, when fully developed, will provide countries with the means of exploring and further improving emission data quality. The work described in this report took data submitted to EMEP and tested data processing procedures that have the potential to aid Countries to optimize the quality of the data they submit and demonstrate compliance with legal requirements. If such testing procedures (e.g. RepDab) are considered to be useful by the countries, they could be made available, where feasible, as a part of the EMEP electronic submission system .

There were three general types of reporting response across all countries reporting to NEC and CLRTAP:

  • a group that attempted to use the NFR/reporting guidelines (and by implication the guidebook as required in the NEC Directive;
  • a group which did not; and
  • some who are in the transition of adapting to the NFR system.

The practical consequences of this are that there is now a group having experience with the NFR system that could work within the TFEIP to clarify the detailed procedures of the Guidelines and the forms used. Conversely, there are a number of countries that do not use the NFR yet. Therefore, it is important to better understand the reasons for this situation, e.g. were administrative or technical difficulties (or a combination of both) responsible? Appropriate targeted training workshops could be one way to further support NFR reporting by Parties.

All country groupings demonstrated broadly similar proportions of inconsistent data, which may be due to a lack of familiarity with systems in this first year of NFR reporting. Similarly, a number of countries submitted apparently inconsistent data to LRTAP and UNFCCC, including national totals. Both of these situations highlight the need for quality management systems and the review of submitted inventory data to support countries to report consistent and reliable emission estimates.

It is recommended that:

  1. the TIERT proposed methodologies be subject to peer review by the TFEIP/EIONET, via an Expert Panel on Review, and used as the basis of further work developing an annual review process;
  1. those areas where there are shown to be systematic problems either as a result of the procedural aspects of reporting (i.e. relating to the NFR or the reporting mechanisms) or technical aspects (such as a lack of comparability or accuracy due to problems with emission factors) should be subject to on-going work;
  1. The TIERT proposes a scheme for evaluation by TFEIP/EIONET, for the consolidation of annual quality control (QC) reviews, where ETC/ACC, the EMEP Centres and Country experts cooperate to analyse the data quality of the reported emissions and report back to the TFEIP/EIONET annual meeting with proposals for further inventory improvement needs. The proposed QC cooperation process has 4 phases as shown in Figure ES5. The TFEIP/EIONET should therefore evaluate the structure and timeline of this proposed annual QC process.

Figure ES5. Proposed timeline for a LRTAP and NEC inventory quality control process

1

[1] There has been a policy decision in EMEP to use the acronym LRTAP instead of CLRTAP

[2] Data used: Pollutants: All. Timeseries data 1990-2001. Includes time series with blank cells or zeros. Only data submitted in 2003 in the new NFR format analysed for the purposes of this Overview Report.

[3] Data used: Pollutants: All. Timeseries data 1990-2001. Only data submitted in 2003 in the new NFR format analysed for the purposes of this Overview Report.

[4] Implied emission factors were calculated for NOx, CO, NMVOC, SOx, and NH3 from reported emissions data from a single year (2000) based on the detailed activity statistics provided by countries to the UNFCCC for greenhouse gas emissions reporting and from the Eurostat New Cronos database.

[5] Data used: Pollutants: CO, NMVOC, NOx and SOx emissions for 1990-2001. from 2003 LRTAP and UNFCCC submissions. NH3 not included as this pollutant is not reported to UNFCCC. Only data submitted in 2003 in the new NFR format analysed for the purposes of this Overview Report.

[6] Vestreng, V. (2003) EMEP/MSC-W Technical report. Review and Revision. Emission data reported to CLRTAP. MSC-W Status Report 2003. EMEP/MSC-W Note 1/2003. ISSN 0804-2446.

[7] Vestreng, V. (2003) EMEP/MSC-W in the report ‘Technical report. Review and Revision. Emission data reported to LRTAP. MSC-W Status Report 2003. EMEP/MSC-W Note 1/2003. ISSN 0804-2446’ outlines the results obtained and REPDAB reporting system.

[8] EEA (2003). Annual European Community CLRTAP emission inventory 1990-2001. Submission to the Executive Body of the UNECE Convention on Long-Range Transboundary Air Pollution. Final draft July 4 2003.

[9] The target level (34,770 values) was calculated based on 96 NFR sectors (including national total), the number of pollutants for which annual reporting is required (33) and the number of years 1990-2001 that are required to be reported for each pollutant (note reporting is required for PM2.5, PM10 and TSP from 2000 only).

[10]Country/pollutant/sector time series data was analysed to identify duplicate reported values between 1990-2001 (not necessarily consecutive). Excludes zero values and nulls.

[11] Data dips and jumps were identified on the basis of 1) if the standard deviation of the timeseries divided by the timeseries average was >50% the series was flagged; or 2) if after fitting a linear relationship through the timeseries, any individual residual (forecast value - reported value) was greater than 2.5 standard deviations of all residuals (i.e. the residual fell outside a 95% confidence interval for all the residuals in a given series) then the value was flagged.