European Statistical System Network / Admin Data ESSnet, Work Package 1
Overview of MSs’ Existing Practices in the Uses of Administrative Data for Business Statistics
Deliverable 1.1
Main findings of the Information Collection on the Use of Admin Data for Business Statistics in EU and EFTA Countries
Reference period / SGA 2010
(29/4/2010 to 30/6/2011)
Date of delivery / 30/6/2011
Dissemination level / Public
WP1 Co-partners / Istat, Italy
LUIGI COSTANZO (WP leader), GRAZIA DI BELLA,
PATRIZIA LEONARDI/MASSIMO VIGNOLA
CBS, Netherlands
INE, Portugal
HUMBERTO J. PEREIRA, SOFIA RODRIGUES
ONS, United Kingdom
ELIZABETH HARGREAVES, ALISON PRITCHARD
1
In memory of Patrizia Leonardi
(1968-2011)
1
Contents
1.BACKGROUND INFORMATION...... 7
1.1Mandate of Work Package 1...... 7
1.2The information collection and its outcomes...... 7
1.3Countries, topics and domains covered...... 8
1.4The second phase of the information collection...... 9
2.MAIN FINDINGS...... 11
2.1Structure of the presentation...... 11
2.2General information about the use of admin data for statistical purposes 11
2.2.1Legal basis and cooperation with admin data holders....11
2.2.2Organisational solutions adopted for the management of admin data within the NSIs 17
2.2.3Projects currently in process to increase/improve the use of admin data for producing business statistics 19
2.3Existing practices in the use of admin data for producing business statistics 20
2.3.1Main findings by business statistics domain...... 21
2.3.1.1Business Register...... 21
2.3.1.2Short-term statistics...... 22
2.3.1.3Structural business statistics...... 24
2.3.1.4Prodcom statistics...... 28
2.3.2Types of admin sources used...... 28
2.3.3Admin data and estimation procedures...... 33
2.3.3.1Admin data used as a source of input for estimation procedures within the domains of STS, SBS and Prodcom 33
2.3.3.2Use of statistical methods to estimate variables which are not available from either admin or survey sources 36
3.CONCLUSIONS...... 41
3.1Legal basis and cooperation with admin data holders...... 41
3.2Existing practices...... 42
1
1BACKGROUND INFORMATION
1.1Mandate of Work Package 1
A primary objective of the Admin Data ESSnet’s Work Package 1 (WP1) is to produce an Overview of the use of administrative data in business statistics in EU and EFTA countries(further on, just “Overview”) – i.e. to collect, organise and make available a set of basic and homogeneous information on this topic, at a national level. Possible beneficiaries of this action are:
-Eurostat and the European Commission, to verify the extent to which the exploitation of administrative sources is going to replace the statistical surveys, and to identify critical areas and development potentialities in the use of administrative data for business statistics – even in order to guide possible actions to increase such use.
-The National Statistical Institutes (NSIs) of the European Statistical System (ESS), to promote the mutual knowledge of common problems and possible solutions in the use of administrative data for business statistics, as well as the dissemination of best practices.
-The Admin Data ESSnet itself, to provide all Work Packages (WPs) with general information on the use of administrative data in business statistics in European countries and a collection of relevant resources available from the literature and the Web.
1.2The information collection and its outcomes
To accomplish this task, an extensivecollection of relevant information and documentation was planned, and carried outin twophases, during the first two budgetary years of the Admin Data ESSnet (2009-2010).
The first phaseconsisted in a literature/web search. Once defined a reference framework, also based on indications received by the other WPs, the search was conducted by the members of WP1 from November 2009 to January 2010, using a standard template. First (partial) results and relevant background information were presented in detail in the Deliverable 1.5/2009,released on March 2010.
In the second phase, carried out from October to December 2010, the results of the literature/web search were submitted to the NSIs in the form of partly pre-filled modules to be edited/completed/updated and validated.Most of the gaps left after the first phase have been filled, so allowing to complete the picture envisaged by the initial framework.
Since the beginning, it was foreseen to make the outcomes of the information collection available to users in different forms, so to meet different kinds of demand.As a result, the Overview consists in a set of three products, which are expected to be updated in 2012-2013, before the closure of the Admin Data ESSnet:
- ThisReport (Deliverable 1.1/2010), which summarises the main findings of the information collectionand provides also background information on its second phase,
and two reference tools (both part of Deliverable 1.2/2010), implemented on the ESSnet Information Centre (IC) in order to give the users direct access to the information and resources collected:
- A Database that collects the complete results of the information collection, updated to the end of 2010;
- AReference librarythat holds, at present, over 350 documents found during the information collection (papers, presentations, reports, etc.) – including all those quoted as a source of information within the Database.
1.3Countries, topics and domains covered
The countries covered by the Overview are 31: the 27MemberStates of the European Union and the four members of the EFTA (Iceland, Liechtenstein, Norway and Switzerland).
Two main topics have been investigated:
- General information about the use of administrative data for statistical purposes,with regard to the following aspects:
- Legal basis of the use of administrative data for statistical purposes and possible legal barriers to such use;
- Cooperation with Public administration in the use of administrative data for producing business statistics;
- Organisational solutions adopted for the management of administrative data within the NSIs;
- Projects currently in process to increase/improve the use of administrative data for producing business statistics.
- Existing practices in the use of administrative data for producing business statistics, withregard to four domains, correspondingto the main relevant EU Regulations:
- Business Register (BR);
- Short-Term Statistics (STS) – further divided by main variables:
-Turnover,
-Production prices/costs,
-New orders,
-Building permits,
-Employment;
- Structural Business Statistics (SBS)– further divided by sector, according to the Annexes to the SBS Regulation:
-Annexes I-IV (Common module, Industry, Trade and Construction),
-Annex V (Insurance),
-Annex VI (Credit institutions),
-Annex VII (Pension funds),
-Annex VIII (Business services),
-Annex IX (Business demography);
- Statistics on the production of manufactured goods (Prodcom).
For each of the above domains, threebasic aspects were considered:
- The combination of sources currently used for the statistical production and – where applicable –the reasons why administrative data are not used (with a focus on the surveys carried out to update the BR);
- The main features of the administrative sources used and of their data:availability of micro data, kind of administrative units recorded, mode of acquisition from providers, periodicity of supplies, right to contact the units to check data quality, type of use of the source (direct – fordata input / indirect – forsampling frames, editing/validation, imputation of missing values, for estimation procedures);
- Admin data and estimation methods (administrative data used as input within estimation procedures, use of estimation methods to estimate variables which are not available from either administrative or survey sources).
1.4The second phase of the information collection
The information collected in the first phase were usedto producefive customised modules per country (a sort of partly pre-filled questionnaires): one for the “General information” and four for the “Existing practices”(one per Regulation). These modules were then sent to the members of the Eurostat’s Business Statistics Directors Group (BSDG) to be completed and validated.
Over 90% of the modules sent were returned – either subscribed by the BSDG members themselves or by other NSI staff responsible for the various domains covered.Therefore, almost all the information on which this Report is based has been verified by relevant NSI staff. Only in very few cases, missing responses have been imputed based on the information collected by WP1 in the first phase (provided that these were sourced by official documentation)[1]. A detail of the response rates is reported in Table 1.
Table 1 – Response rates achieved in the second phase of the information collection (2010)Modules / Sent / Returned / Response rate (%)
- General information
- Existing practices – Business Register
- Existing practices – STS
- Existing practices – SBS
- Existing practices – PRODCOM[2]
Total / 151 / 139 / 92.1
The permission to publish the information collected was asked by introducing a specific question on the modules. Based on the responses given, it was decided to restrict the access to the Database to ESSnet users only (staff of Eurostat and of the NSIs of EU and EFTA).
For further details about the methods and tools used to implement the second phase of the information collection, see the Annexes 1 and 2 to this Report(respectively, the Milestone 1.4/2010 – Strategy for finalising the information collectionand the Modules used for the second phase of the WP1 information collection).
1
2MAIN FINDINGS
2.1Structure of the presentation
The structure of this sectionmirrors the division by topic adopted for the information collection. In particular:
-Chapter 2.2provides a summary of the module on “general information”, which covers issues related to the overall situation of a country/NSI (legal basis, cooperation with Public administration, organisational solutions,current projects in the field of admin data). Basically, this chapter deals with the context (legal, administrative, organisational)in which business statistics are being produced, trying to identify what hampers or prevents a full exploitation of the potential of administrative data by the NSIs. More emphasis is given to legal basis and cooperation, which clearly represent the key issues at a national level.
-Chapter 2.3provides a synthesis of the information collected about the “existing practices”.This information is specific to the four domains investigated, and covers the combination of sources used, the main features of the administrative sources, and the use of admin data associated with estimation methods. Given the quantity of details collected, however, it was necessary to focus on a specific goal, and this was to identify those areas where the prospects for an expansion of the use of administrative data seem to be more promising in the foreseeable future.
Generally speaking, the findings of the information collection are presented here in a synthetic way, because the intention was to produce an executive summary rather than a systematic description of the whole set of data. Unlike the Report released after the first phase of the information collection (Deliverable 1.5/2009), this one does not include a complete tabulation of the data collected.Users can access such detail by querying the Database implemented on the Admin Data ESSnet Information Centre ( and go even more in deep by consulting – on the same website – the Reference library, where all documents quoted as a source of information are stored (
2.2General information about the use of admin data for statistical purposes
2.2.1Legal basis and cooperation with admin data holders
A first, good news is that – in principle – the use of administrative data for producing official statistics is recognised and fostered by all national statistical laws currently in force in the observed countries, since the few laws that still did not explicitly enable such use have been all purposely revised or replaced in the last decade.This means that, at the political (and cultural) level, an important cycle of innovation and harmonization of national statistical laws, which began in the early Nineties after the release of the first BR Regulation, can be considered essentially accomplished.
Despite of this, in several countries, a variety of factors hamper the full implementation of such a principle, so that – in practice – not all NSIs are in the position of exploiting all the potential of administrative data. Some of these factors are intrinsic to the legal basis (specific points of weakness, restrictions due to confidentiality laws), while others concern the cooperation between NSIs and Administrative data providers, although in most cases the two aspects are closely linked.
With regard to the legal basis, two major points of weakness can be identified in several national laws:
- The right of the NSI to access administrative data is established only in principle (so that further authorization or even special legal provisions are needed case by case) or without clearly establishing a corresponding duty for the administrative data holders (so that the data are supplied by virtue of provision agreements, whose terms must be negotiated from time to time). Examples of the first case are reported for Belgium, Germany and United Kingdom;while the second caseis fairly common over the observed countries.
- The law does not give power to the NSI to coordinatethe administrative data providers, e.g. in the design of administrative forms and information systems, for the adoption of shared definitions and classifications, etc. – which makes harder to obtain administrative data of suitable quality and/or to implement integrated systems of data collection/provision.This case is even more common, so that this appears to be the standard situation. In most countries, the law identifies the NSI as the coordinator of a wider national statistical system, in which normally participate all major administrative data holders. Despite of this, the actual participation of the NSIs in the design or the review of administrative information systems is very uncommon.
Furthermore, some countries report restrictions related to data confidentiality, but in most cases these appear to be rather linked to policy issues than to real legal barriers. In fact, current laws on data protection normally provide for the NSIs special rights to access administrative sources, under the bond of statistical secrecy and other limitations such as the solely statistical purpose and the ban on transferring data to third parties. Such exceptions, however, can be questioned or not applied where the NSI is not an autonomous administration, but the department of a Ministry (as reported in Belgium), or where – for various reasons – confidentiality rules tend to be interpreted in a restrictive way (as reported for Germany, Switzerland and United Kingdom).
As already mentioned, the features of the legal basis explain only a part of the variability observed in the actual access of the NSIs to administrative data– much of the rest being explained by something more elusive but equally relevant: the quality of cooperation between NSIs and administrative data holders. This is always indicated as a key factor in all most successful experiences made in this field, starting from the pioneering ones (and still the most advanced) of France and the Nordic countries.
Also data quality issues, which are often indicated as the reason why available administrative data are not being used for statistical purposes, appear, at a closer look, to be strictly related to cooperation issues. In most cases, in fact, the problems reported in this regard are those typical of a lack of coordination within the Public administration (such as incompatibilities of technical standards, mismatch of definitions/classifications, non-shared criteria for the maintenance of data sources, etc.).
The same applies to an even more relevant diseconomy, which is a serious obstacle to the statistical use of admin data, regardless of other circumstances: the absence of a unique identification number, adoptedby all Public administrations for any recording of business activities.This problem is reported only for Germany and Malta, but is likely to exist also in other countries (there was no specific question on this topic). In Ireland, for instance, a project is currently under way for adopting a common business identifier between the NSI and the Tax authorities, whereas in Switzerland a unique business identifier was introduced only in 2010.
Although based on different approaches, France (on one side) and Denmark, Finland, Norway and Sweden clearly represent the European forefront in the statistical use of administrative data. In these countries, efficient integrated systems for producing business statistics have been implemented since long time, within which surveys are normally used – where necessary – only to supplement administrative data (and not vice-versa), so reducing to the minimum the statistical burden on enterprises.
-In the “French model”, highly centralised, the NSI is to directly manage a “hybrid” business register (SIRENE), serving for both administrative and statistical purposes, and also a wide range of sources used to update it – which places under its control much of the generation process of administrative data itself[3].
-In the “Nordic model”, highly participative, the emphasis is more on the interaction between the NSIs and the administrative data holders, which are involved in a steady, regular consultation process, through which common goals and strategies are being continuously defined and implemented.
Both models rely, however, on very strong legal frameworks, which enable the NSIs to effectively act as coordinators of the whole Public administration as regards the production of official statistics, and both have been consolidated over decades of practice.
Compared with 1993, when the BR Regulation was issued by the European Community, the divide that separates France and Nordic countries from the rest of Europe has been greatly reduced. At that time, very few countries were using (and mostly to a marginal extent) administrative data for producing business statistics, whereas today this practice has become quite common. Inall observed countries (except Iceland) a statistical BR has been implemented, is being used as the infrastructure for the entire production of business statistics and is normally updated (in whole or in part) by using administrative data.
Nevertheless, the overall picture remains quite uneven: in many countries there are still important gaps to be filled and, generally speaking, the models mentioned above appear to be hardly replicable in different legal and cultural contexts. For this reason, an enhancement of the statistical cooperation among public bodies appears to be – in the current situation – the main road (if not the only one) to increase and improve the use of administrative data. In such direction, several initiatives were recently undertaken in various countries, which indicate pragmatic routes and goals for the foreseeable future. To name a few:
-In Ireland, the NSI has developed, in cooperation with the Tax authorities, a vast programme of feasibility studies, aimed at realising the Statistical Potential of Administrative Records (SPAR). Based on the results of SPAR, a Memorandum of understanding has been signed in 2009 between the NSI and the Revenue Commissioners Office,in order to eliminate duplications in data submission by the enterprises.