Flash Eurobarometer No 194 – Urban Audit Perception Survey
Flash Eurobarometer Series
#194
Urban Audit
Perception Survey
Technical
Report
THE GALLUPORGANIZATION
HUNGARY
Technical and Evaluation report, page 1
Flash Eurobarometer No 194 – Urban Audit Perception Survey
Summary
This survey (Flash EB 194, “Urban Audit Perception Survey”) was conducted for the European Commission, Directorate-General for Regional Policy by The Gallup Organization, Hungary and its partner institutes.
Telephone interviews were conducted between the 08/11/2006 and the 20/11/2006 by these institutes:
BelgiumBEGallup-Europe(Interviews : 08/11/2006 - 18/11/2006)
Czech Republic CZFocus Agency (Interviews : 11/11/2006 - 17/11/2006)
DenmarkDKHermelin (Interviews : 09/11/2006 - 11/11/2006)
GermanyDEIFAK (Interviews : 08/11/2006 - 20/11/2006)
Estonia EESaar Poll (Interviews : 08/11/2006 - 18/11/2006)
GreeceELMetroanalysis(Interviews : 08/11/2006 - 18/11/2006)
SpainESGallup Spain (Interviews : 08/11/2006 - 18/11/2006)
FranceFREfficience3(Interviews : 08/11/2006 - 18/11/2006)
IrelandIEGallup UK(Interviews : 10/11/2006 - 16/11/2006)
ItalyITDemoskopea (Interviews : 08/11/2006 - 18/11/2006)
CyprusCY CYMAR(Interviews : 11/11/2006 - 13/11/2006)
Latvia LV Latvian Facts(Interviews : 13/11/2006 - 16/11/2006)
LithuaniaLT Baltic Survey(Interviews : 13/11/2006 - 16/11/2006)
LuxembourgCZGallup Europe(Interviews : 08/11/2006 - 10/11/2006)
Hungary HU Gallup Hungary (Interviews : 08/11/2006 - 13/11/2006)
Malta MT MISCO(Interviews : 08/11/2006 - 18/11/2006)
NetherlandsNLTelder (Interviews : 08/11/2006 - 18/11/2006)
AustriaATSpectra(Interviews : 11/11/2006 - 16/11/2006)
Poland PL Gallup Poland (Interviews : 08/11/2006 - 14/11/2006)
PortugalPTConsulmark (Interviews : 09/11/2006 - 13/11/2006)
Slovenia SICati d.o.o. (Interviews : 14/11/2006 - 17/11/2006)
Slovakia SK Focus Agency(Interviews : 08/11/2006 - 18/11/2006)
FinlandFIHermelin (Interviews : 10/11/2006 - 16/11/2006)
SwedenSEHermelin (Interviews : 10/11/2006 - 17/11/2006)
United KingdomUKGallup UK(Interviews : 08/11/2006 - 18/11/2006)
BulgariaBGVitosha(Interviews : 08/11/2006 - 18/11/2006)
CroatiaHRGallup Croatia(Interviews : 08/11/2006 - 18/11/2006)
RomaniaROGallup Romania(Interviews : 12/11/2006 - 16/11/2006)
TurkeyTRKonsensus(Interviews : 08/11/2006 - 18/11/2006)
Representativeness of the results
Each city sample is representative of the population aged 15 years and above.
Sizes of the sample
The target sample sizes amounted to a minimum of 500 respondents in each city.
The table below presents the number of conducted interviews in each city surveyed:
TOTAL INTERVIEWS
Country / City / Conducted Interviews / Country / City / Conducted InterviewsBelgium / Brussel / 523 / Hungary / Budapest / 503
Belgium / Liege / 517 / Hungary / Miskolc / 510
Belgium / Antwerpen / 501 / Malta / Valetta / 503
Czech Rep. / Praha / 503 / Netherlands / Amsterdam / 511
Czech Rep. / Ostrava / 501 / Netherlands / Groningen / 501
Denmark / Kobenhaven / 501 / Netherlands / Rotterdam / 503
Denmark / Aalborg / 508 / Austria / Graz / 500
Germany / Berlin / 500 / Austria / Wien / 500
Germany / Dortmund / 501 / Poland / Bialystok / 501
Germany / Essen / 509 / Poland / Gdansk / 510
Germany / Frankfurt a. d. Oder / 505 / Poland / Krakow / 511
Germany / Hamburg / 512 / Poland / Warszawa / 505
Germany / Leipzig / 502 / Portugal / Lisboa / 503
Germany / Munchen / 506 / Portugal / Braga / 502
Estinia / Tallinn / 505 / Slovenia / Ljubljana / 504
Greece / Athinia / 504 / Slovakia / Bratislava / 502
Greece / Irakleio / 501 / Slovakia / Kosice / 502
Spain / Barcelona / 500 / Finland / Helsinki / 504
Spain / Madrid / 501 / Finland / Oulu / 502
Spain / Malaga / 502 / Sweden / Stockholm / 504
Spain / Oviedo / 503 / Sweden / Malmö / 500
France / Paris / 509 / UK / London / 502
France / Rennes / 502 / UK / Glasgow / 503
France / Bordeaux / 501 / UK / Cardiff / 501
France / Marseille / 507 / UK / Manchester / 502
France / Lille / 500 / UK / Belfast / 503
France / Strasbourg / 500 / UK / Newcastle / 502
Ireland / Dublin / 501 / Bulgaria / Sofia / 500
Italy / Roma / 507 / Bulgaria / Burgas / 500
Italy / Napoli / 500 / Croatia / Zagreb / 506
Italy / Torino / 513 / Romania / Bucuresti / 511
Italy / Palermo / 504 / Romania / Piatra Neamt / 504
Italy / Bologna / 502 / Romania / Cluj-Napoca / 511
Italy / Verona / 505 / Turkey / Ankara / 502
Cyprus / Lefkosia / 506 / Turkey / Antalya / 501
Latvia / Riga / 502 / Turkey / Diyarbakir / 503
Lithuania / Vilnius / 502 / Turkey / Istanbul / 506
Luxembourg / Luxembourg / 511 / Total / 37 815
Questionnaires
1. The questionnaire prepared for this survey is reproduced at the end of this results volume, in English
2.The institutes listed above translated the questionnaire in their respective national language(s).
3. One copy of each national questionnaire is annexed to data CD.
Weighting
Gallup has weighted the sample to adjust for differences between the profile of that sample and the population profile. For more details, see below.
Qualitative evaluation of the fieldwork
Partner institutes did not report particular issues with this general population survey. The translation of the questions did not pose any exceptional challenge. The fieldwork of the survey went smoothly and was finished within the deadline. All deadlines in the project were kept.
The attached tables describe the field outcomes of the study. The overall response rate for the study is 28,7%, ranging from 20,4 in UK to 54,1 in Malta.
The achieved samples are generally good representations of the universe covered. In some cities, however, we found significant deviations regarding the age distribution of the respondents, with typically the oldest age group over- and the youngest age group underrepresented in the achieved samples. See further details below.
Flash Eurobarometer Series
#194
Urban Audit
Perception Survey
Sample
Evaluation
Report
THE GALLUPORGANIZATION
HUNGARY
Evaluation of the samples
The attached tables (after the Technical Report tables, under the heading “Sample Evaluation Report”) provide a detailed insight to the within country weighting of the study.
As mentioned earlier, the achieved samples are in several instances not sufficiently representative to the known age distribution in the given city. Below we discuss several potential reasons for such discrepancies. The bottom line is that non-coverage due to insufficient landline telephone access of the youngest cohort, doubled with their tendency to be out of their home caused coverage problems with a speed-oriented sampling design the Flash Eurobarometer is operating with.
The problem of non-coverage
When conducting a telephone survey, individuals that do not have a landline phone in the residence will evidently not be covered in the sample frame. The most important problem is the growing number of individuals and families who have mobile telephone(s)but no landline telephone in their residence (mobile only households). We know from pervious research that those who only own a mobile phone tend to be (much) younger than those who (also) have a landline phone.
We have sample-based estimations (Standard Eurobarometer 65) of non-coverage problems and the influence on the age distribution. To illustrate the problem, we picked two cities (where the achieved samples in the Urban Audit diverged from population statistics) and used the face to face Eurobarometer to derive the population covered by landline telephone interviewing.
Vienna
Age category / Total population (15+)Vienna
official statistics / Total population
(15+)
Austria
survey estimates
EB65 weighted / Population covered (15+), Austria
(those with landline telephone access)
survey estimates
EB65 weighted / Urban audit
(15+)
Vienna
survey estimates
not weighted
15-24 / 12% / 15% / 12% / 7%
25-55 / 54% / 53% / 49% / 45%
55+ / 33% / 32% / 39% / 49%
Brussels
Age category / Total population (15+)Brussels
official statistics / Total population
(15+)
Belgium
survey estimates
EB65 weighted / Population covered (15+), Belgium
(those with landline telephone access)
survey estimates
EB65 weighted / Urban audit
(15+)
Brussels
survey estimates
not weighted
15-24 / 15% / 15% / 13% / 6%
25-55 / 54% / 52% / 46% / 40%
55+ / 31% / 34% / 41% / 54%
We can also compare the age distribution of the fixed line sample and the mobile phone only samples for Helsinki and Oulu – in Finland we also use mobile numbers to reach respondents.
Helsinki
Age category / Total population (15+)Helsinki
official statistics / Urban audit
(15+)
Helsinki
Fixed line sample
survey estimates
not weighted / Urban audit
(15+)
Helsinki
Mobile phone only sample
survey estimates
not weighted
15-24 / 15% / 2% / 12%
25-55 / 56% / 32% / 54%
55+ / 29% / 66% / 34%
Oulu
Age category / Total population (15+)Oulu
official statistics / Urban audit
(15+)
Oulu
Fixed line sample
survey estimates
not weighted / Urban audit
(15+)
Oulu
Mobile phone only sample
survey estimates
not weighted
15-24 / 21% / 5% / 21%
25-55 / 54% / 40% / 53%
55+ / 24% / 56% / 26%
Evidently, the non-coverage due to unequal access to landline telephones is a significant reason for the skewed age composition of the achieved samples. But this, in itself, does not fully explain the representativity problem we face in several cities.
What else causes the observed differences?
Margins of error
Part of the problem is obviously the “natural” inaccuracy of the estimations. The results in a survey are valid only between the limits of a statistical margin caused by the sampling process.
Statistical margins of error due to the sampling process(at the 95 % level of confidence)
Various sample sizes are in rows, and various observed results are in columns.
5% / 10% / 15% / 20% / 25% / 30% / 35% / 40% / 45% / 50%
N=50 / 6.0 / 8.3 / 9.9 / 11.1 / 12.0 / 12.7 / 13.2 / 13.6 / 13.8 / 13.9
N=500 / 1.9 / 2.6 / 3.1 / 3.5 / 3.8 / 4.0 / 4.2 / 4.3 / 4.4 / 4.4
N=1000 / 1.4 / 1.9 / 2.2 / 2.5 / 2.7 / 2.8 / 3.0 / 3.0 / 3.1 / 3.1
As an example, we look at the age distribution of the population in Helsinki that can be reached in a telephone survey such as the Urban Audit. In the EB65 (n=1000) we find that 9.9% belongs to the youngest age category. The statistical margin is +/- 1.4 around the observed 9.9%. Theresult for the whole covered population of Helsinkilies between 8.5% and 11.3%. In the urban audit (n=500) we find that, in Helsinki, 6.2% belongs to the same youngest age category. The statistical margin is +/- 1.9 around the observed 6.2%. Theresult for the whole population of Helsinkilies between 4.3% and 8.1%. The nominal difference between the lower EB65 and upper Urban Audit estimations is only 0.4 percentage points. This compares to a much larger 9 percentage points difference between the Urban Audit sample and the total universe.
Non-response
Non-coverage and statistical margins cannot explain all the differences that we find between the Urban Audit results and the official statistics.
The selectivity of non-response (non-contact and non-cooperation) also influenced our results.
Contactability is a function of two factors: (1) when household members are at home, and (2) when and how many times the interviewer chooses to visit. Our contract calls for a limited fieldwork period of a couple of days. Indeed, a given city sample was achieved in a few -- typically three to four -- days. Given this very short period of fieldwork it is not possible to make more than 3 recalls(what is contractually specified as well) in case a household or respondent was not reached at the first contact. As a consequence those who are more likely at home are overrepresented, and those who frequently go out are underrepresented.The pattern of being at home of a household,or an eligible interviewee within that household, is a function of socio-demographic attributes (e.g., number of persons in household) and lifestyle (e.g., through working hours, social activities) both of them closely related to the age of the eligible interviewee.
As other pan-European surveys suggest, the lifestyles of younger people involve more out-of-home activities than that of others. In addition, younger people are also more likely to refuse to participate in surveys than older people (they are less at home, care less, and also more busy or too busy to participate).
Out-of-home patterns by age category, EU-ICS (2005)
Age categoryGoes out… / 15-24 / 25-55 / 55+ / Total
almost every day / 32% / 9% / 6% / 10%
at least once a week / 52% / 44% / 28% / 39%
at least once a month / 9% / 22% / 18% / 19%
less often / 5% / 18% / 24% / 19%
never / 1% / 6% / 23% / 12%
DK/NA / 0% / 1% / 2% / 1%
Weighting
As we saw non-response rates and non-coverage indeed vary by social segments, the sample characteristics reflect such differences as well (i.e., there are regularly less males and especially less young people in the achieved samples than in the universe.) During weighting, we compensate for the non-response and non-coverage bias that stems from the field execution process. The most advanced method for eliminating such deviations is the so-called Raking Adjustment for Nonresponse (raking). Gallup applied this method. This procedure performs iterative proportional fitting in contingency table analysis.
This method is also used to deal with the problem of large variability of weights — when weighting classes are formed based on full cross-classification of the auxiliary variables, the result is a large number of weighting classes with unstable response rates. However, one drawback is that raking assumes that the variables used for adjustment are independent.
Raking works in the following way:
1) sets initial weight factor values in each cross-classification term to 1;
2) adjusts the weight factors of the first cross-classification term so the weighted sample is representative for the variables involved;
3) adjusts the weight factors for the next cross-classification term so the weighted sample becomes representative with respect to the variables involved (this might disrupt the representativeness with respect to the variables involved);
4) repeats this adjustment for all cross-classification terms;
5) repeats all steps until the factors do not change.
A common approach to weighting is to determine the sample weights adjusting for unequal probabilities of selection, revise these weights to compensate for different sub-class response rates, and finally modify the weights again to conform the weighted sample distribution for certain variables (e.g., age, gender, occupation, etc.) to the known population distributions of the same variables.
The following variables were used in the weighting:
1)age and sex: 15-24, 25-55, 55 + year age groups in both genders, among those older than 15 years,
2)economic activity (active, inactive)
The weights are trimmed at approximately 3 and 0.33, constraining the maximum range limit in order to prevent that some cases be represented with too much or too little weight (for exact figures per city please refer to the tables at the end of this report.)
The population and occupation reference data are taken from (via New CRONOS), or from local statistical sources through our national institutes, if data for the given city/parameter was not available in the site.
Technical and Evaluation report, page 1