EUROSTAT
Directorate F: Social Statistics and Information Society /
technical issues : sampling
document for item 7 of the agenda
TASK FORCE ON VICTIMISATION
luxembourg, 4-5 june 2009
Discussion paper on sampling for
the European Victimisation Module
John van Kesteren – INTERVICT, Tilburg University – / DISCUSSION PAPER
Tilburg - May, 2009
This discussion paper is prepared for the 4-5 June meeting in Luxembourg of the task force for the preparation of a Victimisation Survey Module, organised by EuroStat. Not meant for circulation beyond the task force. /
Discussion paper on sampling for the European Victimisation Module
John van Kesteren, INTERVICT, TilburgUniversity
Defining the population.
There are some issues concerning the population. First of all the age limits must be decided upon. As from what age? Till what age? Most national surveys opt for interviewing those 16 years of age or more but some have introduced slightly lower age levels. The next issue is whether only permament residents in the country are included (excluding tourists and other short term visitors). It should also be decided whether immigrants who do not master the language should be included. Interviewing respondents in non-national languages will increase costs since special interviewers will have to be selected.
Discussion point 1: The population will be defined as permanent residents between 16 year of age or older who can be interviewed in one of the national languages of the country.
Sample size:
A very important cost factor in survey research is sample size. The victimisation data are generally expressed in prevalence rates, the percentage of the population that has been a victim over a limited period of time. This results in dichotomous data that are fairly skewed, since most people are not a victim of a crime. This type of data is notorious for their wide levels of confidence. To narrow down confidence intervals, relatively large samples are needed.
Furthermore, sample size depends on the level of accuracy required to make comparisons across countries or years.
If there is interest in researching sub groups within the population, then the sample size will have to be further increased. Special problems arise in case the groups at issue form only a small part of the total population. National samples will normally allow comparisons of results in terms of gender or age. But if there is an interest in reporting on a minority group (such as the population of the capital city or immigrants), there is the choice of either drawing a very large sample, or a booster sample for the minority group. In the ICVS 2004/05 booster samples were drawn in the capital cities. A number of national surveys (BCS, Australia) have arranged additional sampling for the immigrant population.
Many questions in a victimisation survey questionnaire are conditional. A question on reporting a crime to the police can only be asked to respondents that have been a victim of a crime. Every time a follow up question is followed up in turn, the number of answers given becomes smaller again. If it is deemed of great importance to collect data on such subsets of respondents, only a very large sample will suffice.
On the other hand, there are some items in the questionnaire besides the victimisation questions that have response categories that resemble a likert scale (E.g. very, fairly, a bit, not at all). If the responses to these questions are not too skewed, a few hundred respondents answering them will suffice. Breaking down the sample in groups of 500 respondents and ask different questions to each group can reduce the costs of the survey.
For comparison, the ICVS has used samples of 2000 with intervals of 4 years. Experiences have taught that this is sufficient to distinguish ‘high’, ‘average’ and ‘low’ country scores for over all victimisation by ten types of crime (average of 16%) at the confidence level of 90%. Such sample size also allows the identification of trends in overall crime per country. However, results on victimisation by individual types of crime such as car thefts or burglaries show larger confidence intervals. The figure below depicting rates of burglary victimisation is a case in point.
Figure 4.1: Overall victimization for burglary; one year prevalence rates in 2003/04 (percentages) of the top 15 countries and results form the earlier surveys. 1989 - 20005 ICVS and 2005 EU/ICS*.
For
The rates for burglary victimisation depicted above allow a crude ranking of groups of countries but differences per country per year are too small to draw conclusions.
In the discussion paper on the questionnaire we have shown confidence intervals for rates calculated on subsets of respondents such as victims who have reported to the police. These examples from the ICVS dataset show confidence intervals that are even larger than those for burglary victimisation. They demonstrate the need to draw samples of 3.000 or 4.000 respondents as a minimum to be able to obtain statistically reliable results on differences between countries and on trends per country per year for a set of key items including victimisation rates of different types of crime and answers to follow up questions to (reporting) victims.
Discussion point 2: The envisaged European crime survey should use samples sizes of 3.000 or 4.000 as a minimum. If more accurate results are required or rates must be calculated for capital cities even larger sample sizes are needed.
Discussion point 3 : Questions on opinions using a likert scale of 4 points or more can be asked to subsets of the total sample as a costs saving device.
Types of sampling
- A number of countries opted for drawing a random sample from the population registry, contacting the selected respondent and offering them alternative ways of participating (either pencil and paper, face to face, by telephone or internet)
- Another option is to work with one method of contacting and interviewing respondents.Random dialling of telephone numbers is one, frequently used method. It has the advantage of more or less forcing the repondent to participate (foot- in- the- door).
- Land-lines are generally linked tot a household; so it is necessary to randomly select a household member after the first contact has been made.
- Cell-phones are generally linked to individuals, but a first contact by cell phone has to be followed up by the interview itself, which quite often cannot be done immediately because the respondent is otherwise engaged at the time. Another complication is that the owner of a cell-phone is quite often also a member of a household with a land-line and has a double chance of being included in the sample.
- Selection of a house from a municipal registry and then select a household member within the household. If such a registry does not exist, other methods of selecting a house are available. But in any case, it is a two stage sampling process to select the respondent.
All methods have there own advantages and limitations. In any case, there should be recent and accurate statistics on the population available to compare the resulting sample with the composition of the real population. It must be possible to determine whether certain groups are over or under represented and introduce the correct weights for re-weighting.
Stratification
An efficient method to ensure that al groups are adequately represented is to stratify. This means breaking up the population in groups and sample within each group. The groups can be determined by geographical regions, such as Nuts 2 regions but also by other characteristics. This requires expert knowledge of the population and sampling
Multi stage sampling
Random dialling of telephone numbers is an example of two stage sampling. First a household is selected and second a random household member. The victimisation surveys done in Japan were sampled in three stages. First a random number of municipalities were drawn (stratified by town size) and then households and next individual within households. This strategy is very efficient in case drawing a direct sample from a population registry is not an option.
Multi-sampling
It is also possible to use a mix of interview methods and sampling methods. As described above, this does introduce a problem of respondents that have a larger chance of being interviewed than others. This is illustrated by random selection of land-line based telephones and cell-phones. Some people are part of a household with a land-line and on top they have one or more personal cell phones. Obviously, these people have a larger chance of being selected than individuals with only a land-line or only one cell phone. People without either have no chance of being interviewed. It requires expert knowledge and experience of sampling to compute weighing variables to compensate for this.
Discussion point 4: Do we opt for one type of sampling, to be applied in every country, or do we allow optimal sampling strategies per country.
Proposal; we opt for CATI with selection of households/respondents with random dialling (including cell-phones). A response rate of 40% should be possible. Individual countries can opt for better methods.
Evaluation of the quality
It has become usual to give much attention to the response rate. In recent years response rates have dropped, especially in the case of telephone interviewing. However, the only kriterion that matters is whether the sample is representative. Even samples obtained with low response rates can be sometimes meet this kriterion. Biases in the samples can often be corrected through re-wieigthing procedures. Quantum sampling is a method to ensure all groups are equally represented. This means that during the sampling priority is given to respondents from the under-represented part of the population till they are sufficiently represented.
Multi sampling could be the answer to reach difficult groups. It is however not clear to what extent the samples overlap and how this can be dealt with cin a re-weigthin procedure.
Discussion point 5: The arrangements for the European crime survey should include a set of standards for the evaluation of the samples obtained.
Motivation
Interviewing people, either by phone, face to face or by pencil and paper is problematic. Telephone surveys about crime and safety (as the ICVS) used to obtain response rates of above 50%. But this has dropped to 30-40%. Piloting with invitiations to use the internet have so far mainly yielded disappointing results. There is a lot of ‘fatigue’ amongst the population and resistance to participate in survey research. A crime survey, however, should still be relatively popular since it serves no commercial purpose and raises isues that are of interest to many. Options to increase response include the sending of introductory letters explaining the public utility of the survey and/or the use of incentives (lottery ticket or a contribution to a good cause such as victim support Europe).
Discussion point 6: The survey should consider using introductory letters and/or incentives to boost response rates.