FINAL QUALITY REPORT
FOR THE PILOT CRIME AND VICTIMISATION SURVEY MODULE
FOR 2009
GRANT AGREEMENT NO. 11002.2008.003-2008.222
Prepared by: Martina Kontelj, Eva Belak
Date: September 2009
Table of Contents
1 Introduction 3
2 Sample design and organisation of interviewing 3
2.1 Sampling design 3
2.1.1 Observation Units 3
2.1.2 Stratification and sample size 3
2.1.3 Weighting 4
2.1.4 Sampling errors 5
2.2 Mixed mode design and organisation of interviewing 6
2.2.1 Costs and burden of the interviewing 6
2.3 Non-response analysis 6
2.4 Advance letters and brochure 7
2.4.1 Advance letters 7
2.4.2 Special letter before module Q 7
2.4.3 Brochure 7
2.5 Interviewers 8
3 Questionnaire Design Issues 8
3.1 Measurement Errors 8
3.1.1 Reasons for the Occurrence of Measurement Errors 9
3.1.2 Actions Taken in the Case of Errors 9
3.1.3 Measures to Reduce the Number of Measurement Errors 9
3.2 Time of Interviewing 10
3.2.1 CATI survey 10
3.2.2 CAPI survey 10
3.3 Data from the register 10
3.4 Questionnaire evaluation 10
3.4.1 Translation of the questionnaire and expert appraisal 10
3.4.2 Interviewer’s appraisal of the questionnaire 11
3.5 Comments to the individual questions 11
3.5.1 Most sensitive questions 11
3.5.2 Order of the questions 11
3.5.3 Wording of the questions 11
3.5.3.1 Module A 11
3.5.3.2 Module B 12
3.5.3.3 Module C 13
3.5.3.4 Module D 13
3.5.3.5 Module E 15
3.5.3.6 Module F 16
3.5.3.7 Module Q 16
3.5.4 General remarks 19
1 Introduction
The aim of the pilot victimisation survey was in the first phase to test the questionnaire and to test the different modes of data collection.
In the second phase the aim is to assess item and unit non-response and to asses precision of estimates for main statistics for Ljubljana and Maribor, two largest settlements in Slovenia.
2 Sample design and organisation of interviewing
2.1 Sampling design
2.1.1 Observation Units
Observation units were persons aged 15 years or over living in private households in Ljubljana and Maribor, the capital and the second largest settlement in Slovenia. The definition of the target population followed the criterion for the resident population, i.e. all persons whose usual place of residence was on the territory of Slovenia. The survey covered only the population living in private households. The source for the number of population is the Central Population Register.
Results from the Crime and Victimisation Survey Module estimates are based on the probability sample and are subject to sampling errors.
2.1.2 Stratification and sample size
A stratified simple random sample is employed for the survey. There were 2000 persons aged 15 years or more selected in the sample. Stratum allocation is proportional to type of settlement (Ljubljana and Maribor) and age groups (4 age groups).
Table 1: Basic information on the population in the sampling frame and the sample
Population / Sample / Response / Non responseNumber / % / Number / % / Number / % / Number / %
All / 312437 / 100 / 2000 / 100 / 1015 / 100 / 985 / 100
Man / 145523 / 46.6 / 965 / 48.2 / 456 / 44.9 / 509 / 51.7
Woman / 166914 / 53.4 / 1035 / 51.8 / 559 / 55.1 / 476 / 48.3
All / 312437 / 100 / 2000 / 100 / 1015 / 100 / 985 / 100
Ljubljana / 228874 / 73.3 / 1465 / 73.3 / 712 / 70.1 / 753 / 76.5
Maribor / 83563 / 26.7 / 535 / 26.7 / 303 / 29.9 / 232 / 23.5
Age / 312437 / 100 / 2000 / 100 / 1015 / 100 / 985 / 100
15-17 / 10650 / 3.4 / 68 / 3.4 / 30 / 3.0 / 38 / 3.9
18-44 / 136888 / 43.8 / 876 / 43.8 / 404 / 39.8 / 472 / 47.9
45-74 / 134640 / 43.1 / 862 / 43.1 / 466 / 45.9 / 396 / 40.2
75+ / 30259 / 9.7 / 194 / 9.7 / 115 / 11.3 / 79 / 8.02
Table 2: Population of Slovenia by age groups and gender in Ljubljana and Maribor, 2008
Ljubljana / Maribor2008H2 / 2008H2
Male / Female / Male / Female
15-19 years / 6.703 / 6.629 / 2.321 / 2.547
20-24 years / 10.692 / 11.112 / 3.971 / 3.895
25-29 years / 10.432 / 9.694 / 3.544 / 3.234
30-34 years / 10.587 / 9.890 / 3.586 / 3.174
35-39 years / 9.703 / 9.345 / 3.383 / 2.982
40-44 years / 9.991 / 9.978 / 3.535 / 3.418
45-49 years / 9.535 / 9.700 / 3.724 / 3.501
50-54 years / 9.596 / 10.213 / 3.826 / 3.708
55-59 years / 9.708 / 10.299 / 3.733 / 3.776
60-64 years / 6.689 / 7.922 / 2.657 / 3.012
65-69 years / 5.606 / 7.342 / 2.396 / 2.995
70-74 years / 5.000 / 6.715 / 1.853 / 2.545
75-79 years / 3.584 / 6.083 / 1.433 / 2.620
80-84 years / 2.188 / 4.701 / 795 / 1.998
85-89 years / 954 / 2.556 / 325 / 1.109
90-94 years / 191 / 661 / 63 / 239
95-99 years / 68 / 280 / 26 / 72
100 + years / 4 / 40 / 4 / 12
111.231 / 123.160 / 41.175 / 44.837 / 320.403
Source: SI-Stat Data Portal, Population by selected age groups and sex, comparative table, Ljubljana and Maribor, half-yearly
A discrepancy of 7966 persons between the population table in the SI-STAT and eCRP, which is used as a sampling frame, (320403-312437=7966) is due to the fact that the sampling frame does not include persons who have only temporary address on the territory of Slovenia.
2.1.3 Weighting
Firstly, we compensated non-response with weighting due to non-response in the two settlements, and then we post stratified data according to age-gender distribution in Ljubljana and Maribor.
At the end the data were calibrated according to size of the household, highest education and age-gender distribution. The data regarding highest education were obtained from the 2008 Labour Force Survey (LFS), and the data regarding the size of the households were obtained from EU-SILC 2008. Age and gender distribution is available from the population tables published in the SI-STAT portal.
The data were calibrated using CALMAR SAS Macro.
2.1.4 Sampling errors
Sampling errors were calculated in SUDAAN[1] with the Taylor linearisation method. We used ”PROC RATIO“ procedures.
Table 3: Estimates, standard errors and coefficient of variation for key statistics (in %), Ljubljana
Estimate % / SE % / Lower limit% / Upper limit %
Theft of cars / 0.1 / 0.149
Theft from cars / 4.9 / 0.844 / 3.3 / 6.6
Car vandalism / 11.1 / 1.238 / 8.7 / 13.5
Theft of motorcycle / 0.3 / 0.195 / 0 / 0.7
Theft of bicycles / 5.9 / 0.935 / 4 / 7.7
Burglary / 2.5 / 0.62 / 1.3 / 3.7
Attempted burglary / 1.4 / 0.471 / 0.5 / 2.3
Robbery / 1.6 / 0.48 / 0.6 / 2.5
Personal theft / 2.4 / 0.579 / 1.3 / 3.6
Sexual harassment / 1.7 / 0.511 / 0.7 / 2.7
Assaults/threats / 1.6 / 0.494 / 0.6 / 2.6
Table 4: Estimates, standard errors and coefficient of variation for key statistics (in %), Maribor
Estimate % / SE % / Lower limit% / Upper limit %
Theft of cars / -
Theft from cars / 1.1 / 0.638 / -0.2 / 2.3
Car vandalism / 11.1 / 1.238 / 8.7 / 13.5
Theft of motorcycle / 9.5 / 1.867 / 5.8 / 13.2
Theft of bicycles / 2 / 0.827 / 0.3 / 3.6
Burglary / 0.8 / 0.57 / -0.3 / 1.9
Attempted burglary / 0.4 / 0.369 / -0.4 / 1.1
Robbery / 0.3 / 0.265 / -0.3 / 0.8
Personal theft / 1 / 0.586 / -0.2 / 2.1
Sexual harassment / 0.8 / 0.577 / -0.3 / 1.9
Assaults/threats / 1 / 0.647 / -0.1 / 2.4
2.2 Mixed mode design and organisation of interviewing
For the pilot survey mixed mode design was employed. Firstly, we assigned a fixed or mobile telephone number to the sample we had selected. We succeeded to assign a telephone number to 54% of the selected persons aged 18 years or more. Persons below 18 years and the rest of the sample were interviewed face-to-face.
In the first phase telephone interviewing was conducted from 9 March until 21 March 2009. Interviewers did not work on Sundays. We tried to interview by telephone all persons aged 18 years or more that we were able to assign telephone numbers. After the telephone interviewing (CATI) was completed, we started the CAPI phase, which lasted from 31 March until 31 May 2009. Some of the CAPI interviewers reported that they worked on Saturday, but none of them tried to do the interview on Sunday, although they reported that the main reason for the non-response was that the selected person was not at home.
The follow-up procedure was implemented also for those persons who, for different reasons, did not cooperate in the telephone interview (e.g. were away form the household at the time of call, not contacted telephone numbers, soft refusals, ill at the time of call, etc.).
2.2.1 Costs and burden of the interviewing
Net costs of field interviewers amounted to approximately EUR 17 000, and costs of telephone interviewers to approximately EUR 3000.
The burden on households was moderate, since interviews took on average 19 minutes (CATI, for PAPI data is not available). The reporting units did not have any expenses, since they were visited by interviewers at their homes, and cooperation was voluntary.
2.3 Non-response analysis
Table 5: Unit non-response analysis
Initial sample size / 2000Ineligibility rate / 2.6%
Response rate / 52.3%
Non-response rate / 47.7%
Refusal rate / 18.6%
Non-contact rate / 11.5%
Interviewing rate / 50.8%
2.4 Advance letters and brochure
2.4.1 Advance letters
In the pilot survey we used different kinds of letters for different survey modes (CATI/CAPI). We also used special letters for the follow-up procedure, since we try to interview those who for different reasons did not cooperate in the telephone interview again in the face-to-face interview: soft refusals, non-contact and persons who were away at the time of telephone interview. To those respondents we sent special letters depending on the reason for non-cooperation.
From the pilot survey we learnt that advance letters should be sent to selected persons by interviewers – this way it can be ensured that time when respondents receive the letter and interviewer’s visit is not too long.
2.4.2 Special letter before module Q
We think this letter is not really necessary. We think it is enough that the interviewer explains to the respondent the following questions and that it is OK if he/she refuses to answer a certain question if it is too sensitive.
2.4.3 Brochure
We prepared a special brochure in which we further explain the purpose of the survey, how the person was selected, who needs such data, show some interesting results of the survey, etc. We think that the brochure is helpful both for respondents as well as for interviewers who had to advocate for the survey.
2.5 Interviewers
The CATI survey was conducted by 12 interviewers. They were all female students, 3 from technical faculties and the rest from social science faculties. Their mean age was 22 years. We have decided to hire only female students due to the sensitive topic of the survey.
The face-to-face interviewing was conducted by 13 female interviewers. Their mean age was 38 years (22 years for 4 student interviewers and 46 years for the interviewers under contract). For most of “non-student” interviewers this is their second job, except for one who is unemployed. Their performance varied very much and these differences should be further analysed.
3 Questionnaire Design Issues
3.1 Measurement Errors
The pilot survey was conducted via CATI and CAPI, which enables the regular control of data entered by the interviewer. The programme has logical data controls embedded so that certain data are checked on the basis of preliminary set rules. Incorrect data entry is prevented by means of socalled light and heavy logical controls. Upon entry of a value which is not very probable, a warning for the interviewer appears on the monitor that the value he/she has entered is not likely, although possible. In the case of such a mistake, the interviewer corrects the value entered, and if the value is correct, he/she needs only to confirm it (light control). When an illogical error occurs (e.g. if the interviewer tries to enter an inappropriate answer), a warning appears on the monitor that it is a serious error. The interviewer can not continue with data entry as long as he/she does not eliminate the error (heavy control).
Table 1: Example of logical controls in the Pilot Victimisation Survey Module