***************************************
Warning note for tutors
Dear Tutor,
This document gives you an example of tutor feedback to an EP103-FA2 assignment. You may find useful to look at it, as well as to other examples posted on the EP staff website, to see a range of styles that have been adopted by different tutors when preparing their feedback. You can also refer to the document entitled ‘Marking and giving feedback on DL Epidemiology (EP) assignments’, available on the EP staff website, to find helpful guidelines and tips on feedback preparation.
Please note that FAs are changed frequently. As a result, the questions and solutions found in the example below may be different from those included in the FA you are currently marking. This example should thus be treated onlyas an illustration of the wayfeedback can be given. Please do not use it as an example for feedback content for this FA.
***************************************
Course: Practical Epidemiology – EP103 – FA 2
Student: XXXXXX
Marked by:XXXXXX
Grade: 2
Overall comments
Thank you for sending me your 2nd FA assignment for marking and comments ‘firstname of student’. You have made a fair effort with your interviewer-administered questionnaire, and included most of the important questions and measurements. Overall your question layout was straight-forward, and you gave instruction to the interviewer in some detail, which was good, but you lost marks as the instructions were such that it is unlikely that the questionnaire would be administered in a standard way (which can introduce bias), the instructions were dissociated from the questions, you did not use linking phrases, skip patterns, or pre-coding, and the questions posed would give insufficient detail to answer the research question. Your mark is a borderline C, because I was not convinced you have completely demonstrated that you have fully graspedall the concepts underpinning effective questionnaire design.
I have made some more detailed comments below about different aspects of your questionnaire design, and please also look at the ‘item assessed’ table below and the model answer to see what we were looking for particularly. In designing interviewer questionnaires it is very important to consider how ‘user-friendly’ the format will be for interviewers, who may each conduct many interviews. The layout and overall ‘look’ can make a big difference to the accuracy and consistency with which a questionnaire is applied. You would have maximised your marks by including space for comments throughout, giving clear instructions on how to code ‘don’t know’ answers for every question (very important for later differentiating what are unknown from truly missing data) and ensuring potential answers were mutually exclusive.
Please do not hesitate to contact me if you have any question.
Detailed comments
Structure: Your page 2 contained lots of useful and clear instructions, and you did very well to detail how measurements should be taken – well done! I am not sure what the first page of your FA is designed to achieve, however. Is the first page for the subject to complete for themselves, and then the 2nd and 3rdpages to be administered by an interviewer? If so, you have not stated this. It is very important to structure your questionnaire in an unambiguous way, and use clear simple language for instructions. Information given by the interviewer needs to be standardised. Try to be ‘user friendly’ – an introduction comes first, so place it first. Include instructions to the interviewer about how to approach women, or how and when to read the passage. What about something like “My name is X and I work for X, and we are performing a study in thisvillage.Our study is investigating how a woman’s height and weight might affect their pregnancies and the health of their babies. The questionnaire contains X questions which will take around X minutes to answer. There are questions about you, and then questions about your pregnancies and your children….etc” Think about consistency and ways to explain consent, confidentiality and protection of personal information.
Items covered, question format and ordering: You did well to compress your questions and measurements onto one page, but the questionnaire includes the bare minimum of information, and probably in insufficient detail needed to answer the research question. How would researchers know the outcome of the woman’s 2nd pregnancy and when this took place? You have not included any question about ethnicity, although this is considered a key aspect of the research study. Although on page 2 you have given clear instructions to the interviewer about measurement, you could have used this technique more by integrating instruction as the questionnaire was applied. Think about linking sentences, and about including instructions to the participant using the ‘statement of intent, then question’ approach, which participants often find reassuring particularly for challenging questions, as they know what will come next. Something like:
“ Interviewers please say ‘I am going to ask you which ethnic group you feel you most belong to’ then interviewers please ask “ are you mongoloid or non-mongoloid or another ethnic group?’.
Another way to enhance a questionnaire is to use bold, italic, highlighted and different font to separate instruction, questions, and possible answers, and to improve the appearance and layout of the questionnaire. Think also about how the questionnaire data will be transferred into a database. Could you have helped by including coding by the interviewer? How could you use matrices?
Type of questions: In page 2 you have done well to recognise and acknowledge that the information you are asking for could be emotionally sensitive, but you have not instructed the interviewer to acknowledge this to the subject, or given a suggested question wording. In fact there are no questions. Try to use mostly used closed questions and offer a ‘don’t know’ or ‘other’ alternative throughout, with clear space for comments and instructions for interviewers on how to clarify answers. With a questionnaire you want to try to maximise the data you get, minimise the inaccurate data and minimise the missing data. You also need to be sure it will be applied consistently – so all interviewers asking the same woman would code her answers in the same way. By carefully thinking out all possible answers to you questions, and using ‘don’t know/other’ options with space for comment you can gather information on participants that would otherwise be missing. What about a woman who felt she was neither mongoloid nor non-mongoloid, was currently pregnant and thought she had been pregnant 4 or maybe 5 times before, and all her babies had been born dead before 9 months? (an extreme example, I know! But how would she answer, and how would an interviewer know to code her answers and how could we see this is what had happened from the data we are left with?)
Question wording: Be aware that answers must be mutually exclusive and precise: look again at your categorisation of pregnancy outcomes. How would an interviewer code a baby that was born before nine months that was born alive? How can we trace the outcome for each event (pregnancy)? What about multiple births?
ITEM ASSESSED
/ Addressed in questionnaire?Yes / Some-what / No
Questionnaire Structure / - / - / -
Does the questionnaire contain? / - / - / -
An introduction (statement read by the interviewer) covering (e.g.): /
Details of research organisation/ who is doing the study /
Who the interviewer is /
Research topic /
Confidentiality /
Purpose of the study /
Survey length/ what is expected from respondent/ invitation to participate /
Instructions relating to specific questions or measurements to be taken (*) /
Linking phrases between topics /
Skip patterns / not applicable (n/a) options for relevant questions (*) /
Acknowledgement to the respondent /
Space for comments /
(Need for consent – or is this implicit) /
Time at which the administration of the questionnaire started/ended (for interviews) /
Items covered/ contents / - / - / -
Is the length of the questionnaire adequate? / - / - / -
Does it include all relevant information? /
Does not have more questions than needed? (not too long?) /
Type of questions / - / - / -
Were types of questions (open-ended, close-ended) chosen adequately? /
Question format / - / - / -
Attractive, easy to identify, code and enter /
Separate type faces for QUESTION, alternative answers and instructions /
Vertical answer formats /
Specific instructions and prompts for interviewers (*) /
Pre-code all closed questions /
Same code number for particular response category (e.g. 1=Yes, 2=No, throughout) /
Record of responses / circling / space or boxes /
Easiness of data entry / computer column codes /
ID on all forms and pages /
Numbering of questions /
Not too congested/ enough space /
Use of matrix format if appropriate /
Question order / - / - / -
Questions on the same topic grouped together /
Going from general to particular /
Logical sequence / backwards chronological order /
Threatening questions at the end /
Question wording / - / - / -
Simple and direct / no unknown abbreviations/ familiar to all/ slang avoided? /
Too vague/imprecise terms avoided? ('regularly', 'usually', 'normally') /
Unambiguous time reference? /
Not too precise? /
Not biased? /
Not threatening? (Place threatening questions late in questionnaire, use familiar words, broad categories, context to soften impact (break up into a series of questions)) /
Not more than one concept presented at once?/ double question? /
No double negative? /
Answers mutually exclusive? /
All possible answers offered, or space left for alternative answers? /
Doest not assume too much about respondent’s knowledge or behaviour? (n/k, n/a) (*) /
Not too demanding? /
Not cryptic? /
Applicable to all respondents? (skips, n/a, n/k used?) (*) /
(*) Items marked like this may appear more than once in different sections.
1/6