2
A Template for Designing Questionnaires
Attached to this document is a sample three-page questionnaire. While not perfect, the sample is included to illustrate various design aspects of a questionnaire. As well, a cross-section of question types is included so students can have examples of how these questions should be formatted. Do NOT feel you have to ask only these questions. This sample is presented mostly for formatting purposes. Your questionnaire should not exceed two pages so that it can be back-printed.
To fully understand what is being presented, you should print this document. Then, as you read along, you can put the sample questionnaire on the right and have the text on the left. You might also want to keep this document as a Microsoft Word file and then edit some of the sample questions for use in your group’s questionnaire.
Points for Observation:
Title: Your questionnaire should have a short title that hints at the purpose of the survey.
Opening Paragraph: There are a few sentences that act as an introduction. Be sure to tell the respondent who is collecting the data, why it is being collected, how the data might be used, and the security they can expect with their answers.
Question Order: This is tricky as there are a few competing priorities at play. First, respondents should be led from general questions to more specific questions. Second, one needs to think about the use of a “jump” question where follow-up questions are asked depending on the answer to a preliminary question. Third, it is critical that the question and its answer choices NOT be split across pages. Usually compromises are needed to balance these priorities.
“Jump” Question: The first question shown is an extreme version of a jump question. You will notice that there are different follow-up questions depending on either of the answers given. Notice how arrows and text boxes are used to help direct a respondent. This is preferable to written instructions. Questions inside the box are “numbered” differently than the jump question (you can use a), b), c) or i), ii), iii) – whichever you prefer). You will also notice that one of the follow-up questions in the first question is another “jump” question. As before, we use arrows and boxes to “nest” the second box inside the first. The second question only has a jump for one of the answers. The third question sees three of the answers leading to a jump but not the fourth.
Multiple Response Question: Question 1a) is called a multiple response question as a respondent can check more than one answer. These questions are easy to ask but provide a challenge when tabulating. Suppose 100 people were surveyed and 60 of them answered “Yes” to the first question. Now suppose that eight people chose “Family Fitness” in question 1a). What percent of people gave that answer? The calculation would be 8/60 or 13.3%. Because people can indicate more than one answer for that question, the sum of all the quoted percentages will exceed 100%. Do not try to “fix” this and re-weight percentages to attain a sum of 100%. Instead, simply tell the reader that people could indicate more than one answer and that is why the sum exceeds 100%.
Ranking Question: Question 1b) is a ranking question. People are given some number of options and are asked to rank these from highest to lowest. In the instructions to the question, please note the explanation for completing the ranking. Without this, some respondents would place a “1” beside the most important item while others would place the highest possible number (in this case a “6”) beside the most important item. Statistically there is a way to test these responses to see if there are unique rankings in the data. It goes beyond the bounds of most courses to teach you these techniques. Instead, when you report the rankings, report a mean score. If two items seem to have roughly the same mean ranking, chances are that these two items are tied in importance.
The “Importance” Scale: The fourth question asks people about the “importance” of something. A four-point scale has been used. The last point on this scale is shown as “not at all important.” Sometimes people use “of no importance” or “unimportant” as the bottom of the scale. Occasionally a group of students wants to have a further category called “very unimportant.” Psychologically this does not make sense. Once something becomes “unimportant” it ceases to have an influence on a consumer. It just can’t become “very unimportant.” This kind of scale can be used for other words asking people to rate how helpful or how interested or how concerned or how willing or how familiar or how satisfied they are with an item. An alternative form is shown in the fifth question. It allows a respondent to use the same scale for a list of factors. The sixth question is also an attempt to use the same scale for a number of factors. Here, the designer has used a seven point importance scale and has defined the highest and lowest points on the scale.
The “Goodness” Scale: The seventh question asks respondents to rate their fitness level. This scale is different from the previous scale as one sees balance or symmetry in the answers. A common error in this scale is to offer unbalanced categories. Students might give the following five words as possible answers – Excellent, Very Good, Good, Fair, Poor. Three of these words have positive connotations while only one word has a negative connotation. This leads to bias in a response. Looking again at the seventh question, you will notice no “neutral” answer has been provided. This was done consciously to force the respondent to lean one way or another in their answers. Adding a fifth “neutral” category is a design decision that your group must make. The same kind of “balanced” scale is shown in the eighth question asking people to show their level of agreement with some statements. Like questions five and six, this question allows a respondent to use the same scale for a list of items.
“Money” Scale: The third, ninth, and tenth questions involve money. Generally, amounts that start a range end in a zero and amounts that end a range end in nine. In the third question, some ranges begin with a five and some end with a four. If whole dollars are being used, there is no need for a decimal and token cents being shown. When smaller amounts are used (say when discussing bus fares), you might see a range of $1.50 to $1.99. Ranges are generally preferable to “fill-in-the-blank” responses. The tenth question allows a respondent to use the same scale for a list of items.
“Time” Scale: In the eleventh question, times are used. A range usually begins on the hour and ends one minute before another hour to avoid overlaps. A range could begin on the half hour or on the quarter hour. Notice that a.m. and p.m. are usually noted. Most people do not know that midnight is 12:00 a.m. and noon is 12:00 p.m. If either of those two specific times is used, it will be clearer for respondents if you simply say noon or midnight. Do not say “o’clock” or show “8 p.m.”
Demographic Questions: A common design mistake is beginning with demographic questions. Unless you are screening survey participants based on a demographic characteristic, these are the least important and, generally, the most personal questions being asked so they are placed at the end of the questionnaire. Because people can be extra-sensitive to these questions, they are in a separate section with its own title and explanatory paragraph. You will also notice that question numbering begins again at “1”. The most sensitive question (usually about income level) is usually asked last. Age and ethnicity questions would come after less sensitive questions about gender or education. Try not to “load” your questionnaire with too many demographic questions.
Gender: Note that we do not ask respondents to tell us their “sex.” It is less jarring to use the word gender.
Age: Note that age ranges usually begin with a number ending in zero and end with a number ending in nine. In some instances, you could start a range with a number ending in a five and end a range with a number ending in a four.
Income: The question shown might seem wordy but unless you are precise, you may not get a useable answer. A marketer is usually interested in household income more than individual income as household income is a more reliable predictor of the purchasing power for that family. Most people have a better sense of their pre-tax income levels even though it is after-tax income which is deposited into their bank accounts. If someone is working two or three jobs, we want them to add all income to give us a total amount. Finally, 2005 was used as it represents the last taxation year. People become most familiar with their income levels when they are completing their income taxes. This occurs between the end of February and the end of April each year. The last taxation year is used to help with their memory. Finally, you will notice that ranges begin with numbers ending in zeroes and end with numbers ending in nines. Students often use income ranges which are too low. They are biased by student incomes which tend to be much lower than household income levels.
Ethnicity: This is always a sensitive question and one which is often avoided. However, many businesses do define their target markets in terms of a consumer’s cultural or ethnic group.
Appreciation Paragraph: At the end of the questionnaire you can place one or two short sentences that thank respondents for participating.
Tabulating and Presenting Data
The most common method of summarizing findings is to complete a tabulation of the answers for each question. Suppose you asked a question on gender and found that 32 people were male and 46 were female. The absolute numbers aren’t very helpful. The reader will always compare it to the total. But why make readers do the math in their heads? After establishing the sample size early in your report, it is preferable to quote a percentage. The ratio 32 over 78 (total number of responses) gives 41.02564%. Clearly stating this exact percentage implies a degree of precision which this survey does not have. Even one decimal place implies a precision possible only if 1,000 people were surveyed. It is best to round percentages to the nearest whole number. In this case, 41% of respondents were male.
How is a tabulation completed? By now, you have been exposed to many statistical options. You can use a spreadsheet and the power of Microsoft Excel. You learned Minitab in second year and you can use it here. You can use SPSS which you would have learned if you enrolled in the Marketing Research course. Even hand-tabulating is possible!
When quoting tabulations, a common challenge is missing cases. Sometimes a respondent skips or refuses to answer a question. The data in the survey is still valuable so you would likely still want to use it. If the number of missing cases is small compared to the sample size (say one or two missing from a sample of 78), you would probably not draw this to the attention of the reader. If the number is larger, you would mention it. When calculating the percentages, you would change the denominator to reflect only people who gave proper responses. In the earlier example about gender, suppose 32 people indicated male, 44 people indicated female, and two people chose not to answer. The percentage of male respondents would be 32 / (78-2) = 42%. When you use a statistical software package, this is also known as the “valid percent.”
The tabulation of multiple response questions was covered earlier. When a large scale is used (as in the seven-point scale of Question 6), a common method of summarizing data is to use a mean or average score. Here you can use one decimal place in your answers.
Graphs/Charts: Most students will want to incorporate graphs or charts into the main body of the report. This is a visual way to present data and replaces large amounts of prose. Look at the second box for Question 1). There is a question which asks people why they have not purchased a gym membership. A tabulation chart might appear as follows:
Table 1 Reasons Why People Did Not Purchase a Gym Membership
14% / Not enough time20% / Intimidation by other members
30% / Health problems
14% / No motivation
20% / High cost of memberships
2% / No help to get started correctly
Students most often construct a graph or table using the order of answers given in the questionnaire. It can be quite helpful to the reader if the order was changed to show answers in increasing or decreasing order.
This tip only makes sense when the data is nominal. In other words the order of categories is arbitrary. When a natural order is suggested in a category (See Question 4), you should use that category in your graph.
Fitness Industry Survey
This survey was prepared by fourth year students in the Entrepreneurship course at the DeGroote School of Business, McMaster University. In that course, a business plan is being created to help launch a new business. To better understand consumer needs, a short survey is being conducted. Your participation would be appreciated. Please be assured that your answers will be kept confidential and anonymous.