A note on the calculation for Reaction Plus Statistics
Jane Tang
Tuesday, August 03, 2010
This note details the calculation used in the Reaction Plus test for the following statistics,
· Net Promoter Score (NPS)
· Impact Score (IS)
· Reaction Score (RS)
The Data
A Reaction Plus test for a video or audio stimulus material collects data from each respondent as the stimulus is being shown, and after he has seen/heard the stimulus.
As the stimulus is being shown, each respondent is asked for his reaction to the stimulus at any moment by clicking on any of the reaction buttons. When a reaction button is clicked, the time (offset in second) and type of reaction is recorded. A maximum of one click for each reaction is allowed during a 3-second interval.
After the stimulus, each respondent is asked the following questions:
[Overall Feeling]
Overall, how did you feel about …?
10 Very positive
9
8
7
6
5 Neutral
4
3
2
1
0 Very negative
[Likelihood to recommend]
How likely is it that you would recommend this … to a friend or colleague?"
Extremely likely = 10
9
8
7
6
5 Neutral
4
3
2
1
0 Not at all likely
Net Promoter Score (NPS)
Let
= Percentage of Promoters. Promoters are respondent who have ratings of 9 or 10 in their likelihood to recommend question,
and
= Percentage of Detractors. Detractors are respondent who have ratings of 0 to 6 in their likelihood to recommend question.
The NPS is calculated as:
Impact Score (IS)
For each stimulus, we divide the sample into those who report overall positive feeling and those who report overall negative feeling about it.
Let
= number of respondent reporting overall positive feeling, i.e. rated 7 to 10,
and
= number of respondent reporting overall negative, i.e. rated 0 to 3.
To calculate the Impact of the ith reaction, we compare how often, on average, the two groups of respondents (positive vs. negative) clicked on that reaction.
Let
= Number of times the ith reaction is clicked among those respondents reporting overall positive feeling,
and
= Number of times the ith reaction is clicked among those respondents reporting overall negative feeling,
The Impact Score of the ith reaction () is the difference in the average frequencies between the two groups, adjusted for the length of the stimulus.
Where
= the length of the stimulus measured in seconds.
Reaction Score (RS)
Let
= Number of times the ith reaction is clicked for the entire duration of the stimulus across the entire sample,
The Reaction Score (RS) is calculated as:
where
= Total number of respondents,
= the length of the stimulus measured in seconds
= the number of reactions used in the test