Division of Psychology
School of Health Sciences
Professor Michael J. Mouse
Editor, Journal of Adolescent Health
Department of Psychology
University of Disneyland
Anaheim CA 800921
28 May 2009
Dear Professor Mouse,
Thank you for the opportunity to review this manuscript titled Sex Differences in the Relationship Between Television Viewing and Overweight in Young Adolescents, which was submitted to the Journal of Adolescent Health. After carefully reviewing the manuscript, I’m afraid I cannot recommend it for publication. Further, given the serious faults inherent in the manuscript I am recommending that the article be rejected outright and that the author not be given the opportunity to re-submit an amended version of the manuscript.
To be blunt, this is one of worst papers that I have ever reviewed. The manuscript is riddled with numerous errors in design, method, analysis, interpretation and presentation, which I have itemised below. I can only hope that for the sake of psychological science, this article is not accepted by a journal of lesser standing. My review is as follows:
Abstract
Manova did not do his study justice in terms of convincing a prospective searcher that his report is worth reading. Rather than having included an incomplete abstract at the conclusion of the introduction, Manova would have been better served by having conformed to standard American Psychological Association (APA) style by having included a cogent abstract at the beginning of his report.
Introduction
Manova identified an important topical issue, adolescent obesity which, given the current level of public interest, is worthy of further investigation. However, Manova failed to adequately articulate why this study was warranted and what we already knew about the topic in terms of previous research.
Body Mass Index
Manova made a fundamental error when referring to BMI in determining obesity among adolescents. Although BMI is calculated the same way as for adults, it is thence compared to adolescents of the same age. In lieu of using set thresholds, percentiles are used, thus allowing a comparison to be made between adolescents of the same sex and age. Hence, a BMI that is less than the 5th percentile is considered underweight while a BMI greater than the 95th percentile is considered obese.
Hypotheses
Manova made a number of references to sex differences in relation to television viewing hours and obesity levels, including the report’s title, the latter part of his introduction and the results section. Manova then proceeded to exclude the secondary hypothesis relating to whether sex moderates the correlation between television viewing hours and obesity, citing paucity of related research as justification for exclusion.
Subjects
Manova referred to a total sample size of 50 young adolescents (M = 13.9 years, SD = 2.5 years) comprising 30 males (M = 13.3 years, SD = 1.2 years) and 20 females (M = 14.2 years, SD = 2.6 years). However, SPSS descriptive statistics indicated a total sample size of 40 young adolescents comprising 20 males and 20 females. In light of this discrepancy, I have assumed Manova’s reference to 50 was a typographical error. Also, given the fact that sample ages were not representative, Manova’s calculations pertaining to the mean and standard deviation of ages had to be taken on face value as being accurate. Finally, the age of the participants appears to have been restricted to early adolescence, thus posing a threat to internal validity by not covering the full gamut of adolescence in terms of age.
Threats to External Validity
A small sample of participants was recruited from a single geographic location; hence cause-effect relationships may not apply to different populations, settings and conditions.
Consent and Coercion
This study violated a number of sections pertaining to the APA Ethical Principles of Psychologists and Code of Conduct. Firstly, consent was not sought from parents of adolescents under the age of 18 years, or from participants 18 to 19 years of age, thus violating section 3.10 and section 8.02. Secondly, the study involved coercion in terms of compulsory participation in lieu of voluntary participation, thus violating section 3.08.
Privacy and Confidentiality
This study also violated section 4.01 of the APA Ethical Principles of Psychologists and Code of Conduct by asking participants to place their names on the surveys. In addition, section 3.04 and section 4.01 would also have been violated if an individual’s physical data was obtained in the presence of peers and teachers.
How-Much-Do-I-Exercise Questionnaire
Manova surprisingly introduced an additional variable, exercise level, which was neither referred to in the title of the report nor posed as an hypothesis.
Test-Retest Reliability
Manova incorrectly referred to a correlation coefficient, r = .5, as being high when in fact it is considered moderate. For a correlation coefficient to be considered strong to very strong, r >= .8. Furthermore, a more prudent method for interpreting the correlation coefficient would have been for Manova to have squared its value in order to determine the coefficient of determination, r2.
Results
Manova encoded six variables in lieu of the five variables stated in his report. The sixth variable coded was exercise-level (scale in lieu of ordinal) which is superfluous, as it bears no relevance to the topic or the aim of the report. In relation to both the title and aim of Manova’s report, the only variables which required coding were sex, TV viewing hours and BMI. BMI is a function of a person’s weight and height; hence the inclusion of weight and height was gratuitous and indicated a lack of understanding pertaining to BMI. For the reasons articulated above, I intend to ignore any further reference to weight, height and exercise level in relation to both descriptive statistics and Pearson correlations. Although superfluous to this report, I will however examine Manova’s use of a One-Way ANOVA in order to verify whether the results in relation to differences between exercise groups on TV viewing hours were correct.
Manova referred to the use of a variety of techniques (e.g., histograms, normality plots and Levene tests) which were used to check for such things as data entry errors, normal distribution and homogeneity of variance.
Manova was incorrect to claim there was no evidence of non-normality pertaining to any variable. The inspection of relevant histograms revealed the following:
· In the case of male TV viewing hours and BMI, both plots were skewed right underpinned by a positive skewness of 0.44 and 1.13 respectively.
· In the case of female TV viewing hours and BMI, the former plot was skewed left underpinned by a negative skewness of -0.63, while the latter plot was skewed right underpinned by a positive skewness of 0.92.
· In the case of the total sample of TV viewing hours and BMI, the former plot was skewed left underpinned by a negative skewness of -0.17, while the latter plot was skewed right underpinned by a positive skewness of 1.07.
A scatterplot, along with the computation of z-scores, revealed two outliers pertaining to two females who each had a BMI of 46.38 and 47.75 respectively. However, due to the strong, positive, significant correlation between TV viewing hours and BMI, the removal of these two outliers would have proved futile.
Descriptive Statistics – Table 1 and Table 3
Manova violated a fundamental rule relating to descriptive statistics by inferring the results clearly demonstrated strong and significant sex differences in all measured variables. Descriptive statistics can only be used to describe the main features of a collection of data in quantitative terms (e.g., mean and standard deviation). Descriptive statistics cannot be used to support statements about the population that the data are thought to represent. Manova failed to include descriptive statistics in the results section, using APA style. Manova was correct to point out that TV viewing hours for females (M = 31.15, SD = 15.80) were higher than those for males (M = 26.77, SD = 11.81). Manova was also correct to point out that BMI for females (M = 29.55, SD = 7.71) was higher than males (M = 27.96, SD = 5.60). However, it was noted that Manova entered a number of erroneous values pertaining to males and females. Given Manova’s hypothesis, the inclusion of total TV viewing hours (M = 28.96, SD = 13.95) and BMI (M = 28.75, SD = 6.70) was pertinent and in this instance, correct values were entered.
Pearson Correlation – Table 2 and Table 4
Manova failed to include correlation statistics in the results section, using APA style. However, Manova was correct to state that overall correlations, and the correlations for males and females respectively were strong, positive and significant as follows:
· Total sample - There was a strong, positive, significant correlation between TV viewing hours and BMI, r(N = 40) = .82, p <.001, r2 = .67 (2-tailed).
· Males - There was a strong, positive, significant correlation between TV viewing hours and BMI, r(N = 20) = .82, p <.001, r2 = .68 (2-tailed).
· Females - There was a strong, positive, significant correlation between TV viewing hours and BMI, r(N = 20) = .82, p <.001, r2 = .67 (2-tailed).
Although the correlation score between TV viewing hours and BMI for males (r = .822) was slightly higher than for females (r = .815), the difference was negligible, .007. So for the sake of pragmatism, there was no difference in the above-mentioned. In relation to Table 2, Manova made a number of omissions and erroneous entries. Manova should have used SPSS Bivariate Correlations and thence presented Table 2 using standard APA style.
t-tests – Table 5
Manova was incorrect to claim strong evidence of heterogeneity of variance for TV viewing hours, as a Levene test found that the assumption of homogeneity of variance was met, F(1, 38) = 2.35, p = .13; therefore a two-tailed independent-samples (as opposed to paired-samples) t-test based on equal variances was carried out. No significant sex difference in relation to TV viewing hours was found, t(38) = -0.99, p = .33, d = 0.31, 95%CI (-13.31, 4.56).
In reference to BMI, Manova was again incorrect to have performed a paired-samples t-test in lieu of an independent-samples t-test. A Levene test found that the assumption of homogeneity of variance was met, F(1, 38) = 0.46, p = .50; therefore a two-tailed independent-samples t-test based on equal variances was carried out. No significant sex difference in relation to BMI was found, t(38) = -0.74, p = .46, d = 0.24, 95%CI (-5.89, 2.73).
While an argument can be made for always reporting the t value for unequal variances and thus avoiding homogeneity of variance assumption, there was no need for Manova to have done so as Levene tests found no statistical significance, hence the equality of variance assumption was not violated in either case, as reported by Manova.
A One-Way Independent Groups (or Single-Factor Between-Subjects) ANOVA With Post-Hoc Tests – Table 6, Table 7 and Table 8
Apart from the fact that Manova failed to include relevant tables and results, I reject Manova’s claim that no significant differences were found between exercise groups on TV viewing hours.
A single-factor between-subjects ANOVA was used to analyse the relationship between three exercise levels and TV viewing hours; the ANOVA was significant F(2, 37) = 8.96, p = .001, η2 = .33. Subsequent post-hoc tests using Tukey’s HSD procedure (α = .05) revealed significant differences at p = .021 between low and moderate pairwise comparisons among the exercise level groups. Significant differences at p < .001 were also revealed between low and high pairwise comparisons among the exercise level groups.
Discussion
Manova’s report contained a copious number of errors, specifically in relation to statistical omission and accuracy, and the violation of APA Ethical Principles of Psychologists and Code of Conduct. On the one hand, Manova claimed his results accorded with previous related research and yet Manova excluded the secondary hypothesis relating to whether sex moderates the correlation between television viewing hours and obesity. There were numerous instances whereby this report failed to adhere to standard APA style; specifically in relation to tables, not providing a list of references, and not using the correct APA in-text reference style (Rudd et al., 1999).
Manova referred to a causal relationship between television viewing and obesity which demonstrated an inadequate knowledge pertaining to research designs, as this study used a correlation approach as opposed to an experimental approach. Correlation does not imply causation, which effectively means that correlation cannot be validly used in this case to infer a causal relationship between TV viewing hours and obesity. The causes underlying the correlation, if any, may be indirect and unknown. Consequently, establishing a correlation between TV viewing hours and obesity is not a sufficient condition to establish a causal relationship (in either direction). I unequivocally reject the above finding, and hence the manuscript in its entirety.
Please do not hesitate to contact me should you require more information on any of the points raised above.
Sincerely,
Division of Psychology
School of Health Sciences
RMIT University
Melbourne 3000
AUSTRALIA