A critical reading of a research paper
REVISED LAB with emphasis on the process of getting research published.
Group work, then Create List on Board of Sources of Information about a topic.
Evaluate credibility of information from each of those sources
Examples: Google, Google Scholar, PubMed, Science Direct, Physician, Trainer, Textbook, product label, vendor website, FDA website, Cosmo, Men’s Health, etc.
Rate Credibility 1-10 of JAMA, Science/Nature, Men’s Health, Cosmo and discuss reasons.
Group work, then Create List on Board of Sources of Funding for Scientific Research
Examples: Gov’t (NIH, NSF, etc), Private Foundations, Individuals, Academic Institutional in-house funds, Non-profit organizations, companies, investors, etc.
Group work, then Create List on Board of Components of a Research Grant Proposal.
Have student pretend they are on a committee responsible for allocating research money to fund research. What information would they expect in a Research Grant Proposal that they would need to determine whether or not to fund that project, given limited available funds.
List should include budget (personel, supplies, lab equipment, overhead (be sure to explain)), timeline, purpose/hypothesis, relevance, what is already known, what is not known (The Question), why it is important to know (justification), risks, possible reward/benefit, plausibility, credentials, publication history of the proposers, academic affiliation and institutional reputation, etc.
Evaluate the credentials and affiliations of authors this Andro Paper.
Rate Iowa State U, Clemson, Johns Hopkins, Harvard, Wofford, PC.
Discuss first authorship, senior author, and corresponding author.
Board Diagram of Members of a Research Team
PI, post-doctoral fellows, Ph.D. candidates, Masters students, undergraduates.
Group work, then Create List on Board of Parts of a Research Paper.
Title (two types: what they did vs. what they found. Which is used for this paper. Create a new title for this paper in the other format.) Why would an author choose one type or the other. Remind them they’ll have to create a title for their abstract later in the course and should choose wisely.
Abstract, Intro, Methods, Results, Discussion, Acknowledgments, Funding, Reference. What should be found reliably in each section of a research paper.
Getting a manuscript published. Compare and contrast to choosing a college: seek most reputable, prestigious college/scientific journal. Unlike college, may submit a manuscript to only one journal at a time. Discuss how choice is made: reputation, selectivity, prestige, likelihood of acceptance for publication.
The Anonymous Peer Review Process diagram on board
Authors select a journal, submit to Journal Editor who selects two peer reviewers who are peers in the sense they are qualified experts on this topic, Reviewers know names of authors, but authors don’t know who the reviewers are. Advantages of this system. Role/responsibility of the reviewers. Why they agree to review without pay (professional courtesy, stay current, the honor of being selected by a journal editor as a reviewer, add to resume as professional credential). Three options of reviewers: accept, accept with revision, reject. Why a paper may be rejected by a journal. What happens in case of disagreement between reviewers (JE may decide or send out to a third reviewer.)
At end of lab, students will be asked to report as if they were a peer reviewer and recommend accept, accept with revisions (state the revisions you’d require) or reject (and state why.)
Begin Critical Reading of the Andro Paper
Provide historical background: Mark Macguire and baseball’s homerun record… he supplemented with Andro because testosterone is illegal with all sports organizations.
Read section by section, with discussion following each section the Intro and methods up to “Resistance training.” Summarize the methods sections after that.
Rate the INTRO 1-10 on your notecard, based on what is expected in an Introduction.
Check the on 50 year old reference to Andro experiment in two! Women, and.
Diagram the hormones involved in the negative feedback pathway, and make predictions about the levels of gonadotropins if Andro does get converted to testosterone.
Discuss the number of subjects, how they were selected, whether they were paid to participate, the two sets of experiments.
Carefully examine and interpret Figures 1-5, noting missing error bars on some, and groups starting values different In the 8 week study! Does this discredit the work?
Look at Table 2 Muscle strength and draw own conclusions.
See who funded the paper, and consider the irony!
Why was it published if it essentially a negative result?
Record on your notecard your recommendation as a peer reviewer.
Note the disagreement and need for two anonymous peer reviewers.
The History of Andro
What triggered the public’s interest in androstenedione in 1990?
What claims are made by the manufacturers and vendors of Andro?
Do their claims make sense based on the chemical synthetic pathways in human cells?
Sources of Information: Types and Credibility
How does one evaluate the claims made by vendors of nutriceuticals?
Where might one seek additional information about Andro?
What factors should be considered when evaluating the credibility of a source?
Doing Research: The Plan, The Money, The People, and The Paper
By what process might a scientist obtain money to conduct an investigation into the effects of Andro?
What information would a funding agency need to determine whether they should allocate money for a study?
In an academic research lab setting, what are the categories of people who conduct the research and what are the roles and educational backgrounds of those persons?
What is first authorship on a research paper considered most prestigious?
What does a person typically have to do to earn first authorship?
How can you determine whether a paper has a senior author?
What is prestigious about being a senior author?
Getting Published
Once a manuscript is ready for publication, what are the steps in getting the manuscript accepted?
What factors are considered when determining which journal to submit for publication?
What are the roles of a journal editor?
What are the responsibilities of peer reviewers?
What criteria does a journal editor use to secure peer reviewers for a manuscript?
Do peer reviewers get paid for their work?
Why should a peer reviewer agree to review a manuscript for a journal?
Do the peer reviewers know who the authors are?
Do the authors of a paper know who the peer reviewers are?
What are the supposed advantages of this system of “anonymous” peer review?
What are the potential shortcomings of this system?
What are the options for peer reviewers when responding to the journal editor?
How are disagreements among peer reviewers resolved regarding acceptability of a manuscript?
Why is research published in peer-reviewed journals usually considered a reliable source of information?
The Primary Literature: The Anatomy of a Typical Research Paper
What are the sections of a typical research paper and what is to be found in each of those sections?
In which section would you expect to find the hypothesis or a purpose statement?
In which section would you expect to read about how the results relate to what is already known?
In which section would you expect to read about any additional work that might need to be done?
In which section would you expect a description of why this research needed to be done?
In which section would you expect to find graphs and tables?
In which section would you expect to find the source of funding for the research?
What are the two types of titles and what influences which type might be selected?
A Critical Reading of King et al., 1999.
Which type of title does this paper have? Is it appropriate or would the other type be more suitable?
Who probably wrote the grant to fund this research?
Who probably wrote the first draft of the manuscript?
Do the authors have appropriate credentials?
Are they affiliated with reputable institutions?
What is the stated purpose of this research?
Does the introduction have the appropriate information?
Are the references cited from peer-reviewed scientific journals?
Are the methods described in sufficient detail to allow replication?
Why was it necessary to conduct two separate experiments?
Were there a sufficient number of subjects in each experiment?
What criteria were expected of all subjects? Are these subjects appropriate for this experiment?
How did the authors recruit their subjects?
What was the role of the ISU Human Subjects Committee?
What is meant by resistance training? How is this different from endurance training?
Why did the authors randomly assigned the subjects into the two groups?
What should be the only difference in the two groups at the onset of the experiments?
Why were the subjects in the short-term study tested twice?
What is the meaning of “double blind” in this context?
What is a placebo?
In the short term experiment, why were blood samples taken at 30 minute intervals for 6 hours?
What is the question that can be answered by the short-term study?
Why did the participants in the long-term study “cycle” with Andro and placebo?
What steps were taken to determine the purity of the product being tested?
How were the doses of Andro used in these experiments established?
Of what use was the 1-RM at the beginning and midpoint of the long-term study?
Why did only one experimenter make all the muscle measurements?
Was the diet of the participants “controlled?” If not, why not?
What were the investigators looking for when they examined biopsy samples of muscles at three time points in the long-term subjects? What did they expect to see in the muscle tissue if Andro did indeed get converted to testosterone?
What is ANOVA?
Why was the data from one subject excluded from the analysis? Should it have been excluded?
In the short term study, does Andro get into the bloodstream of the subjects?
In the short term study, why did the investigators measure FSH and LH?
Why did the investigators not measure the bound testosterone?
In the short term study, does Andro get converted to testosterone?
(For each graph in the paper, be able to state the findings and relationships in words.)
In a graph, how is a statistically significant difference indicated?
What are “error bars” and what does it mean if they “overlap?”
How are some of the graphs misleading in terms of the inclusion of asterisks?
Some of the results presented in Figures 3 and 4 are unexpected. What is difficult to explain in these figures?
In Figure 3, what is the indirect evidence that supplemental Andro is not begin converted to testosterone?
If you’re a vendor promoting Andro as an anabolic steroid precursor, what do you find disturbing about the results presented in Figure 5?
What do the results presented in Figure 5 mean to you if you’re a young male considering supplementing with Andro?
From Table 2, do both groups (Andro and placebo) get stronger with resistance training?
From Table 2, do both groups start off at the same strength levels even before they begin taking Andro or placebo? Is this problematic? Is it still possible to draw meaningful conclusions from this study? If so, explain why.
Did the Andro group get disproportionally stronger than the placebo group over the 8 weeks?
Why might the authors choose to present some of the results in graphs and other results in tables?
What is the mistake in the title of Table 3?
No figures are provided that graphically illustrate the results of the muscle histochemistry analysis. If Andro was actually getting converted to testosterone, what changes would be expected in the muscle tissue? (We’ll re-examine this part of the paper in more detail when we do muscle physiology in lecture.)
Did the investigators observe changes in the muscle specimens that indicate Andro was indeed converted to testosterone? (Again, we’ll re-examine this part of the paper when we do muscle physiology in lecture.)
Considering both the short-term and long-term studies presented here, is there any evidence that Andro supplementation results
· in elevated testosterone?
· In greater gains in strength with resistance training?
· In greater muscle mass with resistance training?
· In any potentially undesirable effects?
Describe any problems, shortcomings, and/or flaws in the experiment.
If you were the peer reviewer of this manuscript, what would you recommend to the journal editor?
If the study were to be repeated, what changes would you recommend?
Who funded the study?
Normally studies that show no changes in response to a manipulation don’t get published? Should this one have been published? Defend your answer.