What to choose: internal reporting guidelines, external reporting guidelinesor both? A step closer to answering this dilemma.
Rodrigues A; Silva C; Cerqueira L; Carvoeiro A; Costa M; Mota P; Senra C; Almeida J; Lázaro J; Ladeiras R; Tender J; Ferreira A; Cunha P
Rodrigues P. – Class 8
ABSTRACT
Background: An article must have quality, not only in the content but also in the writing standard, in order to allow a good comprehension and use of its results.Reporting guidelines give information related to the way authors should write their articles. While some journals choose to use external reporting guidelines, others prefer to create their own, on the other hand, others mix both types and there is also a group thatdecide not to present reporting guidelines. However, since there are not sufficient studies about the advantages of using internal or external reporting guidelines,it remains uncertain which of the options lead to better and more influent articles. Aim: Verify if there is an association between the use of internal and/or external reporting guidelines and the impact of scientific articles published in journals of General and Internal Medicine.Methods:The study is a transversal, analytic and observational one. It was made a selection of journals of General and Internal Medicine using ISI Web of Knowledge, excluding the ones with an unknown language and, then, their impact factor from the years of 2008, 2009 and 2010 was collected. The section of the instructions for the authors presented in the web site of each of the selected journals was analyzed and the journals were divided in four groups: the ones that use internal reporting guidelines, the ones that use external reporting guidelines, the ones that use both of the types and the ones that don’t present reporting guidelines online. Using data from ISI Web of Knowledge, the impact of each article of each journal selected was calculated. The impact was compared to the type of reporting guidelines presented by the journal in which they were included, as well as to the impact factor of its source. Other statistics were applied to the impact factor of the journals, concerning to their evolution during the years of analysis and comparing it to the kind of reporting guidelines presented. Data was inserted and analyzed in SPSS. Results:The journals with a higher IF (category 3) have the lowest percentage of articles with no citations. Furthermore, within the journals that recommend both type of guidelines, only 2,5% have an impact factor decrease. Within the journals that have a higher impact factor (group 3), 48.6%, correspond to journals that recommend both types of guidelines.Conclusions: Conclusions: By the analysis done, it was possible to conclude that when both types of reporting guidelines are recommended or when there are no reporting guidelines presented, the journals have, on their majority, an impact factor different from zero. When analyzing the type of reporting guidelines used by the journals with the highest impact factor, it is possible to draw the conclusion that the majority of them uses both types of reporting guidelines. Looking at the evolution of the impact factors during the two years analyzed, there appears to be a relation between the use of both types of reporting guidelines and the raising in the impact factor.
KEY WORDS:Internal Reporting Guidelines, External Reporting Guidelines, General and Internal Medicine, Journal impact factor, Impact of scientific articles, Quality of Health Care, Quality Improvement, Evidence-Based Medicine, Journal Article [Publication Type].
INTRODUCTION
Scientific knowledge has evolved markedly in the recent years in part due to the new technologies that enhance its transmission,allowing more people to have access to it. In fact, there are several ways of transmitting breakthroughs: scientific journals, books, conferences, governmental and technical reports, flyers, among others. These are often available online and allow fast and easy access.[1]Nevertheless, one important way to publish new research is through scientific articles.
Scientific articles are published in scientific journals, bothin printedand inonlineversion. “Philosophical Transactions of the Royal Society” was the first scientific journal being published, in 1665, and the first medical journal, “New England Journal of Medicine”, appeared in 1812.[1] Both exist until nowadays.
A scientific paper explains the motivation of the author for doing an experiment or a certain research, the experimental design and execution, the meaning of the results and the conclusions and recommendations derived from them. Clarity, concision, organization and objectivity are crucial characteristics of a well written paper. [2] Besides,a paper should have a clear propose, be organized systematically and target a specific audience.Thus, articles have a higher quality, which allows for better understanding and more effective use of its results and hence a spread of science more clearly and successful.[3]
It is stated thatmedical professionals need to be awareof the crucial role that new information plays in their ability to make good decisions. In fact, clinicians can improve their performance and advance in their strategies to manage patientsand resources by studying the new research results.[1]Since articles are one privileged vehicle for transmission of breakthroughs, complete and transparent reporting is essential for doctors to evaluate the validity of the conducted research and decide whether or not to apply the findings in their daily practice.[4]
Readers commonly give higher attention and credibility to articles published in reference journals. It is possible to understand the influence of journals by analyzing the number of times its articles are cited, particularly through the impact factor (IF) of the journals.[5, 6]This one is the average number of times an article published in a given journal was cited in the last two years. For example, the IF of a journal in 2009 is calculated as shown in figure 1. Impact factors are evaluated by the Journal Citation Report, annually published online in ISI Web of Knowledge. Sometimes, IF is used to assess the quality of the journal but comparison between IFsof journals belonging to different scientific areas can induce biased conclusions, because some areas can be more studiedthan others.[1] When concerning the articles itself, their impact can be analyzed by determining their number of citations. However, these can be influenced by, for example, the size of the articles or the IF of the journal in which they were published.
In order to enhance reporting of scientific research, several guidelines have been created. These areinformation for the authors on how they must write their manuscript. There are divergent views about therole of its publication: while some think they are made to help authors, particularly the less experienced ones, others find that the researchers must publish as clear as possible because it is a moral duty. [7]Consequently, some journals choose to report the recommendation or obligation to use guidelines, frequently in a section called “Instructions to Authors”, while others do not.Additionally, guidelines can concern only the editing of the article or also the content and structure of the text.The latter are called Reporting Guidelines. While some journals create their own ones (internal reporting guidelines), others adopt pre-existing ones (external reporting guidelines), like the Uniform Requirements and others (see figure 2).
The Uniform Requirements for Manuscripts Submitted to Biomedical Journals are general reporting guidelines created by the International Committee of Medical Journal Editors (ICMJE)[8], also including ethical principles of biomedical research and science communication.[9] The group was initially called as the Vancouver Group, since that the first meeting occurred in Vancouver, British Columbia, in 1978. Only later it established himself as ICMJE. The first requirements were published in 1979 and the last update was done in 2010.[8]
In addition, nowadays there are several reporting guidelines which refer to a particular type of study and to other specific aspects of scientific publication.[7]Some examples are presented in Table 1.They generally provide checklists for each type of study, this is, lists with sentences referring what the manuscript must contain.[10, 11, 12, 13]
Table 1 - Examples of external reporting guidelines (acronyms and respective meanings).
GNOSIS / Guidelines for neuro-oncology: Standards for investigational studiesCONSORT / Consolidated Standards of Reporting Trials
COREQ / Consolidated criteria for reporting qualitative research
MOOSE / Meta-analysis of Observational Studies in Epidemiology
ORION / Outbreak Reports and Intervention Studies Of Nosocomial infection
PRISMA / Preferred Reporting Items for Systematic Reviews and Meta-Analyses
QUORUM / Quality of Reporting of Meta-analyses of Randomized Controlled Trials
REMARK / REporting recommendations for tumour MARKer prognostic studies
SQUIRE / Standards for Quality Improvement Reporting Excellence
STARD / Standards for the Reporting of Diagnostic Accuracy Studies
STREGA / Strengthening the Reporting of Genetic Association studies
STROBE / Strengthening the Reporting of Observational studies in Epidemiology
TREND / Transparent Reporting of Evaluations with Nonrandomized Designs
There is not enough evidence about the advantages of using reporting guidelines and its influence on quality of the articles.[4]However, even considering those guidelines necessary, there isnot information about which guidelines lead to a better paper: whether internal or external. In this context, it is important to underline that the writing of an article is of tremendous importance, because itsquality does not only depend on its scientific content, but also on the way that it is written. Quality determinates the attention the scientific community pays to the article and, consequently, the applicability of the information it contains. [14]Moreover, completeness of publication is one main factor contributing to acceptance of a paper by editors. Nevertheless, some editors consider endorsement of guidelines as burdensome and find difficult to choose which to endorse. [4] For these reasons, is important to evaluate reporting guidelines. A study will be conducted to assess which guidelines have been evaluated and which are associated with improved completeness of reporting.[4] However, is still essential to understand which option is the best between internal or external guidelines.Therefore, our study aimed to comprehend which guidelines,this is, internal, external or both types, that, when presented by journals of General and Internal Medicine, lead to a greater impact of their articles.
PARTICIPANTS AND METHODS
In this transversal, analytic and observational study, journals of General and Internal Medicine were analysed. More precisely, it was analysed the impact of articles of 2008 published in this category of journals, as well as the evolution of the impact factors of the selected journals, analysed by the type of reporting guidelines used by the journals.
The sample was obtained on the web site ISI Web of Knowledge, using as a criterion to exclude some journals from this study the language in which they were written. In fact, the journals had to be written in a language that can be understood.
The impact factor of the journals selected of 2008, 2009 and 2010 was collected also in ISI Web of Knowledge. The impact factors were analysed and divided into three groups: journals with a raising impact factor on the two years of analysis, journals with a lowering impact factor on the two years of analysis and journals that had a raising impact factor on one of the years and a lowering on the other.
Articles of 2008 of the selected journals was researched in ISI Web of Knowledge and data was exported to the notebook of Windows, followed by OpenOffice Calc, then to Excel and finally to SPSS. These data included the title of the article, the authors, the source and the number of citations of the article. The impact of each article was calculated using the following formula
The impact factor of a journal is calculated like referred in the introduction. For the impact of an article, it was needed to use the total number of citations and divide it by the impact factor of the two years after the publication of the article divided by two. This process gave information about the position of a specific article when compared to the average of publications of its source, i.e., if the article is beyond the average of the publications of that journal. Also it allowed comparing an article of a journal with other of another journal. As some of the journals didn’t present reporting guidelines for the years of 2008 and/or 2009, on the formula, all the missing values of impact factors were substituted by the impact factor of 2010, which was never missing.
By researching on the section of the instructions for authors of each of the selected journals, it was verified which kind of reporting guidelines was used so that an article could be submitted to that particular journal (internal reporting guidelines, external reporting guidelines or both types of reporting guidelines) or if the journals didn’t use reporting guidelines (in this group, were included journals which had not an available website and journals which had guidelines only related to the format of the article and not to its content). A database was designed using SPSS and this one included the type of reporting guidelines each of the selected journals use.
Several analysis were done on the hope of answering to the research question:
a)The journals were separated into four groups: one with the journals using internal reporting guidelines, other with journals using external reporting guidelines, another with journals using both types of reporting guidelines and finally one with journals which don’t use/don’t present on their website reporting guidelines. Then, the impact of the articles which best defines each of the groups analysed was calculated using the median. As this particular step of the methodology didn’t bring results that could be considered relevant, other statistical analysis was done. So, boxplots were done and the maximum and minimum impact for each group was calculated. Also using this division, it was verified the percentage of non-cited articles, articles with an impact bellow one and articles with an impact bigger than one for each of the groups.
b)The journals were also separated based on their impact factor, with the journals organized on a descending order (journals with the highest impact factors on the top). In this sense, four groups were defined by percentiles: journals below the percentile 25 were the ones with the highest impact factors and with an implicit better quality, journals between the percentiles 25 and 50 which had a good quality following the same logic, journals between the percentiles 50 and 75 which had a median quality and journals above the percentile 75 which had a poor quality. Then, it was verified the type of reporting guidelines most used in each of the groups. Cross tabulations and scatter plots were done using SPSS.
c)Using the division of journals that was made based on the behaviour of their impact factor, it was verified the proportion of journals that used internal reporting guidelines, external reporting guidelines, both types of reporting guidelines or no reporting guidelines in each of the groups previously defined (journals with a high impact factor, journals with a good impact factor, journals with a median impact factor and journals with a poor impact factor).
RESULTS
Table 2 - Table evidencing the relation between the impact factor of the journals and the availability of website by journals.
Impact Factor of 2010 / TotalMinimum-Percentil 25 / Percentil 25-Percentil 50 / Percentil 50-Percentil 75 / Percentil 75-Maximum
Available website? / Yes / Count / 37 / 29 / 35 / 36 / 137
% within Available website / 27,0% / 21,2% / 25,5% / 26,3% / 100,0%
% within Impact Factor of 2010 / 88,1% / 78,4% / 94,6% / 97,3% / 89,5%
No / Count / 5 / 8 / 2 / 1 / 16
% within Available website / 31,2% / 50,0% / 12,5% / 6,2% / 100,0%
% within Impact Factor of 2010 / 11,9% / 21,6% / 5,4% / 2,7% / 10,5%
Total / Count / 42 / 37 / 37 / 37 / 153
% within Available website / 27,5% / 24,2% / 24,2% / 24,2% / 100,0%
% within Impact Factor of 2010 / 100,0% / 100,0% / 100,0% / 100,0% / 100,0%
We wanted to establish a relation between the use of a web site and the impact factor of the journals. As we can see from the analysis of table 2, within the journals with highest IF, 97,3% have an available web site. Despite existing a high percentage of journals with available web site in the group of journals with IF categorized as 0 (88,1%), this proportion is lower. We verified that within the journals without web site, 50,0% have a low impact factor (category 1- percentile 25-50).
Table 3 - Table evidencing the relation between the impact factor of the journals and the availability of website in English by journals.
Impact Factor of 2010 / TotalMinimum-Percentil 25 / Percentil 25-Percentil 50 / Percentil 50-Percentil 75 / Percentil 75-Maximum
English website? / Yes / Count / 32 / 27 / 34 / 36 / 129
% within English website / 24,8% / 20,9% / 26,4% / 27,9% / 100,0%
% within Impact Factor of 2010 / 86,5% / 93,1% / 97,1% / 100,0% / 94,2%
No / Count / 5 / 2 / 1 / 0 / 8
% within English website / 62,5% / 25,0% / 12,5% / 0,0% / 100,0%
% within Impact Factor of 2010 / 13,5% / 6,9% / 2,9% / 0,0% / 5,8%
Total / Count / 37 / 29 / 35 / 36 / 137
% within English website / 27,0% / 21,2% / 25,5% / 26,3% / 100,0%
% within Impact Factor of 2010 / 100,0% / 100,0% / 100,0% / 100,0% / 100,0%
We also wanted to establish a relation between the impact factor and the presence of a web site written in English. We found that within all the categories of IF, the majority of journals have an English web site. For the group without English web site, we verified that the highest percentage correspond to the journals which have lowest IF (62,5%). Moreover, for the group with English web site, we verified that the highest percentage correspond to the journals which have highest IF (27,9%).
Table 4 - Table evidencing the relation between the impact of the articles and the type of guidelines of each journal.Type of guidelines / Total
None / External / Internal / External and Internal
Impact of the articles / = 0 / Count / 1919 / 3842 / 5597 / 6914 / 18272
% within Impact of the articles / 10,5% / 21,0% / 30,6% / 37,8% / 100,0%
% within Type of guidelines / 50,4% / 54,1% / 54,1% / 43,8% / 49,3%
>0 and <=1 / Count / 237 / 1114 / 1154 / 3628 / 6133
% Impact of the articles / 3,9% / 18,2% / 18,8% / 59,2% / 100,0%
% within Type of guidelines / 6,2% / 15,7% / 11,2% / 23,0% / 16,6%
>1 / Count / 1654 / 2151 / 3593 / 5232 / 12630
% within Impact of the articles / 13,1% / 17,0% / 28,4% / 41,4% / 100,0%
% within Type of guidelines / 43,4% / 30,3% / 34,7% / 33,2% / 34,1%
Total / Count / 3810 / 7107 / 10344 / 15774 / 37035
% within Impacto f the articles / 10,3% / 19,2% / 27,9% / 42,6% / 100,0%
% within Type of guidelines / 100,0% / 100,0% / 100,0% / 100,0% / 100,0%
We found (tab.3) that within the journals that do not recommend reporting guidelines, half (50,4%) of the articles are not cited. However, there are also a great proportion of articles with impact higher than 1 (43,4%). Within journals that recommend external guidelines, these proportions are similar, but there are an increase in the proportion of articles with zero citations, and a decrease in the proportion of articles with impact higher than 1. Comparing this group with journals that present internal guidelines, the proportion of articles with zero citations is the same, but there are a higher proportion of articles with an impact higher than 1. For journals presenting both types of guidelines, the proportion of articles with zero citations is the smaller one, but the proportion of articles with impact higher than 1 is only higher than the one for journals presenting external guidelines. Within all the groups of impact of the articles, there is an increase in the proportion of articles for each type of guidelines following the order none, external, internal and both reporting guidelines.