Appendix 2. Critical appraisal tool for diagnostic studies with an interventiona

Date of appraisal: / / / Article reference:
Appraised by:
Abstract:

Section A, Is the diagnostic test study valid?

Answer options
1.  Was there a clear question for the study to address? It should include:
Population: / Yes ☐, No ☐, Unclear ☐
Test: / Yes ☐, No ☐, Unclear ☐
Setting: / Yes ☐, No ☐, Unclear ☐
Outcome: / Yes ☐, No ☐, Unclear ☐
Research question:
Research aim:
2.  Was there a comparison with an appropriate reference (gold) standard? The reference standard should be the best available indicator of the target disorder. / Yes ☐, No ☐, Unclear ☐
Comments on section A:

If the answer is yes to questions 1 and 2, continue with the analysis

Section B, Population

Answer options
3.  Are the inclusion and exclusion criteria stated? / Yes ☐, No ☐, Unclear ☐
4.  Are the setting and location data collected? / Yes ☐, No ☐, Unclear ☐
5.  Recruitment was based on:
Presenting symptoms ☐, Results from previous tests ☐, Participants had received the index test or the reference standard ☐, Other ☐ (Explain):
6.  Did the study include people with all the common presentations of the target disorder? For example, symptoms of early manifestations as well as people with more severe symptoms, and/or people with other disorders that are commonly confused with the target disorder. / Yes ☐, No ☐, Unclear ☐
7.  Is the disease status of the tested population clearly described? Consider presenting symptoms, disease stage or severity, co-morbidity, differential diagnosis (spectrum of bias) / Yes ☐, No ☐, Unclear ☐
Comments on section B:

Section C, Blinding

Answer options
8.  Were the people assessing the results of the index diagnostic test blinded to the results of the reference (gold) standard? Or were the tests performed independently? (review bias) / Yes ☐, No ☐, Unclear ☐
Comments on section C:

Section D, Testing

Answer options
9.  Did all the patients get the diagnosis test and the reference (gold) standard? Check the 2×2 table (verification bias). / Yes ☐, No ☐, Unclear ☐
2×2 tables reproduction:
10.  How was the data collection conducted?
Prospective ☐, Retrospective ☐, Other ☐
11.  Was the diagnostic test validated in a second independent group of patients? / Yes ☐, No ☐, Unclear ☐
Comments on section D:

Section E, Methods for diagnostic test and intervention for test application

Answer options
Diagnostic test: were the methods of the diagnostic test described in sufficient detail? Consider if the descriptions of the following are included:
12.  Rationale for the reference standard:
Describe rationale: / Yes ☐, No ☐, Unclear ☐
13.  Technical specifications or references for running the index test and reference standard (e.g., including enough information so the tests could be replicated): / Yes ☐, No ☐, Unclear ☐
14.  Definition of a rationale for the units, cut-offs and/or categories of the results of the index test and the reference standard are described: / Yes ☐, No ☐, Unclear ☐
15.  Methods for calculating or comparing measures of diagnostic accuracy:
Sensitivity ☐, Specificity ☐, Statistical uncertainty (95% CI) ☐, Unclear ☐,NR ☐
16.  Was the test reproducibility (inter reliability) assessed? If so, how?
Yes ☐, No ☐, Explain:
Intervention for test application: was the intervention to apply the index test described is sufficient detail, considering measures for evaluating the intervention? The following questions must be considered.
17.  Was there an intervention planned to implement the diagnostic test?
If so, what was the name of the intervention? / Yes ☐, No ☐, NA ☐, NR ☐
18.  Is the intervention information available? / Yes ☐, No ☐, NA ☐, NR ☐
19.  If there was an intervention, describe:
-  Any physical or informational materials used in the intervention, including those provided to participants or used in intervention delivery or in training of intervention providers.
Description:
-  Any procedure, activities, and/or processes used in the intervention, including any enabling or support activities.
Description:
-  The role and qualifications of any provider/practitioner involved in the intervention.
Description:
-  The training and follow up (if applicable) the intervention providers received previous and during the intervention as well as the strategies for adherence.
Description:
-  The location/unit/setting where the intervention was delivered.
Description:
-  The number of times the intervention was delivered and over what period of time including the number of sessions, their schedule, and their duration, intensity or dose. If the intervention is complex, provide details for each component.
Description:
-  Other details:
20.  Is there is any process measure established to evaluate the intervention?
If so, explain (i.e. if the screening/alert system instrument was applied/use in all patients, if it was properly used): / Yes ☐, No ☐, NA ☐, NR ☐
21.  Is the replication of the intervention possible?
Explain: / Yes ☐, No ☐, Unclear ☐
Comments on section E:

If the answer is ‘no’ to most of these questions (3 to 21), is it worth continuing as the results of the study are unlikely to be valid?

Section F, Results

Answer options
Diagnostic test
22.  When the study was performed? Include beginning and end dates of recruitment.
Recruitment begins: Recruitment ends: Unclear ☐
23.  Time interval between the index test and the reference standard and any treatment in between.
Months: Treatment: Not applicable ☐
24.  Are there clinical and demographic characteristics of the people included in the study (e.g., age, sex, spectrum of presenting symptoms, comorbidity, current treatments, recruitment centres)? / Yes ☐, No ☐, Unclear ☐
25.  Do the results include information about the people who satisfied inclusion criteria for the study but did not receive the diagnostic index or reference standard test? / Yes ☐, No ☐, Unclear ☐ NA ☐
26.  Do the results include how indeterminate results, missing results, and outliers of the index or reference standard test were handled? / Yes ☐, No ☐, Unclear ☐
NA ☐
27.  Do the results include criteria for defining the severity of the target disorder? / Yes ☐, No ☐, Unclear ☐
28.  Do the results include cross tabulation of the index test results by the reference standard results? Or enough information to generate this table? / Yes ☐, No ☐, Unclear ☐
29.  Do the results include estimates of diagnostic test accuracy and statistical uncertainty? Or do they provide data to calculate them? Provide or calculate values.
Sensitivity: Specificity: Positive predictive value: Negative predictive value:
95% CI: Likelihood ratios: Not available ☐ Not applicable ☐
30.  How sure are you about the results? Could the results have occurred by chance? Check P value and 95% CI.
Explain:
31.  Is the index test cost effective?
Explain:
Intervention
32.  Are the results of the intervention reported? If the intervention is complex, provide details for each component. / Yes ☐, No ☐, Unclear ☐NA ☐, NR ☐
33.  Was the intervention applied as intended?
If the intervention was modified during the course of the study, describe the changes (what, why, when, and how, including if there was any drift over time in terms of the intervention components or the extent the intervention was delivered as planned).
Description: / Yes ☐, No ☐, Unclear ☐
NA ☐, NR ☐
34.  What are the results of the intervention?
Describe (primary, secondary, and other outcomes, including the statistical test applied):
35.  Is the intervention cost effective?
Explain:
Comments on section F:

Section G, Application of findings

Answer options
36.  Are your patients similar to the people in the study in terms of clinical and demographic characteristics? / Yes ☐, No ☐, Unclear ☐
37.  Does this diagnostic test apply to your specific patients or population of interest? Consider resources, cost and level of expertise required to interpret the test. / Yes ☐, No ☐, Unclear ☐
38.  Is the diagnostic test available or the intervention possible to undertake in your setting? And if so, does it reflect current practice?
Explain: / Yes ☐, No ☐, Unclear ☐
39.  Will the test result change the way the patient is managed?
Explain:
Comments on section G:

aBased on: (i) BMJ Diagnostic test studies and critical appraisal (Accessed on October 2015: http://clinicalevidence.bmj.com/x/set/static/ebm/toolbox/665061.html), (ii) ©Critical Appraisal Skills Programme (CASP) Diagnostic Test Study Checklist 31.05.13, and (iii) STARD checklist for reporting of studies of diagnostic accuracy, and c. Hoffmann TC, Glasziou PP, Boutron I, Milne R, Perera R, Moher D, Altman DG, Barbour V, Macdonald H, Johnston M, Lamb SE, Dixon-Woods M, McCulloch P, Wyatt JC, Chan AW, Michie S. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ. 2014 Mar 7;348:g1687. doi: 10.1136/bmj.g1687.

1

Note: when necessary elaborate next to the question. Abbreviations: NR= not reported, NA = not applicable