Supplemental Table 3. Summaryof pre-meeting survey on guideline / recommendation development
Organization / Target Audience / Factors involved in assessing the robustness of the scientific evidence / Standardized guidance on identification of key elements / Standardized guidance on composition of members / Management of conflict of interests / Standardized criteria for synthesizing the evidence / Use of systematic reviews / Use of external reviewAmerican College of Medical Genetics and Genomics (ACMG) / Target audience for the Professional Practice and Guidelines (PP&G) committee is any practitioner caring for individuals (or populations) relevant to the guideline.
Target audience for the Laboratory Quality Assurance (Lab QA) committee is genetic laboratories and laboratorians. / Determined by two ACMG committees: PP&G committee (handles all clinical practice guidelines) and the Lab QA committee (deals with laboratory practice guidelines).
Assessments use:
- Published evidence
- Unpublished evidence
- Professional opinions
- Convene working group to: reviewing conflict of interest, review of evidence, and preparation of the draft guideline
- Draft reviewed by the responsible committee (PP&G or Lab QA)
- Draft presented to the board of directors that provides comments but ultimately approves the draft to be released for review by the entire ACMG membership
- ACMG member comments considered and incorporated where appropriate resulting in a final guideline approved by the Board
- Guidelines published on ACMG website and submitted for publication to Genetics in Medicine (GIM)
- Peer review by GIM
- Given the rarity of most conditions, a small group of content experts lead working groups
- Membership of PPG and Lab QA committees are determined by the Board, and chairs of the committees are appointed by the ACMG president
- All committee members and chairs undergo COI review before appointment and annually
- All policies and procedures are defined in a committee charter developed by the Governance committee and approved by the Board
- COI performed by committee with final approval by the board of directors
- In rare disease, experts often have commercial relationships
- Managed by making sure workgroups have representation from members who may not be content experts but have enough experience in the subject to participate and monitor for conflicts
- Board plays hands-on role in these instances to monitor the conflict
- General policy that only one individual from a given institution can participate (with exceptions)
- Knowledge synthesis does not use systematic approaches, heavy reliance on expert opinion
- Moving towards performing evidence grading
- Laboratory Quality Assurance Committee has access to information from Quality Control and Proficiency Testing activities and uses this real world information to inform guidelines
American Society of Clinical Oncology (ASCO) / Oncologists /
- Quality appraisal of the evidence is dependent on the type of studies being considered
- Therapeutic guidelines: Randomized controlled trials ( RCTs) or meta-analyses of RCTs
- Prognostic/predictive guidelines: prospective cohort studies or retrospective analyses with identifiable control groups
- Clinical Practice Guideline Committee approves proposed topics, a protocol is generated, co-chairs identified, panel members selected, disclosures and COI addressed, systematic review planned and conducted via an approved protocol
- Results are presented to the panel of content and methods experts
- Recommendations generated and reviewed internally and externally as well as by reviewers at the J Clin Oncol prior to finalization, publication, and posting on the ASCO website
- Members including chairs must not have 51% or more COI
- Balance of content and methods experts and at least one patient representative
- Described in more detail in ASCO Methods manual
- Systematic reviews required for main line guidelines
- ASCO will endorse up to date evidence-based guidelines based on systematic reviews from other credible organizations
- ASCO has adopted a Consensus Guideline process based on a modified Delphi technique when high level evidence is not available for important clinical issues
Blue Cross Blue Shield Association (BCBS TEC) / Health plans and the public / Technologies must have established:
- analytic validity
- clinical validity
- clinical utility
- 19-member Medical Advisory Panel includes appointees from the American College of Physicians, American College of Medical Genetics, American Academy of Family Physicians, American College of Cardiology, American Academy of Pediatrics, and American College of Surgeons
Clinical Pharmacogenetics Implementation Consortium (CPIC)
See publication: PMID: 21270786 / Clinicians who use pharmacogenomic testing / 3-tier scheme used to rate the quality of evidence linking drug-gene associations
Levels of Quality:
- High: consistent evidence from welldesigned, well-conducted studies
- Moderate: sufficient evidence of the effects, but evidence strength is limited by the number, quality, or consistency of the individual studies, by the inability to generalize to routine practice, or by the indirect nature of the evidence
- Weak: insufficient evidence to assess effects because of the limited number of studies, insufficient power of the studies, important flaws in their design or in the way they were conducted, gaps in the chain of evidence, or lack of information
- The key elements to include in CPIC guidelines are standardized and were agreed upon by consensus of CPIC members
- Guidelines are peer-reviewed and published in a leading journal with simultaneous posting to PharmGKB with supplemental information/data and updates
- For additional information see:
- CPIC members review guidelines prior to submitting publication
- Senior author must be a leader in the content area addressed by the guideline and co-authors should have a track record of publication or expertise in the specific topic area of the guideline
- Steering Committee approves the composition of the guideline authors and the writing plans for each new gene/drug CPIC guideline
- Welcomes other national and international groups to assist with the clinical implementation of pharmacogenetics (email: )
- CPIC members are published on the PharmGKB website
- Authors (and family members) of CPIC guidelines must fully declare all possible conflicts (including NIH funding) that could be interpreted to indicate that authors are “advocates” of the enclosed recommendations, as well as any sources of revenue from patents, stock ownership, etc
- Knowledge synthesis aims to address:
2. “What drugs are so clearly affected that you would be wrong not to act on the result if (a) you knew the genotype and (b) you wanted to prescribe that drug?”
- Authors systematically search the literature to find evidence relevant for making dosing recommendations
- Any existing consensus guidelines for the gene or drug are also reviewed.
- Guideline will not go forward if there is lack of consensus
- Dosing recommendations are based on weighting the evidence from a combination of preclinical functional and clinical data, as well as on some existing disease-specific consensus guidelines.
- Uses a slight modification of a transparent and simple system for just three categories for recommendations adopted from the rating scale for evidence-based recommendations on the use of retroviral agents
- All CPIC manuscripts are externally peer-reviewed and published
- Additional information at PharmGKB is curator-reviewed
- Regular updates are peer-reviewed and published
- Guidelines are re-examined approximately every 2-3 years
- The entire project supporting CPIC is a peer-reviewed research project supported by NIGMS, NIH
Evaluation of Genomic Applications in Practice and Prevention (EGAPP) / Researchers and clinicians interested in genomic applications /
- Quality of evidence based on study design and conduct
- Robustness of evidence based on number of studies and sample size
- Consistency and precision of estimated effects, completeness of evaluation (
- Applicability of evidence to the population for whom the answer is needed]
- Directness of evidence (in terms of outcomes studied and the health impacts important to understand)
- Ethical, legal, and social contexts, including current policy and practice
- Follow established methods for systematic review, adapted to the rules established by the decision-maker for whom we are reviewing the evidence
- Adhere to quality recommendations for reporting of findings (such as PRISMA) relevant to our work
- Systematic reviews are key to an EGAPP recommendation, they articulate the series of questions decision-makers need to have answered in order to consider the pertinent evidence relevant to an evidence-based guideline or recommendation
Dutch Pharmacogenetics Working Group (DPWG) / Dutch clinicians, pharmacists, and physicians / For each reviewed paper 2 core parameters arescored:
1) Level of evidence of the gene–drug interaction. This indicates the quality of the evidence found in literature for the gene–drug interaction, and is scored on a five-point scale with a range from 0 (lowest evidence) to 4 (highest evidence)
2) Clinical relevance of the potential adverse drug event, decreased therapeutic response, or other clinical effect resulting from the gene–drug interaction. The clinical relevance was scored on a seven-point scale derived from the National Cancer Institute's Common Toxicity Criteria.18 A clinical or pharmacokinetic effect that was not statistically significant was classified as AA (lowest impact), whereas death, for example, was classified as F (highest impact).Please also see full details in: PMID:18253145 and PMID:21412232 /
- The pharmacogenetics-based therapeutic (dose) recommendations are developed according to a strict predefined format that contains the same components for each of the individual gene-drug combinations.
- In the final report for each gene-drug combination a concise rationale is provided in order to clarify how the DPWG has arrived at its recommendations
- Members from multiple professions including: clinical pharmacists, physicians (internal medicine), clinical pharmacologists, clinical chemists, epidemiologists, and toxicologists
- Members are recruited among clinician (researchers) with a well recognized track record in any of these fields
- 2 researchers perform a systematic review of literature
- After data collection, the level of evidence and clinical relevance of each article are scored
- Results are independently reviewed by two members of the DPWG
- Result evaluated by the complete DPWG
- Relevant professional societies are consulted before the final version of the guideline is released
- Results are published in peer reviewed journals
National Comprehensive Cancer Network (NCCN) / Oncologists or other clinicians who interact with cancer patients. Guidelines are free and available to anyone who registers on the NCCN website. / High level evidence constitutes randomized clinical trials and meta-analyses. Lower level evidence can comprise smaller clinical trials, and for diseases with less available trial data, clinical experience. /
- Institutional reviews are solicited by panel members 8-12 weeks prior to annual panel meetings. External submissions, from non-member institution clinicians, industry, payers, or patients are collated by NCCN and submitted to the panels prior to annual meetings.
- Institutional reviews and external submissions are reviewed and discussed. Panel votes are taken and recorded for substantive changes to the Guidelines.
- Changes to recommendations for use of drugs and biologics are listed in transparency documents (available at nccn.org) along with the actions taken by the Guidelines Panel including votes.
- Disclosure of all relevant financial interests are required for each member of the NCCN Guidelines Panel, including chairs/co-chairs, vice chairs, panel members, NCCN headquarters staff, member of the Guidelines Steering Committee, and the NCCN Board of Directors.
- The NCCN policy for management of conflicts of interest mandates the disclosure of all financial relationships and the recusal from deliberations and/or votes of individuals with a meaningful conflict of interest.
- Category 1: High-level evidence there is uniform NCCN consensus (85% majority vote) that the intervention is appropriate.
- Category 2A: Lower level evidence there is uniform NCCN consensus (85% majority vote) that the intervention is appropriate.
- Category 2B: Lower-level evidence, there is NCCN consensus (50-85% majority vote) that the intervention is appropriate.
- Category 3: Any level of evidence, but major NCCN disagreement (at least 3 panel member from 3 institutions) that the intervention is appropriate.
National Society of Genetic Counselors (NSGC) / Practicing genetic counselors, though some guidelines are written more broadly /
- No current system to rank evidence
- Review the supporting literature for study design, sample size, rigor of analyses, and number of studies with concordant finding
- Multiple methodologically robust papers with similar findings, or meta-analyses, carry more weight than single-institution studies, especially those with smaller sample sizes or descriptive/pilot data.
- Due to limited literature in some areas of genetic counseling one cannot always rely on large trials for supporting evidence
- Focus on making recommendations that are well supported by existing literature that will guide the specific practice of genetic counseling
- Author groups are evaluated for real or perceived conflicts of interest by requiring that each author submit a CV and conflict of interest statement at the initiation of guideline writing
- Lead author cannot have COI, nor can a majority of the authors.
- Evaluate any potential bias due to authors' COI
- Authors and Practice Guidelines Committee are involved in each guideline
- Each co-author has to be approved by the Committee
- Lead authors cannot have COI, nor can the majority of a guideline's authors
- Committee members sign a COI disclosure statement annually
- Those with potential COI are not excluded from Committee membership, but must recluse themselves from reviewing a guideline on a topic with which they have real or perceived COI
- Issue concrete timelines and instructions to author groups at initiation of guideline development, and by directing the Practice Guidelines Committee to focus their review on big-picture issues such as whether each recommendation in a guideline is sufficiently supported and whether there's apparent bias.
- Ethics Advisory Committee, legal representative and Executive Board review and approve each guideline
- Guidelines undergo peer review through the Journal of Genetic Counseling
- Committee selects a liaison for each guideline
- Guideline authors bear the responsibility for synthesizing the evidence, typically by a thorough literature review
- Practice Guidelines Committee (PGC) reviews the synthesis of evidence compiled by the authors and determines whether it's sufficiently thorough and balanced
- Do not require systematic reviews for all guidelines, particularly those for which such reviews have been published recently
- When no such reviews exist, and when having a review in print would aid the guideline writing process, recommend that guideline author groups publish a systematic review prior to drafting a guideline
1