Guideline harmonization and implementation plan for the BETTER trial: Building on Existing Tools to Improve Chronic Disease Prevention and Screening in Family Practice
=======================================================================================================================================================================

* Denise Campbell-Scherer
* Jess Rogers
* Donna Manca
* Kelly Lang-Robertson
* Stephanie Bell
* Ginetta Salvalaggio
* Michelle Greiver
* Christina Korownyk
* Doug Klein
* June C. Carroll
* Mel Kahan
* Jamie Meuser
* Sandy Buchman
* Rebekah M. Barrett
* Eva Grunfeld

## Abstract

**Background** The aim of the Building on Existing Tools to Improve Chronic Disease Prevention and Screening in Family Practice (BETTER) randomized controlled trial is to improve the primary prevention of and screening for multiple conditions (diabetes, cardiovascular disease, cancer) and some of the associated lifestyle factors (tobacco use, alcohol overuse, poor nutrition, physical inactivity). In this article, we describe how we harmonized the evidence-based clinical practice guideline recommendations and patient tools to determine the content for the BETTER trial.

**Methods** We identified clinical practice guidelines and tools through a structured literature search; we included both indexed and grey literature. From these guidelines, recommendations were extracted and integrated into knowledge products and outcome measures for use in the BETTER trial. End-users (family physicians, nurse practitioners, nurses and dieticians) were engaged in reviewing the recommendations and tools, as well as tailoring the content to the needs of the BETTER trial and family practice.

**Results** In total, 3–5 high-quality guidelines were identified for each condition; from these, we identified high-grade recommendations for the prevention of and screening for chronic disease. The guideline recommendations were limited by conflicting recommendations, vague wording and different taxonomies for strength of recommendation. There was a lack of quality evidence for manoeuvres to improve the uptake of guidelines among patients with depression. We developed the BETTER clinical algorithms for the implementation plan. Although it was difficult to identify high-quality tools, 180 tools of interest were identified.

**Interpretation** The intervention for the BETTER trial was built by integrating existing guidelines and tools, and working with end-users throughout the process to increase the intervention’s utility for practice. Trial registration: ISRCTN07170460

Most of the health care activities for improving the prevention of and screening for chronic diseases are implemented in family practice.1–5 About 7.4 hours per day of provider time are required to deliver all manoeuvres recommended within current practice structures.6–13 Because of the multiple demands of family practice, this is not feasible. Primary care practitioners also require effective strategies to concurrently address multiple conditions, because 45% of people have more than one chronic disease.14 In particular, depression — a prevalent comorbidity in family practice — is associated with poor uptake of prevention and screening manoeuvres.15–19 Therefore, promoting practice organization that delivers prevention and screening in an effective and time-efficient manner is a priority.

Cardiovascular disease, diabetes and breast, colorectal and cervical cancer, as well as their associated lifestyle risk factors (e.g., tobacco use, alcohol overuse, poor nutrition, physical inactivity), are prevalent in primary care. There are evidence-based recommendations for the prevention of and screening for these conditions, and following these recommendations can reduce morbidity and mortality. However, there are dozens of clinical practice guidelines with recommendations for each of these conditions, which creates a substantial challenge for providers. This is one of the reasons why the effectiveness of guidelines as tools to facilitate the translation of evidence into practice has been inconsistent.20–22

The harmonization of guidelines and their integration into an implementation plan represent important first steps in addressing this issue.23 In this article, we describe the process by which we harmonized recommendations from clinical practice guidelines and implementation tools for the target conditions. The results of this process were used to inform the interventions in the Building on Existing Tools to Improve Primary Prevention and Screening in Family Practice (BETTER) randomized controlled trial.24

## Methods

The knowledge-to-action cycle was the starting framework for the design of the BETTER evidence integration process, the core of which is “knowledge creation.”25 With engagement from end-users (family physicians, nurse practitioners, nurses and dieticians), we identified high-quality clinical practice guidelines, harmonized them to standardize the recommendations, and created knowledge products for the implementation process. Building on previous work by the Centre for Effective Practice and incorporating processes of guideline knowledge translation, we developed a structured approach to the evidence integration (Figure 1).23,25–27

![Figure 1:](http://www.cmajopen.ca/https://www.cmajopen.ca/content/cmajo/2/1/E1/F1.medium.gif)

[Figure 1:](http://www.cmajopen.ca/content/2/1/E1/F1)

Figure 1:  
Process for identifying and integrating evidence for the Building on Existing Tools to Improve Chronic Disease Prevention and Screening in Family Practice (BETTER) trial. Note: AGREE = Appraisal of Guidelines for Research and Evaluation.

From November 2009 to March 2010, a clinical working group was convened and facilitated by the Centre for Effective Practice; this group included clinicians (family physicians, a nurse practitioner, registered nurses, a licensed practical nurse and a dietician) and methodologists, including a medical librarian. Appendix 1 presents details on the members of the clinical working group (available online at [www.cmajopen.ca/content/2/1/E1/suppl/DC1](http://www.cmajopen.ca/content/2/1/E1/suppl/DC1)).

The purpose of the working group was to select and integrate high-quality recommendations for the chronic conditions selected for inclusion in the BETTER trial. These included cardiovascular disease, diabetes and breast, colorectal, lung and cervical cancer, as well as the associated lifestyle risk factors (e.g., tobacco use, alcohol overuse, poor nutrition, physical inactivity). The working group considered recommendations relevant to patients aged 40–65 years and those with strong evidence linked to a target or clear health outcome. A separate search was performed for interventions for depression and for patients with strong family history of disease.

### Harmonization process

The key steps in the harmonization process are outlined in Figure 1; the complete process, including implementation, based on the knowledge-to-action cycle, is shown in Figure 2.

![Figure 2:](http://www.cmajopen.ca/https://www.cmajopen.ca/content/cmajo/2/1/E1/F2.medium.gif)

[Figure 2:](http://www.cmajopen.ca/content/2/1/E1/F2)

Figure 2:  
The evidence integration and implementation process for the Building on Existing Tools to Improve Chronic Disease Prevention and Screening in Family Practice (BETTER) trial. The triangle in the centre of the diagram is an extension of the “knowledge creation funnel” in the knowledge-to-action cycle.25 In our process, there is knowledge synthesis with each funnel representing existing literature captured in high-quality clinical practice guidelines. This is then contextually integrated for each patient’s family history and modifiable risk factors. The side bar shows the structured evidence review process of the clinical working group. The boxes around the circumference of the cycle refer to the steps for implementing the recommendations and tools in both the practice- and patient-level interventions of the BETTER trial. Note: CVD = cardiovascular disease, EMR = electronic medical record.

#### Step 1: Identify, evaluate and select high-quality clinical practice guidelines and tools

We identified high-quality guidelines through a focused 4-step process (Box 1). This process was based on the search strategy developed by the Guidelines Advisory Committee at the Centre for Effective Practice (*Canadian Medical Association Handbook on Clinical Practice Guidelines*10). The inclusion criteria were diabetes, cardiovascular disease, breast cancer, colorectal cancer, cervical cancer, skin cancer and lung cancer. The search was limited to guidelines published in English between 2004 and 2009. Guidelines were excluded if they did not include any recommendations related to screening or primary prevention for the areas of interest or were not applicable to family practice. We also performed a search for effective interventions for patients with depression (Box 1).

Box 1:  
### Four-step search process for high-quality clinical practice guidelines

1. Guideline repositories:

• CMA Infobase: Clinical Practice Guidelines: [http://mdm.ca/cpgsnew/cpgs/index.asp](http://mdm.ca/cpgsnew/cpgs/index.asp)

• National Guideline Clearinghouse: [www.guidelines.gov](http://www.guidelines.gov)

2. Renowned developers:

• National Institute for Health and Clinical Excellence: [www.nice.org.uk/](http://www.nice.org.uk/)

• New Zealand Guidelines Group: [www.nzgg.org.nz](http://www.nzgg.org.nz)

• Scottish Intercollegiate Guidelines Network: [www.sign.ac.uk/guidelines/index.html](http://www.sign.ac.uk/guidelines/index.html)

• University of Michigan: [http://cme.med.umich.edu/iCME/default.asp](http://cme.med.umich.edu/iCME/default.asp)

• US Preventive Services Task Force: [www.ahrq.gov/clinic/prevnew.htm](http://www.ahrq.gov/clinic/prevnew.htm)

3. Websites of national and international specialty societies (e.g., for diabetes: the Canadian Diabetes Association, American Diabetes Association)

4. General Internet Search:

• Google: [Topic] AND [Guideline(s)]. First 3 pages of results examined

5. For the depression CDPS uptake literature review

• Medline and PsychINFO were searched using relevant depression, primary care and prevention MeSH search terms; 239 abstracts were found. A clinical filter was applied; a clinician reviewed the abstracts and identified 17 articles of interest. These articles were then reviewed by 3 other investigators for relevance; debate was resolved by consensus. Of these, 3 were considered of sufficient quality and applicability for subsequent review with the larger clinical working group of investigators for consideration of clinical applicability and acceptance in primary care.

We retrieved the full-text versions of the guidelines identified and evaluated them using the AGREE (Appraisal of Guidelines for Research and Evaluation) instrument,27 with a particular focus on the domains of “rigour of development” and “editorial independence.” This ensured that a systematic search was conducted, recommendations in the guidelines were linked directly to the level of evidence, and that the guideline was developed with minimal bias. In an effort to build on existing work in guideline assessment and adaptation, we asked the Canadian Partnership Against Cancer to provide the full AGREE scores from their previous work for all guidelines for breast, cervical, colorectal, lung and skin cancer published between 2004 and 2008. We identified and assessed additional guidelines in these areas published between 2008 and 2009. We located 3–5 high-quality clinical practice guidelines in each topic area. Canadian guidelines and those published most recently were included if they satisfied the quality criteria.

The BETTER trial’s search and appraisal strategy for existing patient tools is shown in Box 2. Using a structured search of the grey literature, we identified and reviewed 180 clinical tools (> 1000 were considered) for potential inclusion in the intervention component of the BETTER trial (the BETTER tool kit). A selection of tools was chosen for the assessment of alcohol, depression, smoking, diet, exercise and family history. To avoid redundancy, we used a ranking and indexing scheme to help facilitate cataloguing and retrieving high-quality tools to decide which tools to include in the implementation.

Box 2:  
### Search process for high-quality clinical tools

1. Criteria for consideration

• Published in English

• Address screening and primary prevention of topic area recommendations

2. Search strategy

• Public Health Agency of Canada’s Canadian Best Practices Portal

• Agency for Healthcare Research and Quality

• Preventive, chronic and primary care

• Internet search using Google:

• [Topic] AND [tools] AND [prevention/screening] AND [Canada] AND [patients] AND [providers]

• [Country] AND [topic]. The first 3 pages of results were examined

• Websites of national and provincial specialty societies

3. Criteria for inclusion:

• Screening and prevention tools

• More weight was given to the most up-to-date tools and those from Canadian and American organizations

4. Extraction of tools:

• Unique name, description of tool, focus, source, organization

• Reviewer instructions (type/usefulness and quality/applicability)

• Clinical working group’s overall review for inclusion in tool kit

5. Review process:

• Two independent reviewers, process for resolving differences

• Clear process for criteria to evaluate tools:

• Type of tool/clinical usefulness

• Quality of the tool/applicability

• Validation of the tool in multiple settings

6. Placing tools into tables:

• Includes name, description of tool, focus, source and organization

• Reviewer of Instructions (type/usefulness and quality/applicability)

• Clinical working group’s overall review for inclusion in the tool kit

7. Need a method to index, search and retrieve tools in the literature:

• Dedicated repositories exist, but they would benefit from indexing and preappraisal of the tools

#### Step 2: Extraction of the relevant recommendations for review

We extracted into evidence tables all relevant recommendations from the highest-quality guidelines in each topic area. These recommendations, along with the level of evidence and strength of recommendation according to the evidence taxonomy used by the particular guideline developer, were presented to the working group for review. The levels of evidence taxonomy for each guideline recommendation for a patient intervention were compared with the prespecified standards for quality. The exception to this quality metric was areas for which a decision for practice implementation had to be made on the basis of lower-quality recommendations, such as the frequency of repeat manoeuvres.

#### Step 3: Evidence review

In this phase, the guidelines were reviewed by smaller, disease-specific theme groups created from the members of the full clinical working group. The themes included diabetes, cardiovascular disease, cancer, lifestyle modification (smoking, alcohol, physical activity, nutrition), family history and depression. Members with content expertise were active in each group. At least 2 members of each group independently reviewed each recommendation and clinical tool. Disagreements were resolved through discussion among the group. All recommendations identified were presented to the entire working group for final review. For clinically controversial recommendations (i.e., because of information that may have postdated the publication of the guideline), we considered the primary literature. A literature review was conducted to identify strategies to increase the uptake of manoeuvres among patients with depression. To ensure that recommendations were appropriate within the specific provincial contexts, we identified local guidelines and policies from provincial bodies (e.g., provincial cancer screening policies). We compared these local guidelines with the selected recommendations to ensure consistency and applicability.

#### Step 4: Recommendation integration and knowledge product development

The clinical working group reviewed the recommendations for final inclusion. The group integrated the selected recommendations into a format that was explicit, actionable and measurable. Appendix 2 (available online at [www.cmajopen.ca/content/2/1/E1/suppl/DC1](http://www.cmajopen.ca/content/2/1/E1/suppl/DC1)) summarizes one example of this, showing the recommendations from 4 different guidelines concerning the timing of screening for diabetes. Each of guideline had a different taxonomy for indicating level of evidence and strength of the recommendation. In the example in Appendix 2, recommendation number 2 is the only one that is specific, comprehensive and actionable.

The clinical working group, Centre for Effective Practice and the research team worked together to integrate the recommendations into knowledge products that would inform the measures to be used in the BETTER trial (defining the manoeuvres, targets and outcomes); inform the clinical approach that would be used to implement the recommendations in family practice; and assist in the training of the trial team and primary care providers about the evidence and how to engage patients in meaningful discussions about the prevention of and screening for chronic diseases.

## Results

### Integrated recommendations from guidelines

The selected guidelines are presented in Appendix 3 (available online at [www.cmajopen.ca/content/2/1/E1/suppl/DC1](http://www.cmajopen.ca/content/2/1/E1/suppl/DC1)). We found strong evidence for manoeuvres for the prevention of and screening for breast, colorectal and cervical cancer, diabetes and cardiovascular disease and for the prevention of lung cancer. There was an absence of evidence for screening the general population for skin and lung cancer and for screening for colorectal and breast cancer among patients aged 40–49 years. Given this, the working group recommended that these conditions not be included in the BETTER trial. Screening for breast and colorectal cancer was only recommended for patients aged 40–49 years with a family history indicative of higher risk. The working group requested additional searches for prostate cancer guidelines. Following an additional review, the working group concluded that there was insufficient evidence to recommend screening. The working group concluded that there was no high-quality evidence showing effectiveness of strategies to improve screening and primary prevention in patients with depression.

### Knowledge products

We developed clinical algorithms to relay the principal results of the guideline review of the strongest available recommendations for the prevention of and screening for chronic diseases. The algorithms provide a clear approach specifically tailored to the intervention. We developed algorithms for the primary prevention of and screening for cardiovascular disease and diabetes (Figure 3), the prevention of and screening for cardiovascular disease among patients with existing type 2 diabetes (Figure 4), and the prevention of and screening for breast, cervical and colorectal cancer and for the primary prevention of lung cancer (Figure 5).

![Figure 3:](http://www.cmajopen.ca/https://www.cmajopen.ca/content/cmajo/2/1/E1/F3.medium.gif)

[Figure 3:](http://www.cmajopen.ca/content/2/1/E1/F3)

Figure 3:  
The Building on Existing Tools to Improve Chronic Disease Prevention and Screening in Family Practice (BETTER) map for primary prevention and screening for type 2 diabetes and coronary artery disease summarizes all of the relevant recommendations from the clinical practice guidelines. Starting at the rectangular boxes in the middle of the diagram, each parameter is evaluated for the patient. If the patient is on target, follow the arrows up and reinforce positive lifestyle behaviour. If the patient is not on target, follow the arrow down, consider appropriate interventions and intervene on lifestyle behaviours. The risk calculator is used for shared decision-making to illustrate the impact of behaviours like smoking on cardiovascular health. Note: CAD = coronary artery disease.

![Figure 4:](http://www.cmajopen.ca/https://www.cmajopen.ca/content/cmajo/2/1/E1/F4.medium.gif)

[Figure 4:](http://www.cmajopen.ca/content/2/1/E1/F4)

Figure 4:  
The Building on Existing Tools to Improve Chronic Disease Prevention and Screening in Family Practice (BETTER) map for primary prevention and screening for coronary artery disease in patients with pre-existing type 2 diabetes. This map summarizes the recommendations of the clinical practice guidelines. Note: CAD = coronary artery disease, LDL = low-density lipoprotein, UKPDS = United Kingdom Prospective Diabetes Study Risk Engine.

![Figure 5:](http://www.cmajopen.ca/https://www.cmajopen.ca/content/cmajo/2/1/E1/F5.medium.gif)

[Figure 5:](http://www.cmajopen.ca/content/2/1/E1/F5)

Figure 5:  
The Building on Existing Tools to Improve Chronic Disease Prevention and Screening in Family Practice (BETTER) map for primary prevention and screening for common cancers in primary care. This algorithm incorporates tools that take family history into account to assess where routine population screening measures should be modified for patients at higher risk. It incorporates some regional differences in screening protocols between jurisdictions, all of which were deemed reasonable by the clinical working group. Note: AB = Alberta, FOBT = fecal occult blood test, HNCPP = hereditary nonpolyposis colorectal cancer, ON = Ontario, Pap = Papanicolaou smear.

## Interpretation

The dynamic, iterative process used to integrate the guideline recommendations was critical for effective implementation of the intervention in the BETTER trial. Our findings are consistent with the literature: for a recommendation to be clinically useful, it not only has to have robust evidence, it also has to be formulated in a way that is actionable.20–23,28

We disassembled high-quality clinical practice guideline recommendations into their core components and reassembled them into integrated, implementable and measurable recommendations for use in the BETTER trial. We accomplished this using a participatory approach through direct engagement with end-users and researchers, ensuring direct interactivity between the “knowledge creation” process and the “knowledge application” process from the knowledge-to-action cycle, while facilitating greater uptake within the BETTER trial. With end-user engagement, we reformatted the recommendations for patients aged 40–65 years into an algorithm to facilitate clinical decision-making and encourage implementation. Figure 2 illustrates the process of harmonizing of guideline recommendations and implementing them at the practice and patient level. At the patient level, the recommendations implemented were tailored to patients’ personal risk factors and family history.

There are a number of effective, yet under-used, evidence-based manoeuvres for the screening for and prevention of chronic diseases. These manoeuvres are particularly under-used for patients with depression.6–8 No robust primary studies demonstrating solid outcomes improving uptake of manoeuvres were identified for patients with depression in primary care; the lack of high-level evidence for such interventions suggests that further research into this area is needed. Although there is a paucity of studies on harmonizing guidelines across multiple conditions, there is interesting work in this direction with the C-CHANGE initiative in cardiovascular disease diagnosis and management.29

The BETTER trial’s evidence integration process highlights practical considerations contributing to our understanding of guideline implementation challenges in family practice settings and for guidelines encompassing multiple clinical areas. Our findings are consistent with previously published studies showing the complexity in facilitating a process of adaptation or harmonization.23,28 Our iterative review of guideline recommendations, with an eye to clinical applicability, showed that the imprecise language of recommendations results in a discussion about what we should actually “do.” We realized that for use across multiple diseases, we needed to focus on the recommendations that have the greatest effect and best evidence. The variation in taxonomy regarding the strength of recommendations challenged the proper interpretation of the strength of evidence for each guideline. We believe that this is a limitation to the uptake of guidelines and concur with others that to facilitate uptake there is potential value in adopting a consistent taxonomy for level of evidence and strength of recommendation.30 Some have advised that clinicians need to “find guidelines that are systematic and transparent, and then make informed judgments;”31 however, the half-life of all clinical research is 5.5 years,32 making the primary appraisal of all guidelines at point of care impractical. There is a need for guideline developers and knowledge users to develop processes to harmonize clinical practice guidelines.

An important aspect of this work was the iterative approach used to ensure that the evidence and knowledge products were informed by the needs of the users. The process for this continues to be explored through the participatory approach of the ADAPTE process and emerging process for integrated knowledge translation.23,28,33 The BETTER trial gives one example of bringing together end-users and researchers to collaboratively develop knowledge products and training approaches to support integrated guidelines for chronic disease prevention and screening implementation in family practice.

### Limitations

It is possible that we may have missed a relevant clinical practice guideline that may have had different recommendations. However, we are confident that our sample was robust because we included recommendations with good evidence supporting their impact on patients, we achieved general thematic consensus across the different guidelines, and our clinical working group did not identify any areas of deficit.

The working group, while attempting to follow best evidence, was required to rely on consensus when the evidence was insufficient or unclear, as in the case of the frequency of repeat measures in practice. Varying interpretations of this evidence, along with differing values with regards to harms and benefits, while ultimately resolved by consensus, relied on group dynamics and decision-making, which are subject to bias.

The rapid rate of new knowledge across the included topics means that the algorithms created for the BETTER trial need to be modified regularly. We have recently undertaken this for the BETTER2 project, which is implementing the patient-level intervention in urban, rural and remote communities in Canada.

### Conclusion

A solid evidence base and the inclusion of target end-users in the development of knowledge products are required for initiatives to improve clinical care processes in family practice. An important aspect of this work was the iterative, deliberative process that yielded clinical recommendations that were explicit and measurable, as well as the BETTER clinical algorithms, which were an effective way to summarize the integrated recommendations for implementation. Clinical practice guidelines are useful syntheses of information about single diseases. Improved uptake in practice may be achieved by increasing collaboration between guideline developers and end-users, with a focus on strategies for implementation.

### Supplemental information

For reviewer comments and the original submission of this manuscript, please see [www.cmajopen.ca/content/2/1/E1/suppl/DC1](http://www.cmajopen.ca/content/2/1/E1/suppl/DC1)

## Acknowledgements

We acknowledge the clinical contributions of Dr. Andrew Cave, Department of Family Medicine University of Alberta; Dr. Sheila Dunn, Department of Family and Community Medicine, University of Toronto; all of the BETTER clinical working group and implementation team clinicians; Ms. Rachael Wymer and the prevention practitioners. We thank Ms. Shelley Bible RN for her ongoing engagement in training the next generation of prevention practitioners for BETTER2. We also acknowledge the work of the Guidelines Advisory Committee, and specifically thank Dr. Valerie Palda and Dr. Melissa Brouwers for their willingness to share the AGREE scores for the cancer guidelines.

## Footnotes

*   Competing interests: Jess Rogers was employed by the Centre for Effective Practice through the BETTER project to facilitate the guideline development process. Sandy Buchman has served as the regional primary care lead for the Toronto Regional Cancer Program. None declared for Denise Campbell-Scherer, Donna Manca, Kelly Lang-Robertson, Stephanie Bell, Ginetta Salvalaggio, Michelle Greiver, Christina Korownyk, Doug Klein, June Carroll, Mel Kahan, Jamie Meuser, Rebekah Barrett and Eva Grunfeld.

*   Contributors: All of the authors contributed substantially to the conception and design of the article and to the drafting and revising of the manuscript. All of the authors approved the final version submitted for publication.

*   Funding: This research was made possible through a financial contribution from the Canadian Partnership Against Cancer and Health Canada. This research is supported by the Heart and Stroke Foundation of Ontario, Grant No. PG 10-0479. Eva Grunfeld is supported by a clinician scientist award from the Ontario Institute for Cancer Research (OICR) with funds from the Ontario Ministry of Research and Innovation (MRI). The opinions, results and conclusions reported are those of the authors and are independent of the funding sources. No endorsement by OICR or MRI is intended or should be inferred.

## References

1.  Krueger H., et al. Establishing priorities among effective clinical prevention services in British Columbia: summary and technical report. British Columbia Clinical Prevention Policy Review Committee; 2008.
    
    

2.  McIsaac WJ, Fuller-Thomson E, Talbot Y. Does having regular care by a family physician improve preventive care? Can Fam Physician 2001;47:70-6.pmid:11212436
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiY2ZwIjtzOjU6InJlc2lkIjtzOjc6IjQ3LzEvNzAiO3M6NDoiYXRvbSI7czoxODoiL2NtYWpvLzIvMS9FMS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

3.  Upshur R, Leong A, Maaten S, et al. Patterns of primary and secondary prevention. In: Jaakkimainen L, Upshur R, Klein-Geltink JE, et al., editors. *Primary care in Ontario: ICES Atlas*. Toronto (ON): Institute for Clinical and Evaluative Sciences; 2006:78-89.
    
    

4.  Starfield B, Shi L, Macinko J. Contribution of primary care to health systems and health. Milbank Q 2005;83:457-502.pmid:16202000
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1111/j.1468-0009.2005.00409.x&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=16202000&link_type=MED&atom=%2Fcmajo%2F2%2F1%2FE1.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000232231200005&link_type=ISI) 

5.  Stead LF, Bergson G, Lancaster T. Physician advice for smoking cessation. Cochrane Database Syst Rev 2008; (2):CD000165.pmid:18425860
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=18425860&link_type=MED&atom=%2Fcmajo%2F2%2F1%2FE1.atom) 

6.  Yarnall KSH, Pollak KI, Ostbye T, et al. Primary care: Is there enough time for prevention? Am J Public Health 2003;93:635-41.pmid:12660210
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.2105/AJPH.93.4.635&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=12660210&link_type=MED&atom=%2Fcmajo%2F2%2F1%2FE1.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000181775600033&link_type=ISI) 

7.  Feifer C, Ornstein SM. Strategies for increasing adherence to clinical guidelines and improving patient outcomes in small primary care practices. Jt Comm J Qual Saf 2004;30:432-41.pmid:15357133
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=15357133&link_type=MED&atom=%2Fcmajo%2F2%2F1%2FE1.atom) 

8.  Lemelin J, Hogg W, Baskerville N. Evidence to action: a tailored multifaceted approach to changing family physician practice patterns and improving preventive care. CMAJ 2001;164:757-63.pmid:11276541
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czo5OiIxNjQvNi83NTciO3M6NDoiYXRvbSI7czoxODoiL2NtYWpvLzIvMS9FMS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

9.  Feifer C, Ornstein SM, Jenkins RG, et al. The logic behind a multimethod intervention to improve adherence to clinical practice guidelines in a nationwide network of primary care practices. Eval Health Prof 2006;29:65-88.pmid:16510880
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToic3BlaHAiO3M6NToicmVzaWQiO3M6NzoiMjkvMS82NSI7czo0OiJhdG9tIjtzOjE4OiIvY21ham8vMi8xL0UxLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

10. Davis D, Goldman J, Palda VA. *Canadian Medical Association handbook on clinical practice guidelines*. Ottawa (ON): Canadian Medical Association; 2007. Available: [www.cma.ca//multimedia/CMA/Content_Images/ClinicalResources/PDF/English/CPGHandbook.pdf](http://www.cma.ca//multimedia/CMA/Content_Images/ClinicalResources/PDF/English/CPGHandbook.pdf) (accessed 2013 Nov. 5).
    
    

11. Grimshaw JM, Thomas RE, MacLennan G, et al. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess 2004;8:iii-iv, 1-72.pmid:14960256
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=15298823&link_type=MED&atom=%2Fcmajo%2F2%2F1%2FE1.atom) 

12. Grol R. Successes and failures in the implementation of evidence-based guidelines for clinical practice. Med Care 2001;39(Suppl 2):II46-54.pmid:11583121
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=11583121&link_type=MED&atom=%2Fcmajo%2F2%2F1%2FE1.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000170313600003&link_type=ISI) 

13. Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change in patients’ care. Lancet 2003;362:1225-30.pmid:14568747
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/S0140-6736(03)14546-1&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=14568747&link_type=MED&atom=%2Fcmajo%2F2%2F1%2FE1.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000185924100026&link_type=ISI) 

14. Hoffman C, Rice D, Sung HY. Persons with chronic conditions: their prevalence and costs. JAMA 1996;276:1473-9.pmid:8903258
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1001/jama.1996.03540180029029&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=8903258&link_type=MED&atom=%2Fcmajo%2F2%2F1%2FE1.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=A1996VQ77400029&link_type=ISI) 

15. Egede LE, Grubaugh AL, Ellis C. The effect of major depression on preventive care and quality of life among adults with diabetes. Gen Hosp Psychiatry 2010;32:563-9.pmid:21112446
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/j.genhosppsych.2010.08.002&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=21112446&link_type=MED&atom=%2Fcmajo%2F2%2F1%2FE1.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000285235000001&link_type=ISI) 

16. Druss BG, Rosenheck RA, Desai MM, et al. Quality of preventive medical care for patients with mental disorders. Med Care 2002;40:129-36.pmid:11802085
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1097/00005650-200202000-00007&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=11802085&link_type=MED&atom=%2Fcmajo%2F2%2F1%2FE1.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000173594900007&link_type=ISI) 

17. Carney CP, Jones LE. The influence of type and severity of mental illness on receipt of screening mammography. J Gen Intern Med 2006;21:1097-104.pmid:16970559
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1111/j.1525-1497.2006.00565.x&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=16970559&link_type=MED&atom=%2Fcmajo%2F2%2F1%2FE1.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000240291700015&link_type=ISI) 

18. Pirraglia PA, Sanyal P, Daniels D, et al. Depressive symptom burden as a barrier to screening for breast and cervical cancers. J Womens Health (Larchmt) 2004;13:731-8.pmid:15333288
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1089/jwh.2004.13.731&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=15333288&link_type=MED&atom=%2Fcmajo%2F2%2F1%2FE1.atom) 

19. Kaida A, Colman I, Janssen P. Recent Pap tests among Canadian women: Is depression a barrier to cervical cancer screening? J Womens Health (Larchmt) 2008;17:1175-81.pmid:18699731
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1089/jwh.2007.0626&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=18699731&link_type=MED&atom=%2Fcmajo%2F2%2F1%2FE1.atom) 

20. Grimshaw J, Eccles M, Thomas R, et al. Toward evidence-based quality improvement: evidence (and its limitations) of the effectiveness of guideline dissemination and implementation strategies 1966–1998. J Gen Intern Med 2006;21(Suppl 2):S14-20.pmid:16637955
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=16637955&link_type=MED&atom=%2Fcmajo%2F2%2F1%2FE1.atom) 

21. Developing trusted clinical practice guidelines. In: Eden J, Wheatley B, McNeil B, et al., editors. *Knowing what works in health care: a roadmap for the Nation*. Washington (DC): The National Academies Press; 2008:133-4.
    
    

22. Han PKH, Klabunde CN, Breen N, et al. Multiple clinical practice guidelines for breast and cervical cancer screening. Perceptions of US primary care physicians. Med Care 2011;49:139-48.pmid:21206294
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1097/MLR.0b013e318202858e&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=21206294&link_type=MED&atom=%2Fcmajo%2F2%2F1%2FE1.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000286191000005&link_type=ISI) 

23. Harrison MB, Légaré F, Graham I, et al. Adapting clinical practice guidelines to local context and assessing barriers to their use. CMAJ 2010;182:E78-84.pmid:19969563
    
    [FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czo5OiIxODIvMi9FNzgiO3M6NDoiYXRvbSI7czoxODoiL2NtYWpvLzIvMS9FMS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

24. Grunfeld E, Manca D, Moineddin R, et al., BETTER Trial Investigators. Improving chronic disease prevention and screening in primary care: results of the BETTER pragmatic cluster randomized controlled trial. BMC Fam Pract 2013;14:175 .doi:10.1186/1471-2296-14-175pmid:24252125
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1002/chp.47&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=16557505&link_type=MED&atom=%2Fcmajo%2F2%2F1%2FE1.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000236855300003&link_type=ISI) 

25. Graham ID, Logan J, Harrison MB, et al. Lost in knowledge translation: Time for a map? J Contin Educ Health Prof 2006;26:13-24.pmid:16557505
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiY2ZwIjtzOjU6InJlc2lkIjtzOjEwOiI1Ni8xMC9lMzY4IjtzOjQ6ImF0b20iO3M6MTg6Ii9jbWFqby8yLzEvRTEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

26. Goldman J, Meuser J, Rogers J, et al. Interprofessional collaboration in family health teams: an Ontario-based study. Can Fam Physician 2010;56:e368-74.pmid:20944025
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjc6IjEyLzEvMTgiO3M6NDoiYXRvbSI7czoxODoiL2NtYWpvLzIvMS9FMS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

27. AGREE Collaboration. Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Qual Saf Health Care 2003;12:18-23.pmid:12571340
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1186/1748-5908-8-49&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=23656884&link_type=MED&atom=%2Fcmajo%2F2%2F1%2FE1.atom) 

28. Harrison MB, Graham ID, van den Hoek J, et al. Guideline adaptation and implementation planning: a prospective observational study. Implement Sci 2013;8:49.pmid:23656884
    
    [FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czoxMjoiMTgzLzE1L0UxMTM1IjtzOjQ6ImF0b20iO3M6MTg6Ii9jbWFqby8yLzEvRTEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

29. Tobe SW, Stone JA, Brouwers M, et al. Harmonization of guidelines for the prevention and treatment of cardiovascular disease: the C-CHANGE initiative. CMAJ 2011;183:E1135-50.pmid:21911548
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/j.jclinepi.2012.03.013&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=23312392&link_type=MED&atom=%2Fcmajo%2F2%2F1%2FE1.atom) 

30. Andrews J, Guyatt G, Oxman D, et al. GRADE guidelines: 14. Going from evidence to recommendations: the significance and precision of recommendations. J Clin Epidemiol 2013;66:719-25.pmid:23312392
    
    [FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE3OiIzMzcvbm92MjhfMi9hMjUzMCI7czo0OiJhdG9tIjtzOjE4OiIvY21ham8vMi8xL0UxLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

31. Oxman AD, Glasziou P, William JW. What should clinicians do when faced with conflicting recommendations? BMJ 2008;337:a2530.pmid:19042938
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.7326/0003-4819-147-4-200708210-00179&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=17638714&link_type=MED&atom=%2Fcmajo%2F2%2F1%2FE1.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000248804200002&link_type=ISI) 

32. Shojania KG, Sampson M, Ansari MT, et al. How quickly do systematic reviews go out of date? A survival analysis. Ann Intern Med 2007;147:224-33.pmid:17638714
    
    

33. Parry D, Salsberg J, Macaulay AC. on behalf of Participatory Research at McGill University. Guide to researcher and knowledge-user collaboration in health research. Ottawa (ON): Canadian Institutes of Health Research; 2009; modified 2013. Available: [www.cihr-irsc.gc.ca/e/documents/Guide\_to\_Researcher\_and\_KU\_Collaboration.pdf](http://www.cihr-irsc.gc.ca/e/documents/Guide\_to\_Researcher_and_KU_Collaboration.pdf) (accessed 2013 Oct. 31).
    
    

*   © 2014 Canadian Medical Association or its licensors