Abstract
Background Research shows that guidelines featuring implementation tools (GItools) are more likely to be used than those without GItools, however few guidelines offer GItools and guidance on developing GItools is lacking. The objective of this study was to identify common processes and considerations for developing GItools.
Methods Interviews were conducted with developers of 4 types of GItools (implementation, patient engagement, point-of-care decision-making and evaluation) accompanying guidelines on various topics created in 2008 or later identified in the National Guideline Clearinghouse. Participants were asked to describe the GItool development process and related considerations. A descriptive qualitative approach was used to collect and analyze data.
Results Interviews were conducted with 26 GItool developers in 9 countries. Participants largely agreed on 11 broad steps, each with several tasks and considerations. Response variations identified issues lacking uniform approaches that may require further research including timing of GItool development relative to guideline development; decisions about GItool type, format and content; and whether and how to engage stakeholders. Although developers possessed few dedicated resources, they relied on partnerships to develop, implement and evaluate GItools.
Interpretation GItool developers employed fairly uniform and rigorous processes for developing GItools. By supporting GItool development, the GItool methods identified here may improve guideline implementation and use.
Guidelines are recognized as the foundation of efforts to improve health care because they synthesize all of the available evidence on effective management of a given condition, offer recommendations to inform health care planning and delivery, and are the basis for measuring and improving performance and outcomes.1,2 Unfortunately, guideline use is less than ideal as shown by population-based studies from several countries.3–5 This is because of multiple, interacting contextual factors such as patient noncompliance, provider skepticism about guideline relevance for individual patients, lack of institutional infrastructure, and limited system-level resources and coordination.6 Even when interventions are used to implement guidelines, they have a small or inconsistent impact, perhaps because contextual challenges are not easily addressed.7 New approaches are urgently needed to support guideline implementation and use. Experts have referred to this as an evidence-based medicine crisis, and have called for developers to provide users with tools such as algorithms or decision-making aids with which to implement guidelines.8,9
Incorporating implementation tools with guidelines may be a feasible approach for improving guideline implementation and use. Evaluations of guidelines showed that they were high in quality for scope and purpose, stakeholder involvement, rigor of development and clarity of presentation, but consistently lacking in applicability.10–14 This refers to guideline implementation tools (GItools) such as training material (e.g., workshop slides, self-directed learning kits), guideline summaries or algorithms, patient information, or guidance for evaluation (e.g., quality indicators, audit instructions). Focus groups found that health professionals were frustrated and uncertain about how to implement guidelines.15 GItools have been empirically associated with guideline use. Two trials and a systematic review of 23 studies based on 143 recommendations showed that compliance was greater for guidelines that offered implementation tools compared with guidelines that were not accompanied by implementation tools.16–18 We interviewed 30 guideline developers from 7 countries who said that their target users had requested GItools, but they required direction for developing GItools.19 Existing guideline development instructional manuals lack details on how to develop GItools.20,21 The purpose of this study was to identify common processes and considerations for developing GItools.
Methods
Approach
GItool developers were interviewed to learn how they developed GItools. Qualitative research is useful for revealing tacit knowledge.22 A descriptive qualitative approach was used along with various strategies to optimize rigor.23,24 Consolidated criteria for reporting qualitative research25 guided the reporting of study findings. Conduct of this study was approved by the University Health Network Research Ethics Board (12-0091-AE). Participants gave informed consent before being interviewed.
Sampling and recruitment
The National Guideline Clearinghouse (US Department of Health and Human Services, Agency for Healthcare Research and Quality; www.guideline.gov) was searched for English-language guidelines published in 2008 or later on the management of prevalent conditions that affect both men and women (e.g., arthritis, asthma, cancer, depression, diabetes, heart failure, stroke) and topics relevant to guideline developers who were collaborators on this research (e.g., chiropractic, dermatology, neurology, nephrology). Clearinghouse records for individual guidelines indicate if they have implementation tools but do not specify type. For those guidelines with implementation tools, the guideline documents and the websites of corresponding developers were searched for GItools of the following 4 types: implementation (assessment of adoption barriers, selecting and tailoring implementation strategies), patient engagement (information for patients or the public, shared decision-making guides; self-management resources), point-of-care decision-making (paper or electronic templates or instruments that guide or document care delivery) and evaluation (performance measures or quality indicators, instructions for measurement). Our prior research showed that these were the most common types of GItools that accompanied guidelines.26
Purposive sampling was used to recruit developers of these GItools who represented, in nonmutually exclusive fashion, a variety of countries, types of organizations (e.g., specialty societies or associations, foundations, government agencies) and guideline topics. Eligible participants were those responsible for guideline development or implementation at those organizations. Contact information was identified on organization websites and each was invited by email to participate. A reminder email was sent to nonresponders at 2 and 4 weeks after the initial email was sent. We aimed to recruit a minimum of 5 GItool developers representing each of the 4 types of GItools. Detailed information from representative rather than a large number of cases is needed in qualitative research. As is common in qualitative research, sampling was concurrent with data collection and analysis and proceeded until no further unique themes emerged from successive interviews (saturation). This was determined through discussion between 2 independent reviewers at various times during the iterative data analysis process until both deemed that the most recent interviews produced consistent information. Ultimately, 26 interviews were conducted.
Data collection
Interviews of about 1 hour were conducted via telephone by a trained research assistant who was coached by the principal investigator. A semistructured interview guide was developed to ask participants how GItools were planned, developed, evaluated and implemented; the resources required; challenges they experienced and suggestions for information or guidance that would support GItool development (Appendix 1, available at www.cmajopen.ca/content/3/1/E127/suppl/DC1). Discussion of the first 2 transcripts by the research assistant and principal investigator improved the wording and flow of questions. For example, initial responses revealed planning steps in advance of development and implementation after development, therefore, subsequent participants were asked specifically about planning and implementation. Interviews were audiorecorded and transcribed. Data were collected from June 4, 2013, to Sept. 27, 2013. The researchers had no relationships with the participants, or personal goals.
Data analysis
Themes were identified inductively and iteratively using constant comparative technique and NVivo9 (www.qsrinternational.com).27–29Several transcripts were read independently by the research assistant and principal investigator to identify and define themes (first-level coding). They met to review the themes and their application, and to develop a refined coding scheme. This was applied independently by the research assistant and principal investigator to several more transcripts to add or merge thematic codes (second-level coding) and to generate a more developed version of the coding scheme (Appendix 2, available at www.cmajopen.ca/content/3/1/E127/suppl/DC1). Using this coding scheme, all remaining transcripts were analyzed independently by the research associate and principal investigator. They then met to achieve consensus on the final coding scheme by discussion. To enable interpretation, and identify similarities or differences in participant responses, all quotes were tabulated by theme and type of GItool. As is typical of qualitative research, themes with conflicting responses were identified and discussed because these themes provide insight on challenging issues that warrant interventions or ongoing research. Next, relational analysis was used to integrate data describing GItool development processes.30 By using this technique, all data were perused and each unique finding was tallied. Each instance of a unique process or consideration of GItool development revealed by participants was tallied and compiled by GItool development phase.
Results
A total of 330 guidelines on relevant clinical topics were identified initially. Of these, 124 possessed GItools in the 4 categories of interest (point-of-care decision making [DM], implementation [IM], evaluation [EV], patient engagement [PE]). Of 124 GItool developers invited to participate, 95 either declined or did not respond, 29 consented and 26 were interviewed (Table 1). The GItools about which participants were interviewed addressed a wide range of conditions (Table 2). They were published between 2008 and 2013 and most (19 out of 26, [73.1%]) were supplemental to the guideline itself.
Participants described several phases of GItool development, multiple tasks in each phase (Box 1) and the different types of staff involved in the process (Box 2). They also articulated a number of resource considerations that highlight a lack of dedicated funding to support GItool development (Box 3). Despite the varied types of GItools for a range of conditions, the phases, tasks and considerations were relatively similar across participants. For example, GItools were implemented most commonly by email distribution, on a website, at conferences and publication in journals (Box 4). All unique tasks and considerations were tallied and organized by phase (Table 3).
Participant descriptions of contributors involved in the process of GItool development
Participant descriptions of GItool dissemination and implementation strategies
Participants offered conflicting responses in 4 areas that are discussed here because they represent variations in practice that require ongoing investigation and resolution (see illustrative quotes in Box 5). Notably, these response variations did not appear to be associated with the type of GItool.
Participant views about GItool development processes or considerations that were conflicting
Participants discussed variable processes in terms of the timing of GItool development relative to guideline development. In this study, 1 GItool was pre-existing and integrated with the guideline. Others were developed concurrently with guideline development, but most were planned and developed following guideline development. However, several participants recognized the growing view that implementation planning, including GItools, must be considered early in the guideline development process to better accommodate the needs of target users (Box 5).
The prompt for developing GItools and, by association, the reason for choosing to develop a particular type of GItool, varied across participants. The type of GItool was sometimes informed by a perceived or measured health care problem (Box 5). Other organizations developed GItools for every guideline but tended to develop the same type for all guidelines (Box 5). Therefore, it may not be clear if GItools are warranted for all guidelines and which types of GItools best support the implementation of different types of guidelines.
The format of GItools and views about ideal GItool format varied widely across participants. Most GItools were supplemental products rather than information included in the body or appendices of the guidelines (Box 5). The GItools discussed by participants were paper-based, electronic or available in both formats, and content was organized in a variety of ways including bullets, tables, and flow charts or algorithms. Overall, participants recommended that GItools be simple and concise, offered in multiple formats to suit different users and include graphics. However, the variation in GItool formats and recommendations to issue GItools in all possible formats suggest that the format most conducive to the use and effectiveness of different types of GItools is unknown.
Processes for assembling and reviewing GItool content also varied across participants; in particular, the degree to which target users were engaged. Content was often drawn from existing guidelines or by reviewing published research. Decisions about content were often made by committee members through solicited feedback or structured means such as decision matrix, rating or consensus technique. Although these decision-making techniques appear robust, few participants said that target users were consulted about content or engaged in the decision-making process. Several participants recognized the mounting consensus that target users must be meaningfully involved in the development of guidelines and associated products to ensure that they are relevant and useful (Box 5).
Interpretation
Main findings
This study compiled GItool development methods based on the experiences of GItool developers. They articulated fairly uniform methods despite representing different countries, types of organizations, guideline topics and types of GItools. Processes used by participants to develop GItools appeared to be rigorous because they reflected GItool Framework elements considered important by the international guideline community, such as searching for evidence, engaging stakeholders, pilot-testing and evaluation.31 Variations were identified in the timing of GItool development relative to guideline development; decisions about GItool type, format and content; and whether and how to engage target users.
Explanation and comparison with other studies
In this study, GItool developers had few dedicated resources; instead, they relied on partnerships with professional organizations or researchers to develop or evaluate GItools. This approach also addresses the need to meaningfully engage stakeholders. The principles and practices of action research or integrated knowledge translation could be used by others to establish partnerships for the purpose of planning, developing, implementing and evaluating GItools.32,33 Still, an overall conundrum remains to be resolved. Target users require support to adopt and apply guideline recommendations.15–18 Implementation is not considered the responsibility of guideline developers but of target users.2 Most guideline developers are already challenged by resources and time-to-issue guidelines without the additional burden of concurrently having to work with partners and stakeholders to generate GItools that support user implementation of guidelines. Perhaps the answer lies in resources offered by knowledge-sharing networks. Both the Guidelines International Network (www.g-i-n.net) and the Agency for Health Care Research and Quality (www.guideline.gov) provide access to GItools in guideline repositories. Although not all guidelines have GItools, existing GItools could be tailored to suit different guidelines or contexts by either developers or users. In previous research, we engaged international guideline developers in identifying the ideal characteristics of GItools, defined as any information within or accompanying guidelines that helps users to implement the recommendations.31 This resulted in a 12-item GItool Framework that could be used to assess the merits of existing GItools, and GItool development methods identified here would enable adaptation.
Schünemann and colleagues reviewed the content of guideline development manuals to generate a guideline development planning framework. Although the framework included considerations for dissemination and implementation, this was restricted to only 4 points, one of which was to develop or adapt implementation tools. The limited detail on guideline implementation was noted by Schünemann and colleagues as a gap. This work addresses that gap by providing more detailed guidance for the development of GItools.21
Limitations
These findings may be limited by transferability. We attempted to mitigate this through purposive sampling of participants who varied on several characteristics that may have influenced their views. Although we achieved thematic saturation, and there were no trends by sampling characteristics, participant responses reflected the GItools they described. Further research may confirm if these findings are true of developers in other settings or of different types of GItools.
Conclusion and implications for practice and future research
The process of GItool development appeared to be similar across different types of GItools. The methods identified here may provide a starting point for the development of GItools. However, the application of the findings may be limited because most guideline developers have few resources by which to disseminate or implement guidelines, which may restrict their capacity to develop GItools.
Further research is needed to investigate how to optimize the development process and the type, format and content of GItools. It is hoped that these findings may support the development of more, high-quality GItools to facilitate the implementation of clinical guidelines into practice.
Supplemental information
For reviewer comments and the original version of this manuscript, please see www.cmajopen.ca/content/3/1/E127/suppl/DC1
Footnotes
-
Competing interests: None declared.
-
Contributors: Anna Gagliardi conceived the study, acquired funding, coordinated all aspects of its conduct and prepared this manuscript. All of the authors contributed to study planning and data collection and interpretation, reviewed and approved the final version submitted for publication, and agreed to act as guarantors of the work.
References
- © 2015, 8872147 Canada Inc. or its licensors
In this issue
Article tools
Related Articles
Cited By...
- No citing articles found.