Abstract
Background: Improving global maternal, newborn, child and adolescent health (MNCAH) is a top development priority in Canada, as shown by the $6.35 billion in pledges toward the Muskoka Initiative since 2010. To guide Canadian research investments, we aimed to systematically identify a set of implementation research priorities for MNCAH in low- and middle-income countries.
Methods: We adapted the Child Health and Nutrition Research Initiative method. We scanned the Child Health and Nutrition Research Initiative literature and extracted research questions pertaining to delivery of interventions, inviting Canadian experts on MNCAH to generate additional questions. The experts scored a combined list of 97 questions against 5 criteria: answerability, feasibility, deliverability, impact and effect on equity. These questions were ranked using a research priority score, and the average expert agreement score was calculated for each question.
Results: The overall research priority score ranged from 40.14 to 89.25, with a median of 71.84. The average expert agreement scores ranged from 0.51 to 0.82, with a median of 0.64. Highly-ranked research questions varied across the life course and focused on improving detection and care-seeking for childhood illnesses, overcoming barriers to intervention uptake and delivery, effectively implementing human resources and mobile technology, and increasing coverage among at-risk populations. Children were the most represented target population and most questions pertained to interventions delivered at the household or community level.
Interpretation: Investing in implementation research is critical to achieving the Sustainable Development Goal of ensuring health and well-being for all. The proposed research agenda is expected to drive action and Canadian research investments to improve MNCAH.
The United Nations' Millennium Development Goals, a set of interrelated targets adopted by world leaders in 2000, catalyzed political commitment toward improving child survival and maternal health. Goals 4 and 5 called for a two-thirds reduction in mortality among children less than 5 years of age and a three-quarters reduction in maternal mortality between 1990 and 2015, respectively.1 Five years before the goals came to a close, the Muskoka Initiative was launched at the G8 summit to intensify efforts toward improving maternal, newborn and child health in low- and middle-income countries, with Canada investing $2.85 billion to reduce the burden of disease, improve nutrition and strengthen health systems in areas with the greatest need.2 Although there have been substantial gains in reducing global maternal and child mortality, progress has been insufficient to achieve the Millennium Development Goals' targets.3,4 Unacceptably high numbers of women and children are still dying every year, largely due to conditions that could have been prevented or treated if existing cost-effective interventions were made universally available.5 Currently, there is insufficient knowledge on how to effectively implement proven affordable interventions in resource-limited settings.6 Over the past 5 years, Canadian funding through the Muskoka Initiative has focused on scaling up interventions to improve maternal, newborn and child health; however, investments in implementation research have been limited.7 If we are to achieve high, sustainable and equitable coverage of life-saving interventions, addressing this research gap is essential.
The year 2015 marked the beginning of a new global framework - the Sustainable Development Goals - and an additional Canadian pledge of $3.5 billion toward the Muskoka Initiative.2,8 These renewed commitments toward improving maternal, newborn and child health present an opportunity to address the unfinished agenda of the Millennium Development Goals, bridge the gap in implementation research and expand efforts to address the neglected area of adolescent health. We undertook an expert consensus process using standardized methods to systematically identify the top research priorities on the implementation of maternal, newborn, child and adolescent health (MNCAH) interventions in low- and middle-income countries with the aim of guiding Canadian research investments over the next 15 years - the timeline of the Sustainable Development Goal targets for 2030.
Methods
Study design
We adapted the Child Health and Nutrition Research Initiative method.9 This method was designed to assist policy-makers and investors to identify research gaps and examine the potential risks and benefits of investing in different research options. This systematic and transparent approach is the most frequently used method of health research prioritization since 2001, followed by the Delphi, James Lind Alliance and Combined Approach Matrix methods.10 It has now been applied to a wide range of relevant MNCAH topics, including, but not limited to, birth asphyxia, childhood pneumonia and diarrhea, and adolescent sexual and reproductive health.11-14 The method involves 5 stages: (i) defining the context and criteria for priority-setting with input from investors and policy-makers; (ii) listing and scoring research investment options by technical experts using the proposed criteria; (iii) weighting the criteria according to wider societal values with input from other stakeholders; (iv) calculating research priority scores and average expert agreement scores; and (v) ranking research priorities according to research priority scores. An initial stage was added to the present study, in which the steering committee extracted implementation-focused research priorities from the existing Child Health and Nutrition Research Initiative literature before inviting input from technical experts. This literature review was conducted to build on the foundation of existing Child Health and Nutrition Research Initiative studies, allowing our expert group to review and contribute to a set of research questions previously identified as priority areas.
Setting
This study aimed to inform various stakeholders, including the Canadian Partnership for Women and Children's Health (CanWaCH) community and key Canadian donors and researchers, about research investment options that are expected to improve the implementation of MNCAH interventions in low- and middle-income countries. This process of developing and ranking research questions also allowed researchers to systematically approve a common research agenda and develop a consensus on priorities. The timeline of 15 years was set to coincide with the Sustainable Development Goal targets.
In selecting the criteria against which to evaluate the proposed research questions, the steering committee modified the Child Health and Nutrition Research Initiative criteria from previous exercises to better reflect the context of implementation.9,15 The 5 criteria selected were answerability by research, research feasibility, deliverability, impact and effect on equity. Table 1 shows the 3 specific subquestions evaluated under each criterion.
Literature review
Through an initial literature search, a team member (O.M.) screened published Child Health and Nutrition Research Initiative studies, identifying all research questions potentially relevant to implementation. Two researchers (R.S. and M.B.) then screened this list for research questions explicitly pertaining to implementation, defined as the delivery of interventions (i.e., policies, programs or individual practices) and the translation of research evidence into improved health policy and practice.16,17 These questions were then classified into 4 domains: description (epidemiology), discovery (new interventions), development (improving existing interventions) and delivery (health policy and implementation). We selected the highest-ranked delivery questions from each article, which was followed by an updated search for studies through Google Scholar, PubMed and Web of Science using the keywords: "CHNRI" and "Child Health and Nutrition Research Initiative." The 3 highest-ranked delivery questions from each additional article were then added to the list. Finally, the research questions were mapped by theme and position on the continuum of care, removing duplicates and questions within over-represented health areas.
Technical consultation
This study drew upon the expertise of researchers, clinicians and implementing partners from various institutions across Canada. We specifically engaged Canadian-based voices in global health given the focus of the study on guiding Canadian research investment. Experts either volunteered to participate at the CanWaCH meeting in November 2014, were identified from their affiliations with the Coalition of Centres in Global Child Health or SickKids Centre for Global Child Health, or were selected for their known expertise in the field of MNCAH.
Experts were asked to individually review the research questions identified from the literature and propose additional questions. We then thematically organized the combined list of questions by position on the continuum of care, removing overlapping options and questions outside the scope of the exercise. The remaining questions were organized into a marking tool for scoring.
Experts scored each proposed research question against these 5 predetermined criteria, each with 3 subquestions:
Answerable by research: likelihood that the research question can be answered ethically.
Research feasibility: likelihood that there are sufficient resources and time to carry out the research.
Deliverability: likelihood that the research can result in a deliverable, affordable and sustainable implementation strategy.
Impact: likelihood that the results from this research will fill crucial knowledge gaps and shape future planning in implementation research.
Effect on equity: likelihood that the implementation strategy will reduce inequity.
We asked experts to score 1 for yes, 0 for no and 0.5 if they were informed but undecided. If the experts did not feel sufficiently knowledgeable to answer a particular question, they were instructed to leave the cell blank. These blank cells were not included in the calculation of scores.
Analysis
The relative importance of the scoring criteria may vary among stakeholders. In a previous exercise,18 a wide range of stakeholders was polled to weight the Child Health and Nutrition Research Initiative criteria; however, before scoring, the steering committee decided not to assign weights for this exercise. We weighed all 5 criteria equally in the analysis, as we felt they were of equal importance.
The research priority score and average expert agreement scores were calculated for each research question. The research priority score is the mean of the scores across the 5 criteria, expressed as a percentage. The average expert agreement score is the average proportion of scorers who chose the mode (most common score) across the 15 subquestions asked. The average expert agreement score was calculated as follows:
where q is a question that experts are being asked to evaluate competing research investment options, ranging from 1 to 15, n is the number of scorers who provided the most frequent response and N is the total number of scorers. A Pearson correlation coefficient was calculated to examine the association between research priority score and average expert agreement scores.
Results
Figure 1 shows how the steering committee identified relevant research questions from existing Child Health and Nutrition Research Initiative studies.11-15,19-35 The final list from the literature contained 45 research questions. Thirty-eight experts were then formally invited by email to participate. Six experts volunteered to participate at the CanWaCH meeting, and 32 experts were identified from their affiliations or known expertise. Participants' expertise ranged across the continuum of care, representing knowledge of all 4 target populations.
Identification of priority research questions from the Child Health and Nutrition Research Initiative literature. Note: CHW = community health worker, IMCI = integrated management of childhood illness, PMTCT = Prevention of mother-to-child transmission of HIV.
Experts individually reviewed the 45 questions from the literature, with 24 experts proposing 71 additional questions. The steering committee then thematically organized the 116 questions by position on the continuum of care, removing overlapping options. Ninety-seven remaining questions were organized into a marking tool, and 20 experts returned completed scoring sheets.
Table 2 shows the research questions with a rounded research priority score of 80 or more, and Appendix 1 (available at www.cmajopen.ca/content/5/1/E82/suppl/DC1) shows the complete list of ranks and scores for all 97 questions. Both tables present the perceived likelihood that each research question will comply with each of the 5 priority-setting criteria. Research priority scores ranged from 40.14 to 89.25, with a median of 71.84. The average expert agreement scores ranged from 0.51 to 0.82, with a median of 0.64. Similar to past Child Health and Nutrition Research Initiative exercises, average expert agreement showed a strong positive association with research priority score, as evidenced by a Pearson correlation coefficient of 0.783 (p < 0.0001). This finding suggests that there was strong agreement among experts about what were considered priority research questions.
The top 15 research questions varied across the continuum of care. Children were the most represented target population, with 6 out of the 15 questions pertaining to child health. Although there were highly-ranked questions about maternal (questions 10 and 13) and newborn health (questions 5, 8, 9 and 14), there were no top-ranked questions that explicitly mentioned adolescents. The highest ranking for an adolescent health question was number 19 - "What factors facilitate uptake, retention and adherence to antiretroviral therapy and minimize HIV treatment failure among adolescents."
A wide range of topics was covered in the top 15 research questions. Diarrhea was the most frequently mentioned health condition, with 2 questions about oral rehydration solution and 1 question about detection and management of dehydration in children with diarrhea.
Research questions varied in specificity. For example, broad questions like "what are effective delivery strategies to ensure that the most vulnerable individuals receive critical reproductive, maternal, newborn child and adolescent health services?" were scored alongside specific questions like "Can a simplified neonatal resuscitation program delivered by trained health workers reduce deaths due to intrapartum events and complications and birth asphyxia?" Both broad and specific questions were ranked in the top and bottom 15 research questions, suggesting that no bias existed against the kind of question asked.
Interpretation
We engaged a diverse group of Canadian experts with knowledge and experience across the continuum of care. The modified Child Health and Nutrition Research Initiative approach we used offered greater transparency and replicability than Delphi or other consultative processes.10 The systematic ranking of proposed research priorities against predetermined criteria also made apparent the strengths and weaknesses of competing research investment options.
The comprehensive list of research priorities generated by this exercise addressed leading causes of newborn, child and maternal death, including intrapartum events and complications, diarrhea and barriers to facility births.36 The 3 most important coverage gaps identified by the Countdown to 2015 for Maternal, Newborn, and Child Survival group (Countdown) were present in our list of priorities; they included family planning, interventions addressing newborn mortality and case management of childhood diseases.36 Countdown reported that there are relatively smaller inequities in coverage for interventions that are delivered close to home.37 Our list of priorities was consistent with this finding, because most highly ranked research questions pertained to interventions that could be implemented at the household or community level. Seven of the 15 top-ranked questions originated from the Child Health and Nutrition Research Initiative literature and 2 of these questions (questions 7 and 12) came from Child Health and Nutrition Research Initiatives explicitly focused on implementation, indicating strong agreement between our expert group and the existing literature.15,30
Limitations
Although the Child Health and Nutrition Research Initiative method represents a systematic attempt to address the challenges inherent in the complex process of research investment priority setting, the approach is not without limitations. Yoshida and colleagues conducted an analysis of the Child Health and Nutrition Research Initiative methodology,38 examining the concordance among top-ranking research priorities as sample size increases from 15 to 90. They found that a high degree of reproducibility of top-ranking research priorities was achieved with 45-55 experts, suggesting that our relatively small sample of 20 scorers may be a limitation. However, it should be noted that they still found an appreciable degree of reproducibility with a sample of only 15 people. In addition, it is possible that there were sound research options that were not included in the list of questions generated by the literature and experts. These options, therefore, could not have been scored and identified as priorities. Proposed research questions and their subsequent scores were also limited to the opinions of the experts involved in the exercise. In an effort to minimize response bias, we employed a comprehensive process of identifying experts with relevant knowledge and experience. Although this process was nonsystematic, we deliberately invited only Canadian experts given the focus of the exercise on informing Canadian research investments. The predetermined Child Health and Nutrition Research Initiative criteria also ensured that questions were anonymously scored against a transparent and standardized set of values, thus eliminating the advantage of more eloquent speakers advocating for their own research agenda. An additional potential limitation was that experts might have scored questions about patient populations or health conditions outside of their area of expertise. To avoid inaccurate scores, experts were instructed to leave the cell blank when they did not feel sufficiently knowledgeable to answer a particular question.
The top 15 research questions varied across the continuum of care, but there were no highly-ranked questions that explicitly mentioned adolescents. The steering committee noted that existing Child Health and Nutrition Research Initiative literature on adolescent health was limited. In light of this gap, we made an effort to recruit adolescent health experts to propose additional research questions and provide scores. Adolescent health is an emerging global health priority and although this population was not explicitly mentioned among the top-ranked research questions, it should be noted that questions pertaining to maternal and reproductive health could also be relevant to adolescents, especially in low- and middle-income countries. Moreover, since the completion of this study, there has been a Child Health and Nutrition Research Initiative study published, focusing on 8 areas of adolescent health.39
Conclusion
Current investments in health research predominantly target diseases prevalent in high-income countries and tend to favour basic science research. If progress toward improving MNCAH is to be made by 2030, improving implementation is crucial to maximizing the impact of existing interventions and reducing inequity. The research gaps identified through this priority-setting exercise cannot be addressed in isolation; they must be integrated with the measurement and accountability agenda, so as to ensure there is timely data on the quality and coverage of effective interventions. The proposed research agenda is expected to be a valuable tool in guiding research investments that could drive substantial improvement in health outcomes by 2030. We call upon the Canadian community of donors, researchers, policy-makers and program managers to support the translation of these recommendations into appropriate and transparent funding opportunities.
Supplemental information
For reviewer comments and the original submission of this manuscript, please see www.cmajopen.ca/content/5/1/E82/suppl/DC1
Acknowledgements
The authors acknowledge James Blanchard, Andrea Hunter and Rachel Spitzer for their participation; Kerri Wazny for providing technical support; and the Canadian Partnership for Women and Children's Health (CanWaCH) for their support in this collaborative effort. CanWaCH is a collaboration of more than 80 Canadian organizations that are working to improve maternal, newborn, child and adolescent health in low- and middle-income countries.
Footnotes
Competing interests: No authors declare any conflicts of interest related to this work.
Contributors: Renee Sharma designed the study protocol, coordinated the study, mapped research priority questions, conducted the analysis and drafted the initial manuscript. Matthew Buccioni designed the study protocol, mapped research priority questions, assisted with coordination, critically reviewed and revised the manuscript. Michelle Gaffey designed the study protocol, provided feedback throughout the Child Health and Nutrition Research Initiative process and critically reviewed and revised the manuscript. Omair Mansoor conducted the initial literature review of existing research priorities and assisted with the Canadian Partnership for Women and Children's Health consultation. Helen Scott coordinated the Canadian Partnership for Women and Children's Health consultation, assisted with study design and critically reviewed and revised the manuscript. Zulfiqar Bhutta conceptualized and designed the study, obtained funding, oversaw the Child Health and Nutrition Research Initiative process and critically reviewed and revised the manuscript. Members of the Canadian Expert Group on Maternal, Newborn, Child and Adolescent Health generated and scored priority research questions, provided feedback during the study and critically reviewed and revised the manuscript. All of the authors approved the final version of the article to be published and agreed to act as guarantors of the results.
Funding: This project was funded by the Centre for Global Child Health at the Hospital for Sick Children and the Canadian Partnership for Women and Children's Health.
Members of the Canadian Expert Group on Maternal, Newborn, Child and Adolescent Health: Lisa Avery MD MIH (Centre for Global Public Health, University of Manitoba, Winnipeg, Man.), Diego G. Bassani PhD (Centre for Global Child Health, The Hospital for Sick Children, Toronto, Ont.), Alan D. Bocking MD (Department of Obstetrics and Gynecology, University of Toronto; Lunenfeld-Tanenbaum Research Institute, Toronto, Ont.), Kevin J. Chan MD MPH (Department of Pediatrics, Memorial University; Children and Women's Health Program, Eastern Health, St. John's, NL), Maryanne Crockett MD MPH (Centre for Global Public Health, University of Manitoba, Winnipeg, Man.), Stephen B. Freedman MDCM MSc (Alberta Children's Hospital and Research Institute, University of Calgary, Alta.), Miriam Kaufman BSN MD (Department of Adolescent Medicine, The Hospital for Sick Children, Toronto, Ont.), Niranjan Kissoon MBBS (British Columbia Children's Hospital; Department of Pediatrics, University of British Columbia, Vancouver, BC), Eugene A. Krupa PhD Med (Catalyst Research & Development Inc.; University of Alberta, Edmonton, Alta.), Charles P. Larson MD (Department of Pediatrics, University of British Columbia, Vancouver, BC), Vanessa Mathews-Hanna MPH (World Renew, Burlington, Ont.), Douglas McMillan MD (Dalhousie University; IWK Health Centre, Halifax, NS), Shaun Morris MD MPH (Centre for Global Child Health, The Hospital for Sick Children; Department of Pediatrics, University of Toronto, Toronto, Ont.), Zubia Mumtaz MBBS PhD (University of Alberta, Edmonton, Alta.), Gregor Reid PhD DrHS (Canadian Centre for Human Microbiome and Probiotic Research, Lawson Health Research Institute; Schulich School of Medicine & Dentistry, Western University, London, Ont.), Selim Rashed MPH MD (Maisoneuve Rosmont Hospital, University of Montreal; Montreal University Health Center, McGill University, Montréal, Que.), Prakeshkumar S. Shah MBBS MD (Department of Pediatrics and Institute of Health Policy, Management, and Evaluation, University of Toronto, Toronto, Ont.), Nalini Singhal MD (Alberta Children's Hospital and Research Institute and Department of Paediatrics, University of Calgary, Calgary, Alta.), Alan Talens MD MPH (World Renew, Burlington, Ont.); Stanley H. Zlotkin MD PhD (Centre for Global Child Health, The Hospital for Sick Children; Department of Pediatrics, University of Toronto, Toronto, Ont.).
References
- Copyright 2017, Joule Inc. or its licensors
In this issue
Article tools
Related Articles
Cited By...
- No citing articles found.