Abstract
Background: Retinoblastoma is a childhood cancer of the eye that can have lifelong effects on patients and families. The purpose of this study was for people affected by retinoblastoma, clinicians and researchers to jointly determine the top 10 retinoblastoma research priorities in Canada.
Methods: An adaptation of the James Lind Alliance Priority Setting Partnership (PSP) methodology was employed. People were invited to participate in any stage of the priority-setting process if they were a resident of Canada, and were a patient with retinoblastoma (or a family member or friend of someone diagnosed with retinoblastoma) or a clinician or researcher interested in retinoblastoma. Patients were full partners in study design and implementation, and result dissemination, through involvement in a national working group (1 patient and 9 nonpatients) and steering committee (4 patients and 11 nonpatients). In phase 1 of the study, participants responded to an online survey that asked, “What questions about retinoblastoma would you like to see answered by research?” In phase 2, the steering committee reviewed and refined the list of survey responses and decided on a list of 30 questions to be ranked by means of the nominal group technique in phase 3, a priority-setting workshop.
Results: In total, 175 retinoblastoma research questions were suggested by 59 survey participants (38 patients and 21 nonpatients). The categories with the greatest number of questions were genetics and molecular (45 [25.7%]), second cancer (29 [16.6%]) and psychosocial (27 [15.4%]). The top 10 questions as ranked by the workshop participants (10 patients and 10 nonpatients) fell into 7 categories: second cancer (2 questions), follow-up (2), psychosocial (2), treatment (1), diagnosis (1), global health (1) and miscellaneous (1). The early diagnosis of retinoblastoma was identified as the top retinoblastoma research priority in Canada.
Interpretation: The list of priorities will serve as a resource for advocacy groups, research teams and funding agencies that focus on retinoblastoma. The inclusion of researchers as participants was an adaptation of the James Lind Alliance PSP methodology and enriched the research prioritization process.
Plain language summary: A national study of patients with retinoblastoma (childhood eye cancer) and their family members and friends, clinicians and researchers was conducted in Canada to answer the question “What are the top 10 retinoblastoma research priorities in Canada?” The method used was an adaptation of the James Lind Alliance Priority Setting Partnership process, commonly used in such joint priority-setting initiatives. The top priority identified related to early diagnosis of retinoblastoma. Advocacy groups, research teams and funding agencies are encouraged to align their practices with the identified retinoblastoma research priorities.
Research priority setting identifies the research most relevant to and valued by stakeholders. This is necessary given that most funded research does not reflect the priorities of patients and clinicians, potentially reducing its impact.1,2 There are several established methods for research priority setting; the James Lind Alliance Priority Setting Partnership (PSP) method3 is arguably the most popular. The James Lind Alliance PSP method involves patients and clinicians equally in setting a top 10 list of research priorities.
Retinoblastoma is a cancer of the infant retina usually caused by a biallelic RB1 gene mutation.4 About 45% of patients with retinoblastoma have the heritable form, meaning they carry a constitutional RB1 mutation that confers risk of second cancers later in life and can be passed on to offspring. Each year, 8000 children are newly diagnosed with retinoblastoma globally, about 24 of whom are in Canada.4
The retinoblastoma research community in Canada practises patient engagement; however, a formal process to ensure equitable, diverse and sustainable inclusion was established only recently.5,6 The National Retinoblastoma Patient Engagement Strategy was formed in Canada in 2016. The Canadian Retinoblastoma Research Advisory Board (CRRAB), a national multidisciplinary group composed of patients, caregivers, clinicians, researchers and other stakeholders, leads the strategy.5 The aim of the present study, was to identify the top 10 retinoblastoma research priorities in Canada, an early objective collectively agreed on by the CRRAB.
Methods
The purpose of this study was for patients with retinoblastoma, clinicians and researchers to jointly determine the top 10 retinoblastoma research priorities in Canada using an adaptation of the James Lind Alliance PSP method. We use the term “patient” to refer to people with personal experience of a health issue and their informal caregivers, including family and friends.7
Working group and steering committee
All CRRAB members were invited to participate in a working group, which led study design from December 2016 to October 2017 (Table 1). The working group recruited additional members to form the CRRAB Priority Setting Steering Committee in October 2017 (Table 1). The working group included 1 patient. When recruiting for the steering committee, we aimed to represent all stakeholder groups and encompass a range of geographic regions. The steering committee included 4 patients.
Demographic characteristics of the Canadian Retinoblastoma Research Advisory Board working group, Priority Setting Steering Committee and workshop participants
Participants
People were invited to participate in any stage of the priority-setting process if they were a resident of Canada, and were a patient with retinoblastoma (i.e., diagnosed with retinoblastoma or a family member or friend of someone diagnosed with retinoblastoma) or a clinician or researcher interested in retinoblastoma.
The inclusion of researchers is a noteworthy change to the James Lind Alliance PSP method, which traditionally uncovers research priorities only of patients and clinicians.8 In this method, nonclinician researchers are excluded from the priority-setting process, although they may sit on the steering committee.3 Like the James Lind Alliance, we are committed to the inclusion of patients and clinicians in setting research priorities. However, researchers have unique expertise in new research directions, study design and implementation. We also expected that having researchers present would result in accelerated uptake of the identified research priorities.9 With this in mind, we agreed that the activities in the James Lind Alliance PSP methodology could be conducted carefully alongside researchers and set out to establish an equitable process, with a 1:1 ratio of patients and nonpatients (clinicians and researchers).
Design
The study design was adapted from the 3-phase James Lind Alliance PSP method (Figure 1). The study consisted of an online survey (phase 1), an interim ranking exercise (phase 2) and an in-person priority-setting workshop (phase 3).
Overview of James Lind Alliance Priority Setting Partnership method.
One deviation from the James Lind Alliance PSP method was that, as part of phase 1, research questions were not identified through a literature search. Given the relatively small body of retinoblastoma literature, and that steering committee members have been involved in writing seminal retinoblastoma reviews4,10–12 and clinical care guidelines,13,14 participation of steering committee members as respondents in phase 1 ensured that questions identified in the current retinoblastoma literature were put forward for consideration in phase 2.
Phase 1: online survey
Drawing from other publicly available James Lind Alliance PSP surveys, the working group developed and tested a survey using REDCap tools for electronic data capture hosted at The Hospital for Sick Children, Toronto.15 The voluntary, open online survey was available on the senior author’s (H.D.) laboratory website from Oct. 3 to Nov. 12, 2017. We intended to offer the survey online for no less than 1 month; the final survey duration was dictated by pragmatic considerations including research ethics board and administrative timelines, steering committee meetings and workshop scheduling.
A convenience sample was recruited. Members of the CRRAB shared recruitment materials in their eye clinics and on social media, and existing networks such as the Canadian Retinoblastoma Research Registry, the Canadian National Retinoblastoma Tumor Board and the Canadian Association of Genetic Counsellors shared the materials widely (Appendix 1, available at www.cmajopen.ca/content/8/2/E420/suppl/DC1). Snowball sampling was encouraged among the CRRAB community.
After providing implied informed consent, participants provided their demographic information on the first page of the survey (using adaptive questioning, where “prefer not to say” options were offered). The second page of the survey then asked, “What questions about retinoblastoma would you like to see answered by research?”
Survey responses were processed by K.F. and H.D. as follows. Submissions were screened for duplicate entries based on question topic and respondent identifiers (i.e., participant name, email address) and then categorized according to a coding taxonomy developed by H.D. and K.F.: awareness, diagnosis, family planning, follow-up, genetics and molecular, global health, psychosocial, second cancer, treatment, trilateral retinoblastoma, vision and miscellaneous. When a single submission included more than 1 question, the individual questions were separated and reworded (with narrative text removed, if applicable) to result in concise questions. Duplicate questions were combined, questions that were out of scope were removed, and questions known to be answered by existing systematic reviews, clinical care guidelines or individual studies were removed.
Phase 2: steering committee interim ranking
The steering committee reviewed the phase 1 results and completed a second round of processing to produce a refined list of questions. During this processing, based on their respective areas of expertise, members of the steering committee were asked to identify any questions that were answered by existing systematic reviews, clinical care guidelines or individual studies. Such questions were reviewed by the steering committee and removed if consensus was reached on removal. The steering committee then conducted an interim ranking of the refined list of questions.
Since the steering committee had an imbalance of patient to nonpatient members (4:11), to ensure fair weighting, a steering committee patient rank and a steering committee nonpatient rank were calculated individually and then combined to produce an adjusted steering committee interim ranking. By consensus, the steering committee decided on a list of 30 questions to be ranked at the priority-setting workshop.
Phase 3: priority-setting workshop
The priority-setting workshop, held on Dec. 9, 2017, followed the established process.3 A chairperson with experience facilitating other priority-setting workshops was hired to lead the workshop with 2 facilitators (K.F., M.G.). The steering committee was committed to an equitable workshop, employing the nominal group technique to include diverse perspectives, with a 1:1 ratio of patients to nonpatients.
Participants were separated into 3 facilitator-led groups with representation from patients and nonpatients. Each group had a set of 30 cards displaying a question and contextual information from the online survey (e.g., frequency, quotes). Using the cards, each group organized the questions from highest to lowest priority. An aggregate ranking was computed by the chairperson and facilitators, using the individual group rankings, and presented to all participants. Participants were then assigned to 3 new groups with representation from patients and nonpatients, again led by a facilitator. The first aggregate ranking was reviewed and revised by each group. Groups noted their revised ranking, and this was used to compute a second aggregate ranking. The chairperson led all participants in a review of the second aggregate ranking. Ties and suggested refinements were discussed and agreed on by a vote, with a majority needed to break a tie or adopt a refinement.
Patient engagement
The study was designed collaboratively by the working group during 8 video-conference meetings. The survey results were processed by the steering committee during 2 video-conference meetings. Where requested, one-on-one meetings were coordinated with patient partners in the working group and on the steering committee to provide additional support and to accommodate scheduling conflicts. Patients involved in the workshop were offered modest travel stipends. All members of the working group and steering committee were invited to provide feedback on the manuscript, which was coauthored by a patient partner (S.A.J., the patient in the working group). At The Hospital for Sick Children, a parent-in-research role was created on the senior author’s research team.16 This person, alongside members of CRRAB, is leading a national knowledge translation project disseminating the findings of this study.
Statistical analysis
We used summary statistics to characterize the online survey participants.
Ethics approval
Research ethics board approval was obtained from The Hospital for Sick Children.
Results
Phase 1: online survey
Online surveys were completed by 59 people, 38 patients (64%) and 21 nonpatients (36%) (Table 2).
Demographic characteristics of online survey respondents
Respondents were primarily female (50 [85%]) and Ontario residents (34 [58%]). Patients were parents (28 [74%]) or survivors (10 [26%]); the mean patient age was 38 (standard deviation 8) years. Most patients (26 [68%]) had bilateral retinoblastoma. Nonpatients were clinicians (16 [76%]), clinician-scientists (3 [14%]) or researchers (2 [10%]).
In total, 175 questions were suggested (Table 3). The categories with the greatest number of questions were genetics and molecular (45 [25.7%]), second cancer (29 [16.6%]) and psychosocial (27 [15.4%]). Questions most commonly suggested by patients were genetics and molecular (31/114 [27.2%]), second cancer (22/114 [19.3%]) and psychosocial (15/114 [13.2%]). Questions most commonly suggested by nonpatients were treatment (15/61 [25%]), genetics and molecular (14/61 [23%]) and psychosocial (12/61 [20%]). All questions concerning awareness and vision were suggested by patients, whereas all questions concerning global health and most (15/21 [71%]) of those concerning treatment were suggested by nonpatients. After survey responses were processed, 46 questions were removed (Figure 2), resulting in 129 questions across all 12 categories that were presented to the steering committee.
Number of questions from online survey (phase 1) and number presented to the steering committee, by category
Flow diagram showing priority-setting process for retinoblastoma research. Note: CRRAB = Canadian Retinoblastoma Research Advisory Board.
Phase 2: steering committee interim ranking
The steering committee generated a refined list of 96 questions subject to steering committee ranking (Figure 2). The 30 questions ranked by the steering committee as having the highest research priority and 9 additional questions that fell outside the top 30 but were suggested by more than 1 survey respondent were further considered by the steering committee for the priority-setting workshop. A final list of 30 questions was agreed on by the steering committee for consideration at the workshop. Questions from all categories except trilateral retinoblastoma were included in the final list.
Phase 3: priority-setting workshop
There were 20 workshop participants, 10 patients and 10 nonpatients (Table 1). All patients were affected by heritable retinoblastoma. Parents participating included those with young children currently undergoing retinoblastoma treatment and parents of adult survivors.
There were similarities in the first and second aggregate rankings (Table 4). Six of the top 10 questions in the second aggregate ranking were also in the top 10 in the first aggregate ranking. Similarly, 6 of the bottom 10 questions in the second aggregate ranking were also in the bottom 10 in the first aggregate ranking. For the final ranking, 4 ties in the second aggregate ranking were decided by vote. Four additional questions were reorganized in response to suggestions of the group.
Top 30 retinoblastoma research priorities
The top-ranking question was how to increase early diagnosis of retinoblastoma (Table 4). No question from the awareness, family planning, genetics and molecular, or vision categories ranked among the top 10.
Interpretation
This study brought together patients affected by retinoblastoma, clinicians and researchers to jointly identify the top 10 retinoblastoma research priorities in Canada. The top research priority identified was how to increase early diagnosis of retinoblastoma. Four of the top 10 research priorities addressed second cancer or follow-up, 2 addressed psychosocial considerations, 1 addressed retinoblastoma treatment, 1 was concerned with how to provide a detailed pathway of care to patients with retinoblastoma and their families, and 1 (the 10th-highest priority) was concerned with how optimal retinoblastoma care can be delivered in low-resource settings.
Early diagnosis of retinoblastoma increases the possibility of favourable outcomes (vision and survival).13 In its position statement, the Canadian Paediatric Society recommends dilated eye examinations for children, including inspection of the red reflex, when newborn to 3 months of age, and at 6–12 months of age.17 Failure of visualization or abnormalities of the reflex are indications for urgent referral to an ophthalmologist. 17 Yet the mean age at retinoblastoma diagnosis in Canada is 27 months for unilateral disease and 15 months for bilateral disease.10 This suggests that there is poor adherence to current vision screening guidelines, or the guidelines are insufficient for detecting retinoblastoma given the variable timing and topography of tumour development in the infant retina.
There is a paucity of reliable information about second cancer risk for the more recent cohort of survivors of heritable retinoblastoma, who have received new types of first-line therapy, including intra-arterial chemotherapy.18 Consequentially, no standardized plan exists for adult follow-up of heritable retinoblastoma survivors.
There are discordant findings about the psychosocial well-being of patients with retinoblastoma, and these outcomes have yet to be examined in Canada.19–23
The fact that only 1 of the 10 top research priorities addressed retinoblastoma treatment is likely a consequence of the survival rate in Canada: more than 95% of children with retinoblastoma survive, with favourable outcomes.13
The research priority regarding how to provide a detailed pathway of care to patients with retinoblastoma and their families aligns with growing evidence that supports patient-centred care24 and may have been partly motivated by plans to deploy DEPICT HEALTH, a point-of-care retinoblastoma database, globally.25
The priority concerned with how optimal retinoblastoma care can be delivered in low-resource settings was the only priority in the top 10 suggested by a researcher alone. This supports our assertion that there is value in including researchers in research-prioritization exercises.
Notably, although genetics and molecular questions were suggested in the survey more often than any other category, the top 10 research priorities did not include a question from this category. The proportion of genetics and molecular questions suggested in the survey may not reflect their relative importance. Rather, our research has shown that patients have challenges accessing information about and understanding retinoblastoma genetics and thus, may have developed more questions in this domain.26
To our knowledge, the only other research-priority–setting endeavour that mentioned retinoblastoma was the UK Sight Loss and Vision PSP.27 As part of that study, research questions for childhood-onset eye disorders were prioritized, and the question ranked eighth was “How can retinoblastoma be identified, prevented and treated in children?”
Limitations
Our study has limitations that warrant consideration. Patient survey participants may not have been representative of the broader Canadian retinoblastoma community. Only 38 patients participated in the survey, largely female parents from Ontario. Similarly, the majority of patient participants in the workshop (and to a lesser extent, the survey) were affected by heritable retinoblastoma. Given the lifelong implications of the disease, it is not surprising that those with heritable retinoblastoma are particularly incentivized to participate in research. This imbalance may have biased the study results toward survivorship and long-term effects of retinoblastoma.
Two additional deviations from the customary James Lind Alliance PSP method are important to note. First, in phase 1, we were precluded from using only systematic reviews and guidelines to verify that the questions were unanswered; this approach is particularly relevant when a James Lind Alliance PSP is identifying unanswered research questions about disease diagnosis or treatment. Given the broad scope of retinoblastoma research questions being prioritized and the familiarity of the steering committee with the limited number of retinoblastoma systematic reviews (i.e., 1 Cochrane review) and guidelines, we felt it prudent that the steering committee verify whether questions were unanswered using their full breadth of knowledge. Second, when the questions were reworded in phase 1 of the study, the PICO (patient or population, intervention, comparator or control, and outcome) structure was not adopted consistently. Feedback from the workshop participants suggested that they would have valued the opportunity to participate in the question wording or have an orientation to each question with an outline of the meaning and relevant background.
Lessons learned from patient engagement
The CRRAB5 promotes research that meets the Canadian Institutes of Health Research’s definition of successful patient engagement.7 By design, this study valued experiential knowledge as evidence. We partnered with patients in a range of roles including leadership opportunities, working as true partners toward a shared goal. Opportunities to develop capacities of all stakeholders in a safe environment of mutual respect were offered informally. This collaborative approach was reinforced by feedback from previous patient partners.16
We have commented elsewhere on our challenges with patient-oriented research.16,28 Funders seldom support nonresearch activities necessary for sustainable patient-oriented research (e.g., establishing a social media presence, maintaining registries), which is taxing on researchers and patient partner volunteers. Travel reimbursements and stipends for patient partners are embedded in all of the senior author’s research grants, yet it is difficult to include these costs in smaller grants, and, these are not always allowable costs. One approach to addressing these issues is the paid Parent in Research role within the senior author’s research team,16 which was funded initially by a 1-year Canadian Institutes of Health Research Strategy for Patient-Oriented Research Collaboration grant. Establishing fixed funding for this role is a challenge.
Conclusion
We achieved consensus on the top 10 retinoblastoma research priorities in Canada using an adaptation of the James Lind Alliance PSP method. These results are intended to align the retinoblastoma research agenda in Canada to the collective values of patients, clinicians and researchers. By galvanizing research that all stakeholders buy into, the top 10 retinoblastoma research priorities can enhance the quality of retinoblastoma research results and accelerate uptake into practice. Our modification — to include researchers as participants — was a valuable element in identifying a research priority that was subsequently ranked in the top 10 by all participants. By sharing the final research priorities broadly, we expect that the top 10 list will serve as a resource for advocacy groups, research teams and funding agencies that focus on retinoblastoma.
Acknowledgements
The authors acknowledge the participants in the working group, the Canadian Retinoblastoma Research Advisory Board Priority Setting Steering Committee and the priority-setting workshop, and Dr. Karen Born, who served as the workshop chairperson.
Footnotes
Competing interests: None declared.
This article has been peer reviewed.
Contributors: Kaitlyn Flegg and Helen Dimaras conceptualized and designed the study and interpreted the data. Maxwell Gelkopf and Sarah Johnson assisted with the study design. Kaitlyn Flegg and Maxwell Gelkopf collected the data. Helen Dimaras supervised data collection and analysis. Kaitlyn Flegg analyzed the data and drafted the manuscript. Maxwell Gelkopf, Sarah Johnson and Helen Dimaras revised the manuscript critically for important intellectual content. All of the authors approved the final version to be submitted and agreed to be accountable for all aspects of the work.
Data sharing: Aggregate data may be accessed for relevant research ethics board approved research by emailing the corresponding author, Helen Dimaras, helen.dimaras{at}sickkids.ca.
Supplemental information: For reviewer comments and the original submission of this manuscript, please see www.cmajopen.ca/content/8/2/E420/suppl/DC1.
Funding: This work was supported by the SickKids Garron Family Cancer Center Conference Fund, and by Meeting, Planning and Dissemination grant 152652 from the Institute of Cancer Research, Canadian Institutes of Health Research.
References
- Copyright 2020, Joule Inc. or its licensors
In this issue
Article tools
Related Articles
Cited By...
- No citing articles found.