Article Text
Abstract
As technological advancements continue to transform the practice of pathology, new adopters of these technologies will look to guidelines on how best to incorporate them with an eye to preserving and enhancing patient safety and diagnostic quality. Telepathology, using a variety of digital pathology modalities, has tremendous potential to achieve that goal. Pathology departments are increasingly looking to implement different digital pathology platforms, whole slide imaging (WSI) systems in particular, for a broad range of applications in patient care. WSI allows for the acquisition, management and review of completely digitised slides as would be done with a light microscope. WSI also facilitates image analysis that cannot be carried out by a pathologist using traditional microscopy. Over the last few years, the Digital Pathology Association, The Royal College of Pathologists, College of American Pathologists, Canadian Association of Pathologists, the American Telemedicine Association and the Society of Toxicologic Pathology have published guidelines for validating and implementing digital pathology systems. This review summarises, compares and contrasts these published guidelines and discusses pertinent issues that need to be addressed as the guidelines are revised in the future.
- TELEPATHOLOGY
- DIGITAL PATHOLOGY
- DIAGNOSTICS
- QUALITY ASSURANCE
Statistics from Altmetric.com
Introduction
Telepathology is a form of communication where digital pathology images and accompanying clinical information are transmitted between medical professionals for clinical purposes including intraoperative consultations/frozen sections, second opinion consultations, primary diagnosis and quality assurance.1 Interest in using digital pathology solutions for a variety of clinical applications is steadily growing in pathology laboratories around the world. Digital pathology can be broadly defined as the creation, viewing, management, sharing, analysis and interpretation of digital images of glass slides.2 An early objective for digital pathology was to use images and computer screens to simply replicate the visual interpretation diagnostic paradigm that pathologists have done using light microscopy for over 100 years. However, digital pathology is rather unique with respect to workflow as well as its technological capabilities. The continued evolution of digital pathology systems and the emergence of new image review and analysis tools bode well for a promising future for this technology. Owing to the breadth of potential applications and use cases for digital pathology, standardisation and best-practice guidelines are necessary to have sound implementation and safe application of these systems. The components of a whole slide imaging (WSI) system include a scanner with an optical microscope and digital camera for image acquisition, a computer connected to the scanner with software to create and manage image files, a server to store and host images, computer workstations with viewer software that enables image navigation and analysis across a range of magnifications and secure, stable network connectivity to link the workstation to the scanned slides.3
Recently, several guidelines and position statements for digital pathology have appeared in the published literature. These include documents from the Digital Pathology Association (DPA), Royal College of Pathologists (RCP) in the UK, College of American Pathologists (CAP), Canadian Association of Pathologists, American Telemedicine Association (ATA), Society of Toxicologic Pathology and the Royal College of Pathologists of Australasia (RCPA).1–9 Each of these has provided insight into the evolving needs of the field. They also reflect the evolving experiences with WSI and differing priorities of the governing bodies that produced them. This review summarises the key points covered in the aforementioned guidelines under the following categories:
validation of image systems prior to use in regulated clinical practice and non-clinical environments;
implementing, operating and maintaining telepathology systems used in patient care.
It is important to note that these guidelines have a life cycle which includes the need for revision to reflect changes in technology and regulatory environments, shortcomings in earlier versions identified from user experience and published evidence from experienced users and new adopters. To this end, this review identifies a number of lingering issues that need to be addressed going forward.
Validation guidelines
Validation in the context of digital pathology aims to demonstrate equivalent diagnostic performance between digital pathology systems and light microscopy and, more specifically, that the same pathologist will make the same diagnosis regardless of which modality is used when examining the same case. The validation process is complicated and requires advanced planning. As such, guidelines are important to ensure that practical yet thorough validation studies are conducted for specific intended clinical uses to minimise the chances of implementation failure. Validation is best performed on the entire digital pathology system as a whole, as opposed to assessing each component individually (ie, scanners, workstations/monitors, etc).4 Validation is necessary to ensure that the system can reliably work in an everyday clinical setting. Verification (also referred to as qualification) shows that the use of the WSI system can be replicated in the laboratory and is usually directed by predefined specifications such as those found in the manufacturer's user manual. Validation will test the system in place using routine laboratory workflow, while verification demonstrates that proper workflow can be followed. As such, verification is a step of validation.
There are three different types of validation studies for WSI systems: academic, clinical and vendor related. Academic validations are self-defined studies resulting in peer-reviewed publications that focus on a particular component of the validation process. Their findings are derived from self-defined protocols and are not necessarily generalisable. A clinical validation study documents the process and results for a specific intended clinical use. These studies follow accepted guidelines as opposed to the self-defined protocols used in academic studies. Vendor-driven validations aim to obtain approval from regulatory agencies, such as the Food and Drug Administration (FDA) in the USA, Health Canada in Canada and Conformance European (CE) mark in Europe. Such approval allows vendors to market their devices for specific intended uses including primary diagnosis. Primary diagnosis by WSI is defined as establishing a final diagnosis solely by review of digital images without relying on the glass slides and a microscope. To compare traditional diagnostic interpretation from a glass slide using light microscopy, WSI systems need to be properly validated using studies comparing WSI to glass sides. These studies ultimately measure diagnostic concordance between these two modalities (ie, digital vs glass). Many variables can affect the results of concordance studies including the familiarity of a user with WSI, the type of cases included in the study, its design, technology employed and observer (intraobserver vs interobserver) variability.
Several WSI systems have already received Health Canada approval as Class II medical devices and CE Mark status in Europe, allowing these systems to be marketed as tools to assist pathologists in routine diagnostic work including primary diagnosis.10–12 Conversely, the FDA has deemed WSI systems to be Class III devices with respect to primary diagnosis, where insufficient information exists in terms of general controls to establish safety and effectiveness, which places these systems in the highest risk category for patients. In order to obtain FDA clearance for Class III devices, comprehensive validation studies must be performed by the vendor in order to obtain premarket approval. The FDA has only recently released a draft document with non-binding recommendations concerning the technical performance assessment of WSI devices to assist vendors. The document provides the FDA's suggestions for characterising technical aspects relevant to WSI performance for intended uses as well as identifying limitations that could affect safety and effectiveness. It does not, however, provide details on the design of pivotal clinical studies supporting safety and effectiveness that will be required to obtain FDA clearance.13
The first validation guideline document on WSI and its use in a clinical healthcare environment came from the DPA in 2011. This guideline also covered the archiving and retrieval of scanned slide images.2 ,4 The RCP in the UK subsequently released guidelines for the use and practice of telepathology.5 The CAP subsequently published detailed recommendations for validating WSI for diagnostic purposes.3 These CAP validation guidelines were subsequently incorporated into the comprehensive clinical guidelines for telepathology developed by the ATA.1 The Canadian Association of Pathologists clinical guidelines on telepathology, which focuses only on WSI for intraoperative consultation/frozen section, primary diagnosis, second opinions and quality assurance practices, also adopted the CAP WSI validation guidelines.6 While the organisations mentioned above have released guidelines for clinical practice, there are also validation guidelines available for the regulated non-clinical environments.7 ,8
Clinical practice
Clinical practice, in the broadest sense, refers to any activity pertaining to patient care. The DPA, an organisation committed to the advancement of digital pathology, comprises primarily vendors along with non-vendor stakeholders. The 13 guideline statements from the DPA concerning the validation of WSI in the clinical setting are listed in box 1.
Recommendation statements from the Digital Pathology Association for validation of digital pathology in a healthcare environment4
1. All institutions or practices considering the implementation of whole slide imaging (WSI) technology for clinical diagnostic purposes must carry out their own validation.
2. Validation for each diagnostic application is necessary. WSI should not be used for clinical purposes other than the one validated.
3. The validation study should closely emulate the real-world environment.
4. Validation of the entire WSI system should be performed and it is not necessary to separately validate each individual component.
5. A pathologist adequately trained to use the WSI system must be involved in the validation process.
6. Validation of WSI systems should involve specific types of specimens and their preparations, but not specific tissues, diseases, microscopic changes or diagnosis.
7. The validation process should include a sample set of approximately 100 cases that reflect the spectrum and complexity of specimen types and diagnoses likely to be encountered during routine operation.
8. Digital and glass slides should be evaluated in random order to minimise order effect.
9. A washout period of approximately 3 weeks should occur between viewing digital and glass slides.
10. The validation process should ensure that all the material present on a glass slide, or purposefully selected area(s) on a slide to be scanned, are included in the digital image.
11. Measureable outcomes should establish diagnostic concordance between digital and glass slides for the same observer.
12. Approval of WSI systems should be limited to the conditions under which validation occurred. Revalidation is requires whenever a significant change is made to any component of the WSI system.
13. Documentation should be maintained recording the methods, measurements and final approval of validation for the WSI system to be used in the clinical laboratory.
The CAP Pathology and Laboratory Quality Center assembled a working group of non-vendor members with expertise in digital pathology to establish guidelines for validating WSI for clinical use. Evidence-based recommendations were derived from 23 publications referring to WSI and pertaining to clinical or investigative studies covering a broad range of clinical applications for WSI. Publications focusing on robotic digital imaging, technical components, image analysis and purely educational applications were excluded.3 The goals of the CAP's recommendations were to provide evidence-based guidelines by evaluating peer-reviewed literature on accuracy, concordance, sensitivity, specificity and intraobserver and interobserver variability of WSI relative to light microscopy. Interpretation and scanning time were also reviewed; however, both variables were not discussed in the final guidelines.3
The CAP guidelines consist of 12 guideline statements (table 1).3 The process started with 58 potential statements on validation with the final 12 statements being determined by the expert panel. Each of the 12 statements is accompanied by a grade describing the strength of available evidence to support the statement. Grade A evidence can be trusted to guide practice. Grade B can be trusted to guide practice in most situations. Grade C provides some support for recommendation(s) with the need for care in its application. Grade D evidence is weak and such a recommendation must be applied with caution. All statements graded as A or B were designated ‘Recommendations’. Statements with a grade C had inconclusive, conflicting or weak evidence and would benefit from additional supportive data. Grade C statements were categorised as ‘Suggestions’. Any grade D statement, meaning no to poor evidence to support the statement, was designated as an ‘Expert Consensus Opinion’ to indicate that the statement was derived from the experience of the working group.3 It is important to note that the grading was carried out by an expert evidence grader who was not a pathologist or a stakeholder in any aspect of digital pathology. The CAP WSI validation guidelines are set to be reviewed every 4 years, unless an earlier reconvening is necessary to discuss revisions based on significant new evidence.
Of note, based upon the aforementioned recommendations the CAP has recently added two items to its Laboratory Accreditation Program (LAP) checklists concerning the clinical use of WSI. Specifically, all users of the system including technologists, ancillary laboratory personnel and pathologists must be trained on the use of the system and their training must be documented. Laboratories must also document that validation was performed for each intended clinical application, although the LAP checklists do not specify how the validation is to be performed nor do they stipulate what to do with the results of the validation process.
Non-clinical environment
In 1978, the FDA released good laboratory practice regulations stating that equipment used in the generation, measurement or assessment of data shall be of appropriate design and capacity; and they should be sufficiently tested, calibrated and/or standardised.14 Thus, by these terms, a regulated environment should follow good laboratory practices. Although there is no one-size-fits-all model, each organisation should assess the extent of validating their system. Non-clinical laboratories are defined as those in which in vitro or in vivo experiments are performed prospectively on test articles in test systems under laboratory conditions to determine their safety. The term excludes studies using human subjects, clinical studies or field trials in test animals.14
As in the clinical arena, there has been increased interest in the use of digital pathology systems in regulated non-clinical environments, which has driven the development of validation guidelines specific to this setting. In 2011, the DPA was once again the first organisation to produce a guideline for validating WSI systems for use in regulated non-clinical environments.7 The essential elements of the DPA white paper were subsequently published in journal form in 2013.8 These documents provide a high-level overview on how to validate a WSI system, but do not deal with when such systems need to be validated. They provide information on the technical aspects of a WSI platform and its associated software that must be understood in order to carry out a validation process, including setting up the scanner, scanning and archiving slides and the information technology (IT) requirements involved. They also stress the point that validation is a multidisciplinary activity which requires extensive planning and appropriate documentation.
The non-clinical environment guidelines begin by discussing the importance of the system life cycle approach to validation, encompassing commissioning, operational and decommissioning phases. One of the first steps is to secure administrative approval for the intended scope of the validation project, after which a vendor assessment is advised. This communication will help identify any limitations from either the vendor or the client side. Planning should also entail proper security levels, including the establishment of user authorisation access, security of electronic records, data integrity, external data transmission security and read–write permissions. The qualification and commissioning phase includes the creation of test scripts that are step-by-step workflows (based on standard operating procedures) to qualify the installation, operation and performance of the entire system (Table 2). The system operation phase follows validation, where a change control process mediates a controlled environment to initiate the change request, assess the change, approve or decline, implement, review post-implementation data and include a process for emergency changes. The last phase, referred to as system decommissioning, is meant to ameliorate difficulties in the transition to a newer technology (ie, a more advanced WSI system) when the validated system is no longer useful for its intended use(s). Guidelines for this phase include assessing the impact of a new system, the migration process and retrieval of all data and a decommission report documenting the entire assessment of changes in hardware, software and records.
These guidelines also briefly mention validating WSI in comparison with glass slides for image analysis. They recommend first and foremost that the glass slide from which the image is obtained was prepared following good laboratory practices. A complete validation process should include all primary organ systems and basic pathological processes and should retrospectively confirm histological changes in toxicology studies. The validation data derived by image analysis should include data from both the user and the software. The guidelines recommend evaluating interoperator and intraoperator variability as well as reproducibility. The software validation should compare the automated data analysis with the manual red green blue (RGB) process using multispectral images, and the RGB images between different software programmes.
The end result should be a validation protocol that assures that the digital imaging system has been designed, deployed and tested for its intended use. This in turn provides assurance to the users that the digital image truly represents the scanned glass slide, provides quality assurance data and can provide regulatory agencies with documentation that assures that the system was properly validated. WSI systems in regulated environments should heed these guidelines, while those deployed in unregulated environments (ie, not required to follow good laboratory practices) do not necessarily need to be validated.
Telepathology guidelines
There are many potential clinical applications for telepathology, including but not limited to primary diagnosis, intraoperative consultation, secondary consultation, education and quality assurance. Aside from these uses, there are also multiple modalities by which telepathology can be practised including static or video images sent by e-mail, robotic microscopy (static and/or dynamic), real-time video microscopy, WSI and combinations of one or more of these modalities. Among the obstacles to widespread adoption of telepathology for patient care was an absence of best-practice guidelines to cover the diverse applications listed above. Various governing bodies in pathology have addressed this barrier in recent years, based on the body of emerging evidence showing that WSI is not inferior to traditional light microscopy.15–18 In 2013, the RCP and Canadian Association of Pathologists released policy statements and/or guidelines on telepathology. Soon thereafter in 2014, the ATA also released guidelines for telepathology in the USA, representing a major update of a previous version released in 1999 (figure 1).
Similarities between various clinical guidelines
The RCP, Canadian and ATA telepathology guidelines were developed with the aim of achieving safe, secure and reliable patient care based on best practices developed from current evidence and the experience of experts in the field, early adopters of the technology in particular. All three guidelines emphasise the beneficial impacts of telepathology in terms of improved patient safety, the creation of efficient pathology workflows, decreasing the need for pathologists to travel between sites or the need to physically send specimens, slides and/or paraffin blocks between centres, as well as capitalising on the promise of image analysis. Each group has included a list of definitions to delineate the scope of their recommendations. The ATA provided a more detailed glossary of terms. All three guidelines acknowledge the utility of telepathology for clinical purposes including primary diagnosis, secondary/expert consultation, intraoperative consultation, off-site diagnosis, education and quality assurance. Regarding quality management, each guideline mentions the importance of assessing discordant digital versus glass diagnoses and documenting the reasons for any discrepancies on an ongoing basis. The first version of the ATA guidelines in 1999 recommended a 10% random re-examination for quality control.19 In terms of the types of digital images themselves, all guidelines approve of still images, video and/or WSI as viable modalities for telepathology. However, more data concerning important factors such as optimal image resolution, colour balance, data transmission, storage and degree of compression need to be collected before meaningful recommendations can be developed on these aspects. Each guideline importantly discusses the impact of implementing telepathology on the laboratory staff, pathologists and clinicians. This is especially true for intraoperative consultations. By virtue of their time-sensitive nature, training is needed to firmly establish the level of pathologist and technician comfort required to support dependable laboratory workflows. Security is also a focal point in each guideline, where secure data transmission must be assessed and approved by the proper authorities. Additionally, all guidelines address location-specific legal liability issues pertaining to their individual country of practice. The RCP consulted with the National Health Service (NHS) Trust, a division of the English NHS serving a particular hospital or specialised area. The Canadian guidelines consulted the Canadian Medical Protective Association (CMPA), the national provider of malpractice insurance coverage. The CMPA provided important commentary on the difference between guidelines and standards of care in medical practice. In the USA, there are several accrediting agencies to fill that role (ie, CAP and Joint Commission). Moreover, compliance with local (state) laboratory agencies is also needed. All three guidelines report that further revisions will occur as needed in subsequent years.1 ,4 ,6 ,20
Differences between various clinical guidelines
Noteworthy differences between the RCP, Canadian and ATA guidelines include those of a contextual and core content nature. The Canadian publication is the first such telepathology guideline for Canadian pathologists, whereas the RCP and ATA guidelines are both revisions from previous versions in 2005 and 1999, respectively.1 ,4 The RCP guideline was assembled and released by a single author with input from RCP members. The ATA and Canadian guidelines were developed by appointed committees. The committee charged with establishing the Canadian guidelines elected to focus solely on WSI, while the two other groups established their guidelines for a broader range of telepathology modalities.
Of the three guidelines, the ATA was the only one to provide detailed technical specifications for viewing displays (including mobile devices), the required information for consultations (ie, accession number, patient name and block/slide identification) and recommendations regarding the transmission and storage of images. Although each group extensively discusses the multifaceted uses of telepathology, the ATA is the only guideline to discuss its use in rapid on-site assessment of cytology for adequacy of fine-needle aspiration specimens. The Canadian document is the only one which stresses being aware of the current limitations of WSI with respect to multiplanar focusing.
Although security is mentioned in each of the guidelines, none of them describes minimum security standards for uploading and sharing patient health information (PHI) along with digital slides using cloud-based services. In particular, no guidance exists for telepathology situations where the PHI and slides are being shared between pathologists in two different countries and the cloud server operated by a commercial entity is located in a third country. This is an important issue because there are already several commercial telepathology platforms that offer scalable cloud-based solutions. Cloud-based image sharing and storage (repository) platforms add value because they make images easily and rapidly accessible, often at reduced cost.21 However, it is well recognised that these services also raise security and legal issues, including external breaches (‘hacking’), data control/ownership, data loss, infrastructure failure and audit rights. When employing cloud-based services it is imperative to evaluate the privacy and security and encryption policies of the cloud providers. For telepathology transmissions beyond hospital or network firewalls and international consultations, this is a critical issue that needs further attention.
The issues of medical responsibility and malpractice are no different when pathology is practiced by telepathology. As exemplified in a policy statement on telemedicine released in December 2014 by the College of Physicians and Surgeons of Ontario in Canada, telemedicine is medicine. Existing legal and professional obligations are not altered if care is provided by telemedicine. Physicians must use their professional judgement to determine if telemedicine is appropriate in a particular circumstance each and every time its use is contemplated for patient care, consultations and referrals.22 These guiding principles should be no different the world over. The largest legal battle telepathology regulations are likely to face in the USA is that dealing with physician state licensure.23 “Telemedicine practice is not limited by geographic boundaries and easily crosses state lines, but physician licensure does not.”24 The rising demand for telemedicine in general has potential to catalyse unprecedented legal reform. With 50 states and 69 licensing jurisdictions in the USA, it seems unreasonable to have physicians obtain licences in each area they may potentially practice in. If states cooperate to institute a mutual recognition system, where predefined minimum qualifications are established to allow physicians practising telepathology legal privileges, this would alleviate substantial legalities. The alternative, federal medical licensure, is unlikely to succeed due to the arduous nature of implementation and political struggle between federal and state powers.23
Future questions and key issues still to be addressed
As mentioned above, guidelines for digital pathology and telepathology are not one-time efforts, but rather will require periodic revision to account for advances in technology, emerging applications, as well as new information in the literature. While the current versions are by no means all-inclusive or perfect, they nonetheless provide an essential starting point for those interested in implementing this technology for patient-care purposes. These documents provide guidance on many clinical and technical aspects and address key aspects related to validation, training and quality, as well as regulatory, legal and privacy issues. Moreover, they encourage widespread use of digital imaging technology in pathology, help standardise telepathology practice and promote safer practice. Future iterations of telepathology guidelines will need to address several issues in greater detail, as summarised below.
In consultation settings, particularly those involving consultations provided across international borders, a consultant may wish to defer a telepathology diagnosis to glass slide review only to find out that the glass slides cannot leave the country from which the consultation originated. Such eventualities should be explored and taken into account before any consulting relationship is established so that expectations can be clearly managed by both the referring and consultant parties.
With recent large-scale corporate security breaches involving cloud-based storage solutions, security as it relates to PHI is of prime importance. Regulations and standards governing the sharing of PHI using commercially operated cloud solutions will need to be addressed. In addition, security and privacy concerns with regard to the use of existing and future mobile devices in telepathology must be identified and addressed. This will likely take the form of new regulatory frameworks specific to all mobile telemedicine applications. Legal issues pertaining to physician licensure in the USA is a hurdle that needs to be overcome. Regulations based on Clinical Laboratory Improvement Amendments approved telepathology workstations also require attention. Compliance with international trade rules, such as safe harbour regulations, is needed to facilitate telepathology on an international scale.
Technical specifications for image resolution, scanning speeds, colour calibration, network connectivity, biometrics and minimum computing requirements need to be standardised. The issue of image retention for images used in telepathology activities most definitely requires attention as, currently, no such guidelines exist. Digital pathology images are about 10× larger than radiology images; thus, larger storage requirements are needed in digital pathology. The DPA guideline recommends redundant storage systems. The easiest solution is to store every single image used for patient care, regardless of its perceived diagnostic importance. However, this may not be practical or necessary. If selective image retention strategies are used, on what evidence should they be developed, what images need to be retained, how should they be archived and for how long? Should newer, more frequently accessed cases be stored in a manner that allows rapid access (hot storage), while older cases are deliberately archived for slower retrieval (cold storage)? How, if ever, should digital slide data be decommissioned? Are all data to be purged or will only diagnostic sections of images be saved for potentially longer time periods? Digital Imaging and Communications in Medicine Working Group 26 has proposed a Picture Archiving and Communications System for digital slides.25 Having a universally accepted DICOM standard will promote vendor scanner interoperability, or the so-called ‘plug and play’ systems. These would be most desirable for telepathology networks involving WSI platforms from multiple vendors. Additionally, many WSI systems do not use a conventional 40× objective; rather, they digitally magnify a 20× image to a 40× resolution. This may impact diagnostic and prognostic criteria including the counting of mitoses based on 10 high-power fields. Finally, specific studies are needed to better assess the accuracy and clinical outcome of measurements and automated assessments of immunohistochemistry made using image analysis. Undoubtedly, there will be many other questions to answer as telepathology becomes the mainstream modality for diagnosis.
Practice points
Guidelines are necessary to define the current practice and future scope of telepathology. As more users begin to adopt digital imaging technology in accordance with current guidelines, these guidelines will evolve.
Validation establishes user confidence and provides essential documentation that the technology has been prudently implemented for each intended use. It helps assure us that digital pathology is being used accurately, efficiently and consistently.
Regulated environments may change, but will always follow good laboratory practices for which guidelines provide direction. As anticipated updates in technology advance quickly, new applications will continuously emerge. This will keep telepathology standardisation in constant evolution, which in turn will help promote safer medical practice.
References
Footnotes
Contributors All authors have contributed to this invited review article in terms of its organisation, content and final editing.
Competing interests AJE serves as a consultant for clinical trial design for Omnyx. The other authors report no competing interests.
Provenance and peer review Commissioned; internally peer reviewed.