Abstract
Background: Several expert groups, including the United States Preventive Services Task Force and the Canadian Task Force on Preventive Health Care, have recently examined or are currently examining whether primary care physicians should screen asymptomatic adults for hepatitis C virus (HCV) infection. To inform decision-making on HCV screening, we performed a systematic review of the accuracy of antibody tests compared with other immunoassays and RNA detection for screening asymptomatic adults for HCV infection in Canada.
Methods: MEDLINE and Embase databases were searched from 1990 to 2016; resulting citations were uploaded into DistillerSR and independently screened by 2 reviewers. Original research studies, systematic reviews and meta-analyses were eligible for inclusion. At least 80% of the study population had to be asymptomatic, nonpregnant, treatment-naïve adults with unknown liver enzyme values and unknown HCV status. Risk of bias was assessed with the use of the Quality Assessment of Diagnostic Accuracy Studies version 2 (QUADAS-2) tool; the quality of the body of evidence was assessed by means of GRADE (Grading of Recommendations Assessment, Development and Evaluation) methodology.
Results: Of 1537 articles identified, 81 underwent full-text review, and 9 studies met the inclusion criteria. Compared with RNA detection, the sensitivity of the third-generation enzyme-linked immunosorbent assay was variable (61.0%-81.8%), and its specificity was high (97.5%-99.7%). As expected, there were more false-positive results when comparing antibody tests to RNA detection than to other immunoassays. Our GRADE assessment suggested that there was a high concern for risk of bias, particularly verification bias, and substantial inconsistency between studies in terms of their design.
Interpretation: More research is needed to better characterize the accuracy of antibody tests used to screen for HCV infection in the general population. Jurisdictions that recently adopted birth cohort screening for HCV infection are encouraged to evaluate and report on the accuracy of HCV screening tests and screening benefits and harms. PROSPERO registration: no. CRD42016039710.
The incidence of hepatitis C virus (HCV) infection in Canada has declined in recent years.1,2 The population prevalence of chronic HCV infection in this country is estimated at 0.64%-0.71%,2 about half that in the United States.3 An estimated 21%-44% of Canadians with chronic HCV infection are unaware of their infection.1,2 In low-prevalence countries such as Canada and the United Kingdom, the approach to prevention and control of HCV infection has focused on case-finding,4,5 i.e., testing people with risk factors for the infection, such as intravenous drug users and refugees from endemic countries. The recent development of effective but costly treatment for chronic hepatitis C6 has led some to reevaluate the evidence for and against population screening for HCV infection.7 In 2013, the US Preventive Services Task Force revised its 2004 recommendation against screening asymptomatic adults for HCV infection;8 it now recommends one-time screening for all adults born between 1945 and 1965.7 The Canadian Task Force on Preventive Health Care is examining whether primary care physicians should screen asymptomatic adults for HCV infection.9
Guidance from the World Health Organization10,11 and the UK National Screening Committee12 on when screening should be performed emphasizes the fundamental importance of having a "safe, valid, and reliable" screening test. Screening for HCV infection typically relies on antibody testing. Because antibodies may persist13 after HCV infection is spontaneously cleared (which occurs in about 25% of those infected14), antibody testing cannot discriminate current from resolved infections, which leads to false-positive results.15 False-positive results can also occur when other antibodies interact nonspecifically with the test.16 False-positive results can cause harm (e.g., through labelling and anxiety). People with a positive screening result typically undergo further testing, which has resource implications and may carry additional inherent risk. To inform decision-making on screening for chronic HCV infection in Canada, we performed a systematic review of the evidence on the accuracy of antibody tests used to screen asymptomatic adults for HCV infection.
Methods
Research question
Our objective was to carry out a systematic review to estimate the accuracy of antibody tests used in Canada to screen for HCV infection among asymptomatic, nonpregnant, treatment-naïve adults with unknown liver enzyme values. We also sought to assess the accuracy of the 2-step HCV screening procedure (i.e., the combination of the initial and confirmatory tests) currently used in this country. The research protocol to answer this question was registered with PROSPERO (no. CRD42016039710).
Laboratory tests for HCV
Laboratory tests for HCV infection can be divided into 3 categories based on what they detect: 1) antibodies to HCV, 2) antibodies to HCV and HCV core antigen (i.e., antigen-antibody tests) and 3) HCV RNA.16 Antibody tests include the third-generation enzyme-linked immunosorbent assay (ELISA), chemiluminescent immunoassay (CLIA), electrochemiluminescent immunoassay (ECLIA), chemiluminescent microparticle immunoassay (CMIA) and microparticle enzyme immunoassay (MEIA). Each commercial antibody testing kit uses slightly different sets of HCV antigens to bind and detect host antibodies (Table 1). Antigen-antibody tests include fourth-generation ELISAs; because they detect antibodies to HCV, they have the same issues with false-positive results as third-generation ELISAs but higher sensitivity because they also detect HCV antigen. Molecular techniques that detect HCV RNA, such as polymerase chain reaction and nucleic acid amplification testing, are recommended as confirmatory tests,18 even though immunocompromised patients and those undergoing hemodialysis may have false-negative results.16 RNA-based testing can detect HCV before antibodies are produced (early infection) and can differentiate between current and resolved infection (i.e., fewer false-positive results). There is no perfect reference standard for HCV infection, and the US Centers for Disease Control and Prevention have recommended a testing strategy in which the initial test is an antibody test and the confirmatory test may be either RNA detection or serological methods similar to the index test.18
Environmental scan of laboratory testing for HCV
To determine the scope of our systematic review, we first performed an environmental scan of laboratory testing for HCV in Canada. Given that provincial and territorial laboratories generally perform most HCV testing,19 one author (G.C.) searched the grey literature on the websites of all provincial and territorial laboratories and the Canadian AIDS Treatment Information Exchange, and contacted the laboratories by email in February 2016, on behalf of the Canadian Task Force for Preventive Health Care, to obtain information about the tests and testing sequences used to diagnose HCV infection in Canada (Appendix 1, available at www.cmajopen.ca/content/4/4/E737/suppl/DC1). In most of the laboratories, the initial test used was an antibody test, such as the CLIA (British Columbia, Northwest Territory and Saskatchewan), CMIA (Alberta, Manitoba, New Brunswick, Newfoundland and Labrador, Nova Scotia and Ontario) or MEIA (Quebec). Confirmatory testing was typically performed by means of another immunoassay, such as the CMIA (British Columbia and Saskatchewan), CLIA (Ontario), fourth-generation ELISA (Alberta) or recombinant immunoblot assay (Manitoba and Quebec) rather than the polymerase chain reaction (New Brunswick, Newfoundland and Labrador, and Nova Scotia). Use of a second immunoassay to confirm HCV infection,20 rather than RNA detection, is also aligned with current US Centers for Disease Control and Prevention recommendations.18
Literature search strategy
We developed the literature search strategy with the help of a librarian at the University of Toronto Gerstein Science Information Centre (Appendix 2, available at www.cmajopen.ca/content/4/4/E737/suppl/DC1). We systematically searched Ovid MEDLINE (1946-2016), Ovid MEDLINE In-Process and Other Non-Indexed Citations, and Ovid Embase using both controlled vocabulary and keywords. Because the oldest immunoassay of interest (ELISA version 3.0) was first marketed in 1993,21 retrieval was limited to articles published in English or French between Jan. 1, 1990 and May 6, 2016. We excluded conference abstracts from the search results.
Selection criteria
Original research studies, systematic reviews and meta-analyses were eligible for inclusion. At least 80% of the study population had to be asymptomatic, nonpregnant, treatment-naïve adults with unknown liver enzyme values and unknown HCV status (e.g., general population, blood donors). We excluded studies involving high-risk groups such as patients undergoing hemodialysis, transplant/transfusion recipients, intravenous drug users and patients coinfected with other blood-borne infections, as well as blood bank specimens that had previously tested negative for HCV and specimen panels. The index test had to be the CLIA, ECLIA, CMIA, MEIA or ELISA version 3.0; rapid tests, tests performed on specimens other than blood (e.g., saliva), and sero- or genotyping tests were excluded. Within a given study, the reference test had to be different from the index test. In the absence of a perfect reference standard, 2 different sets of reference tests were eligible for inclusion: 1) inferior serological reference tests (CMIA, CLIA, ECLIA, MEIA, third- and fourth-generationELISA or recombinant immunoblot assay), commonly used as confirmatory tests in Canada, and 2) superior RNA-based reference tests (polymerase chain reaction or nucleic acid amplification testing). The reference test had to be applied to some participants with a positive index test result as well as some of those with a negative index test result, so that a 2 × 2 table could be filled and sensitivity and/or specificity estimated. The setting had to resemble primary care (e.g., blood donation centre, population-based screening); hospital-based specialty clinics and inpatient hospital settings were excluded.
Selection methodUsing DistillerSR software, 2 reviewers (G.C. and J.C.) independently screened all titles and abstracts using the predetermined selection criteria. Potentially relevant articles were retrieved, and the reviewers screened all full-text articles using the same selection criteria. Disagreements were resolved through discussion.
Data extraction
One reviewer (G.C.) extracted data on study characteristics and findings from each included study into Tables 2-4, as appropriate. Raw data were extracted to create 2 × 2 tables of index test(s) compared with reference test(s) for each study. The second reviewer (J.C.) verified the accuracy and completeness of the data extraction. Disagreements were resolved through discussion. We obtained additional data not reported in the published article from the authors of 1 included study.23
Risk of bias assessment
One reviewer (G.C.), who has previous experience conducting validation studies33,34 and performing statistical adjustment for verification bias,35 assessed the risk of bias and applicability of each included study using the Quality Assessment of Diagnostic Accuracy Studies version 2 (QUADAS-2) tool. QUADAS-2 focuses on 4 domains: patient selection, index test, reference test, and patient flow and timing of testing.36 The second reviewer (J.C.) verified the accuracy and completeness of the assessment. Disagreements were resolved through discussion.
Statistical analysis
We estimated sensitivity, specificity, positive predictive value, negative predictive value, false-positive rate, false-negative rate and 95% confidence intervals (CIs) using the raw data (2 × 2 tables) extracted from the included studies. For studies that suffered from verification bias as a result of the sampling strategy used, we performed statistical adjustment of sensitivity and specificity estimates (and therefore of the false-positive and false-negative rates).31 Given the small number of included studies and the heterogeneity of index test-reference test pairings between studies, quantitative synthesis was not considered appropriate.
Assessment of quality of body of evidence using GRADE methodology
Whereas studies using inferior serological reference tests reflect current laboratory practice in several provinces and territories, they likely underestimate the true number of false-positive results. For this reason, we further restricted the body of evidence to studies in which the index test was an antibody test (reflecting current laboratory practice) and the reference test detected RNA. Two reviewers (G.C. and J.C.) independently used GRADE (Grading of Recommendations Assessment, Development and Evaluation) methodology for diagnostic testing accuracy studies32,37-39 to assess the body of evidence; disagreements between reviewers were resolved through discussion. The GRADE criteria evaluate the evidence in terms of study design, risk of bias, indirectness, imprecision and publication bias.39
Results
Literature search results and characteristics of included studies
We identified 1537 studies, of which 81 underwent full-text review; 9 studies were included in our systematic review (Figure 1). A list of the excluded studies and each study's reason for exclusion is available in Appendix 3 (available at www.cmajopen.ca/content/4/X/E737/suppl/DC1). We did not identify any eligible studies that evaluated the 2-step HCV screening procedure. Included studies (Table 2) were conducted in countries with low,15,27,28 moderate25,29 and high22-24,26 HCV prevalence.30
Accuracy of immunoassays compared with other immunoassays
The sample size of studies comparing 2 immunoassays ranged from 106 to 5208 (Table 3). For antibody tests compared with other antibody tests, sensitivity ranged from 70.4% to 99.5% and specificity from 98.7% to 99.8%. For an antigen-antibody test compared with an antibody test, sensitivity ranged from 52.4% to 95.6%, and specificity was over 99%.
Accuracy of immunoassays compared with RNA detection
The sample size of studies comparing an antibody test to RNA detection ranged from 106 to 21 115 (Table 4). For antibody tests compared with RNA detection, sensitivity ranged from 61.0% to 81.8% and specificity ranged from 97.5% to 99.7%. As expected, the antigen-antibody test performed better against RNA detection than the antibody tests did; its sensitivity was 90.2% and its specificity was 99.8%. Also as expected, the number of false-positive results was higher and the positive predictive value was lower when comparing antibody tests to RNA detection than to other antibody tests.
Assessment of risk of bias with QUADAS-2
With respect to patient selection, only 2 studies involved routine HCV screening of the general population;23,27 the other 7 studies involved blood donor screening15,22,24-26,28,29 (QUADAS-2 assessment, Appendix 4, available at www.cmajopen.ca/content/4/X/E737/suppl/DC1). The 9 included studies reported on 11 different index test-reference test pairings: 5 studies compared antibody tests to other antibody tests,15,22,23,27,28 2 studies compared antigen-antibody tests to antibody tests,24,25 3 studies compared antibody tests to RNA detection22,23,26 (Amina Benouda, Hôpital universitaire Cheikh-Zaid, Rabat, Morocco: personal communication, 2016), and 1 study compared an antigen-antibody test to RNA detection.29 With respect to patient flow and timing of testing, 7 studies applied the index and reference tests to all samples in parallel.15,22,24,26-29 Two studies had verification bias as a result of applying the reference test to a larger proportion of samples that had tested positive on the index test relative to those that had tested negative on the index test.23,25
Assessment of quality of body of evidence with GRADE methodology
We considered 3 studies comparing antibody tests to RNA detection for inclusion in the body of evidence.22,23,26 One of the 3 studies23 was excluded because, as a result of there being no false-negative results, the correction of the sensitivity estimate for verification bias was overly conservative40 and the uncorrected sensitivity was too biased to be meaningful. Another study22 was excluded because, based on its results, the study prevalence of HCV was very high, at 23.6%; this suggested that either the study population was not reflective of the general population or a case-control design was used (in either case, it did not meet our inclusion criteria). The quality of evidence in the remaining study26 was assessed as very low (Table 5); this study reported a sensitivity of 81.8% (95% CI 59.0%-100%) and a specificity of 99.7% (95% CI 99.6%-99.8%). Assuming an HCV seroprevalence rate of 0.96%, as in the general Canadian population,2 instead of the rate of 0.1% among the 17 840 blood donors in the study,38 the positive predictive value would be 72.7% (95% CI 66.2%-78.8%), and the negative predictive value would be 99.8% (95% CI 99.8%-99.9%). Applying this study's findings to 1000 people drawn from the general Canadian population, we would expect 8 (95% CI 6-10) true-positive results, 987 (95% CI 986-988) true-negative results, 3 (95% CI 2-4) false-positive results and 2 (95% CI 0-4) false-negative results (Table 5).
Interpretation
In our systematic review of the evidence on the accuracy of antibody tests compared with other immunoassays and RNA detection for screening asymptomatic adults for HCV infection, we found that the sensitivity of antibody tests was highly variable (52.4%-99.5%) and the specificity was high (97.5%-99.8%). The lack of a perfect reference test for HCV infection raises concerns that these estimates are biased. In particular, when an inferior serological reference test that shares the same risk of false-positive results as the index test is used, the specificity could be overestimated. As expected, we found that there were more false-positive results when comparing antibody tests to superior RNA-based reference tests than to inferior serological reference tests. Bias correction of the specificity estimate could make the difference even greater. This finding highlights a potential problem with using an inferior serological reference test as a confirmatory test18,20 when screening the general population for HCV infection. Our assessment of the body of evidence using GRADE methodology led us to focus on a single "least-biased" study,26 in which the sensitivity of the ELISA version 3.0 compared with nucleic acid amplification testing was 81.8% (95% CI 59.0%-100%) and the specificity 99.7% (95% CI 99.6%-99.8%). Assuming an HCV seroprevalence rate of 0.96%, as in the general Canadian population,2 the positive predictive value would be 72.7% (95% CI 66.2%-78.8%); in other words, 1 of every 4 positive ELISA version 3.0 test results would be a false-positive result.
The only other systematic review of the accuracy of HCV antibody testing was performed in the context of the US Preventive Services Task Force's 2004 recommendation on HCV screening.8 However, the populations of the studies included in that review (i.e., patients undergoing hemodialysis, patients with histologically verified hepatitis, patients admitted to hospital with suspected acute/chronic hepatitis, blood donors with persistently elevated liver enzyme values and blood donors who previously screened positive for HCV) did not reflect the general population; therefore, those findings are not directly comparable to ours.21 When the US Preventive Services Task Force revised its recommendation on HCV screening in 2013,3 it did not reassess the accuracy of HCV screening tests, despite the introduction of new immunoassays (i.e., CLIA, ECLIA, CMIA, MEIA) since its 2004 systematic review. Similarly, when the World Health Organization published its HCV screening guideline in 2014,41 it did not reassess screening test accuracy, instead citing a 2001 report42 on simple/rapid test accuracy and a 2001 report43 on the accuracy of the ELISA version 3.0 or later immunoassay, both involving blood panels not reflective of the general population. Because antibody tests have not been adequately evaluated for population-based HCV screening, and because the availability of a safe, valid and reliable screening test is a fundamental consideration of any screening recommendation or program,10-12 this highlights a knowledge gap and brings into question the evidence basis for these recommendations.
Limitations
Our findings are limited by the paucity and low quality of the available evidence published in English and French. In particular, we were unable to locate any studies of the accuracy of the CLIA, CMIA or MEIA (the HCV screening tests most commonly used in Canada) compared to RNA detection for HCV screening in the general population. The applicability of our findings to the general Canadian population is limited because most included studies were conducted among blood donors, and people eligible to donate blood are at lower risk for blood-borne infections such as HCV infection than the general population. Rapid and point-of-care tests were beyond the scope of our review. Although these tests are important for reaching some vulnerable populations, most HCV testing in Canada is laboratory-based;19 in addition, a systematic review of the accuracy of rapid tests was recently published.44
Conclusion
The availability of a safe, valid and reliable screening test is a primordial consideration for decision-making about screening, but our study has shown that further research is needed to adequately characterize the accuracy of antibody tests used to screen the general population for chronic HCV infection. We focused on the accuracy of HCV screening tests; however, several other important factors must be considered when making decisions about HCV screening, including the benefits and harms of screening, the benefits and harms of treatment for screen-detected cases, the cost-effectiveness of screening and patient preferences related to screening. A review of the evidence related to these considerations is beyond the scope of the present study, but such a review is being performed by other investigators in the context of the Canadian Task Force on Preventive Health Care's upcoming guidelines on HCV screening. To help inform decision-making about HCV screening, we encourage jurisdictions that have already adopted population-based (birth cohort) screening for chronic HCV infection to carefully evaluate and report on the accuracy of antibody tests as well as screening benefits and harms.
Supplemental information
For reviewer comments and the original submission of this manuscript, please see www.cmajopen.ca/content/4/4/E737/suppl/DC1
Acknowledgements
The authors are grateful to Roland Grad, member of the working group on HCV screening of the Canadian Task Force for Preventive Health Care, for his helpful comments on the study design, findings and manuscript. The authors thank Barry Pakes, Public Health and Preventive Medicine residency program director at the University of Toronto, for his support and editorial suggestions.
Footnotes
Competing interests: None declared.
Contributors: Geneviève Cadieux developed the protocol, performed the systematic review (first reviewer), performed the data analysis and drafted the manuscript. Jennifer Campbell helped develop the study protocol, performed the systematic review (second reviewer) and revised the manuscript critically for important intellectual content. Nandini Dendukuri helped develop the study protocol, oversaw the data analysis and revised the manuscript critically for important intellectual content. All of the authors approved the final version to be published and agreed to act as guarantors of the work.
References
- Copyright 2016, Joule Inc. or its licensors