An administrative data validation study of the accuracy of algorithms for identifying rheumatoid arthritis: the influence of the reference standard on algorithm performance

BMC Musculoskelet Disord. 2014 Jun 23:15:216. doi: 10.1186/1471-2474-15-216.

Abstract

Background: We have previously validated administrative data algorithms to identify patients with rheumatoid arthritis (RA) using rheumatology clinic records as the reference standard. Here we reassessed the accuracy of the algorithms using primary care records as the reference standard.

Methods: We performed a retrospective chart abstraction study using a random sample of 7500 adult patients under the care of 83 family physicians contributing to the Electronic Medical Record Administrative data Linked Database (EMRALD) in Ontario, Canada. Using physician-reported diagnoses as the reference standard, we computed and compared the sensitivity, specificity, and predictive values for over 100 administrative data algorithms for RA case ascertainment.

Results: We identified 69 patients with RA for a lifetime RA prevalence of 0.9%. All algorithms had excellent specificity (>97%). However, sensitivity varied (75-90%) among physician billing algorithms. Despite the low prevalence of RA, most algorithms had adequate positive predictive value (PPV; 51-83%). The algorithm of "[1 hospitalization RA diagnosis code] or [3 physician RA diagnosis codes with ≥1 by a specialist over 2 years]" had a sensitivity of 78% (95% CI 69-88), specificity of 100% (95% CI 100-100), PPV of 78% (95% CI 69-88) and NPV of 100% (95% CI 100-100).

Conclusions: Administrative data algorithms for detecting RA patients achieved a high degree of accuracy amongst the general population. However, results varied slightly from our previous report, which can be attributed to differences in the reference standards with respect to disease prevalence, spectrum of disease, and type of comparator group.

Publication types

  • Research Support, Non-U.S. Gov't
  • Validation Study

MeSH terms

  • Adrenal Cortex Hormones / therapeutic use
  • Adult
  • Aged
  • Aged, 80 and over
  • Algorithms*
  • Anti-Inflammatory Agents, Non-Steroidal / therapeutic use
  • Antirheumatic Agents / therapeutic use
  • Arthritis, Rheumatoid / diagnosis*
  • Arthritis, Rheumatoid / drug therapy
  • Arthritis, Rheumatoid / epidemiology
  • Databases, Factual
  • Diagnosis-Related Groups
  • Electronic Health Records
  • Female
  • Humans
  • Male
  • Mass Screening
  • Medical Record Linkage
  • Middle Aged
  • Ontario / epidemiology
  • Predictive Value of Tests
  • Prevalence
  • Primary Health Care / statistics & numerical data*
  • Reference Standards
  • Retrospective Studies
  • Sampling Studies
  • Sensitivity and Specificity
  • Single-Payer System / statistics & numerical data
  • Universal Health Insurance

Substances

  • Adrenal Cortex Hormones
  • Anti-Inflammatory Agents, Non-Steroidal
  • Antirheumatic Agents