Abstract
Background: There are concerns that alternate payment plans for physicians may be associated with erosion of data quality, given that physicians are paid regardless of whether claims are submitted. Our objective was to determine the proportion of claims submitted by physician specialists using fee-for-service and alternative payment plans, and to identify and compare the validity of information coded in physician billing claims submitted by these specialists in Calgary.
Methods: We conducted a survey of physician specialists to determine their plan status and obtained consent to use physicians' claims data from 4 acute care hospitals in Calgary. Inpatient and emergency department services were identified from the Discharge Abstract Database for Alberta (Canadian Institute for Health Information) and the Alberta Ambulatory Care Classification System database. We linked services to claims by Alberta physicians from 2002 to 2009 by using unique patient and physician identifiers. After identifying the proportion of claims submitted, we reviewed inpatient charts to determine the completeness of submissions as defined by positive predictive value.
Results: Of 182 physicians who responded to the survey, 94 (51.6%) used fee-for-service plans exclusively and 51 (28.0%) used alternative payment plans exclusively. Overall completeness of physician submissions for claims was 91.8% for physicians using fee-for-service plans and 90.0% for physicians using alternative payment plans. Submission rate varied by medical specialty (surgery: 92.4% for fee for service v. 88.6% for alternative payment; internal medicine: 94.1% v. 91.3%; neurology: 95.1% v. 91.0%; and pediatrics: 95.1% v. 89.3%). Among claims submitted, the physician accuracies for billing of medical conditions were 87.8% for fee-for-service and 85.0% for alternative payment.
Interpretation: Overall submission rates and accuracy in recording diagnoses by physicians who used both plans were high. These findings show that the implementation of alternative payment plan programs in Alberta may not have an impact on the quality of physician claims data.
Canada has one of the richest sources of coded health care data in the world, in part because of its universal health care system.1 In the traditional fee-for-service payment model, physicians must submit claims for every medical service provided to receive remuneration.2,3 Physician billing claims databases in Canada contain unique patient identifiers, physician identifiers and clinical information, such as a patient's diagnosis at the time of a visit.4 Health administrators, decision makers and researchers use these data to examine the trajectory of patient care volume, forecast health care budgets, and for disease surveillance and related health services research.5-8 Because physicians using fee-for-service plans must submit a claim to be paid, it has been assumed that this payment system provides relatively complete and accurate estimates of the provision of medical services. However, this reliance on the fee-for-service payment mechanism is an often cited source of inefficiency in the Canadian health system and works poorly for chronic disease management.9
In 2009, over 20% of Canada's 55 000 physicians received some payments for clinical care from an alternative payment plan.10 Generally, physicians using alternative payment plans are specialists (e.g., internal medicine, surgery and pediatrics) rather than general practitioners.11 Physicians in alternative payment plans are paid a fixed amount of money rather than being paid a fee for each medical service they provide. The mechanisms for physicians in these plan types, such as salary or capitation payment, promise greater control over the levels of physician compensation and more efficient use of health care facilities.12-14 Unfortunately, alternative payment plans typically do not provide financial incentives for physicians to submit claims for all their services (i.e., they are not compensated for their time spent submitting claims, referred to as shadow billing), which raises concerns about potential data erosion because of nonsubmission of shadow billings. This gap within the payment models may create potential inequities in the quality of claims and result in data erosion. The impact of incomplete and inaccurate billing submissions is a concomitant decrease in the ability to effectively track health care service volume and use (e.g., the burden of chronic diseases). Worse still, this creates doubt in the usefulness and overall validity of the administrative databases. The occurrence and extent of data erosion has important implications in the use of these data, including the need to statistically adjust for losses in the underestimations of disease burden and budget projections. Finally, given the widespread use of physician claims data for research purposes, confidence in the quality of these data is critical for their future use in health services and population health research.
Our objective was to determine the claim rates of physician specialists using alternative payment and fee-for-service plans, and to identify and compare the validity of the information coded in physician billing claims to these plans submitted by physician specialists practising in Calgary. The occurrence and extent of data erosion has important implications for use of these data, including the need to statistically adjust for losses in estimates of disease burden and budget projections. Finally, given the widespread use of physician claims data for research purposes, confidence in the quality of these data is critical for the future of their use in health services and population health research.
Methods
Setting
Calgary had a population of 1.2 million people in 2009.15 We approached physicians using alternative payment and fee-for-service plans who were practising at 1 or more of 4 adult and pediatric acute care hospitals in Calgary between Jan. 1, 2002, and Dec. 31, 2009. Ethics approval for this study was granted by the Conjoint Health Research Ethics Board at the University of Calgary, Calgary.
Design
To generate a baseline sample, an online survey of physicians in Calgary was conducted to determine their alternative payment, fee-for-service or blended plan status and to obtain consent to use their claims data. Physicians using blended plans have both fee-for-service and alternative payment remuneration components to their contracts. The survey sample population was generated using the list of physicians in the College of Physicians & Surgeons of Alberta directory, in Scott's Canadian Medical Directory (2008 edition), and in the directories of the clinical departments in the hospitals and the University of Calgary. We verified the contact information through the current Calgary telephone directory, the Yellow Pages and physician contact directories posted on the websites of Alberta Health Services, the hospitals and the University of Calgary. The following physician specialties were targeted because they were thought to offer a mix of alternative payment and fee-for service plans: intensive care unit, internal medicine, neurology and neurosurgery, pediatrics, psychiatry and general surgery. We restricted our sample to these specialties because they were established APP and FFS programs and they had a large number of registered physicians in Calgary. We included physicians who were practising in 2009 and who provided inpatient services at 1 or more of the 4 acute care hospitals. We excluded general practitioners because most were remunerated through fee-for-service plans, and medical trainees (i.e., medical students and residents) because they do not submit billing claims in general.
We emailed the survey packages to the selected physicians. The package included an invitation letter, a consent form and a brief, self-administered questionnaire (Appendix 1, available at www.cmajopen.ca/content/3/4/E406/suppl/DC1). Respondents could choose to complete the survey online, by email or by mail. Three email reminders were sent. The time interval between each of the 3 reminders was 3 weeks.
Outcomes and sources of data
Data from the Alberta Discharge Abstract Database and the Ambulatory Care Classification System were linked with the physician survey by physician unique identifier. In Canada, Discharge Abstract Database and inpatient emergency department data are coded using a rigorous and standardized system. This is because coding professionals already have a rich international system of coding standards and education; therefore, the Discharge Abstract Database is thought to have high-quality and valid health data.16 These data are not impacted by physician payment programs. A study evaluating physician claims data found high face validity; however, further validation work is still needed to assess provincial variation in the quality of these data.17 We extracted all services provided by these physicians from Jan. 1, 2002, to Dec. 31, 2009. The linked records formed the denominator (i.e., standard reference; number of services provided by physicians).
We verified the physician unique identifier within the Alberta Health Care Insurance Plan Physician Claims Data and then linked the extracted Discharge Abstract Database and Ambulatory Care Classification System services to Alberta physician claims from Jan. 1, 2002, to Dec. 31, 2009, by unique patient number (i.e., Personal Health Number), service location, service date and physician unique identifier to determine the numerator (i.e., number of services claimed by physicians). In Canada, every citizen is covered by the universal health care system, whether or not they can afford to pay for health care. The insurance registry, which is updated regularly, contains the name, date of birth, sex and postal code for all individuals eligible to receive health care services, and is considered a proxy for census data. It is considered highly valid.
Chart review
To assess the validity of submitted claims, 55 physicians (24 using a fee-for-service plan and 31 using an alternative payment plan) were randomly chosen for chart review. Charts were excluded if they had missing data, did not have corresponding ICD-9 codes to match, were duplicates or were not available when requested from health records. Random numbers were assigned to each claim within the data set. The claims records were then sorted by this random number in ascending order. We selected the first 10-19 sorted records from each stratum, resulting in a total of 1115 claims records.
Corresponding inpatient charts for the extracted claims were located using a combination of the hospital ID, patient chart number, Personal Health Number, admission date and physician unique identifier. Two chart reviewers underwent training in the data extraction process. In the training session, the definition of study variables was discussed, and 20 charts were reviewed by both of the reviewers together to identify any issues involving the interpretation of coding or medical terminology. A sample of 30 charts was then reviewed independently by both reviewers for consensus agreement. Reviewers reached agreement on diagnoses for 29 of the 30 charts with a κ score = 0.92, which showed almost perfect agreement.
Reviewers extracted data independently for evidence of the diagnoses submitted in the physician claims through an examination of the entire chart up to the date of the claim (i.e., medical service date). Administrative data are typically coded using the World Health Organization's International Classification of Disease. In physician claims, diagnoses during the study period were coded using the International Classification of Disease, 9th revision (ICD-9), Alberta modification. The Physician Claims Database contains up to 3 diagnoses. However, about 95% of the claims records contain only 1 diagnosis. Therefore, we verified the diagnosis coded in the first coding field. Reviewers recorded the diagnosis and extracted additional medical notes from the chart and then determined whether there was a match in the chart review with the ICD-9 diagnosis in the physician claims data.
Statistical analysis
The unit of analysis was the "physician service" provided by participants in the survey who consented to data linkage. Descriptive statistics were used to report physician characteristics, submission rate and validity of the claims diagnosis, by payment status, type of service, and medical specialty. We assessed differences between the 3 payment plans using the χ2 test.
To calculate the claims submission rate, the number of services claimed (Alberta Health Care Insurance Plan Physician Claims Data) was divided by the number of services provided (Alberta Discharge Abstract Database and the Ambulatory Care Classification System). Positive predictive values for ICD-9 diagnoses submitted by physicians were calculated based on the assumption that the chart review data were the reference standards. The positive predictive value determined the extent to which a diagnosis present in the ICD-9 claims data was also present in the chart review data and was used to assess the overall validity of the chart review data compared with the ICD-9 claims data.
Results
Of 904 physicians surveyed, 317 consented to have their medical services linked to their claims data (35.1% response rate). A follow-up questionnaire was sent to the 587 physicians who declined to participate to explore reasons for nonresponse. Of those who responded to the follow-up survey (n = 63), 44 (69.8%) were males and 19 (30.1%) were females. Respondents to this follow-up survey were from the following specialties: internal medicine (22, 35.0%), general surgery (18, 28.6%), neurology/neurosurgery (2, 3.0%), pediatrics (11, 17.5%) and psychiatry (10, 15.9%). Of the 317 consenting physicians, 38.1% were reimbursed by fee for service, 14.1% by a partial alternative payment plan and 47.8% by full alternative payment plan.
Of the 317 physicians who responded, we were only able to link 182 physician unique identifiers with the Discharge Abstract Database and the Ambulatory Care Classification System. This was most likely due to a wrong or inaccurate physician unique identifier provided by the physician in the original survey. Of the 135 physicians who were not linked, 53 (39.3%) were reimbursed by fee for service, 60 (44.4%) by full alternative payment plan and 21 (15.6%) by partial alternative payment plan (1 physician did not provide payment information). Of the 182 physicians whose data were linked, 51 (28.0%) used alternative payment, 94 (51.6%) used fee for service and 37 (20.3%) used a blended model. Characteristics of survey participants are outlined in Table 1.
We analyzed 149 380 services provided by 182 physicians (Table 2). The overall submission claims rate was 90.0% for physicians reimbursed by alternative payment, 95.6% for those reimbursed by a blended payment and 91.8% for those reimbursed by fee for service. For the physicians reimbursed by fee for service, psychiatry (97.0%) had the highest submission rate followed by neurology/neurosurgery (95.1%), pediatrics (95.1%), internal medicine (94.1%) and surgery (92.4%). For the physicians reimbursed by alternative payment plan, the submission rate varied by specialty with a high of 91.3% for internal medicine and a low of 77.0% for other specialties. Among physicians reimbursed by alternative payment plan, the proportion of submitted claims was significantly higher when an incentive was provided (93.6% v. 89.4%). Submission rate was associated with length of practice, but not consistently.
We analyzed the differences in submission rates among physicians who changed payment plans from fee for service to alternative payment during the study period. The proportion of submitted claims decreased significantly from 95.6% before the change to alternative payment to 90.1% after changing to alternative payment (Table 3).
Of the 1115 charts requested for review, 849 were included (447 [52.6%] completed by physicians reimbursed by fee for service and 402 [47.3%] for those reimbursed by alternative payment; Table 4). Overall, FFS physicians reimbursed by fee for service had a slightly higher positive predictive value (87.0%) than physicians reimbursed by alternative payment (85.8%). For fee-for-service records, psychiatry had the highest positive predictive value (100%), followed by surgery (91.0%), pediatrics (82.4%) and internal medicine (76.5%). For alternative payment plan records, neurology had the highest positive predictive value (93.3%), followed by surgery (92.0%), pediatrics (91.0%) and internal medicine (81.0%).
Interpretation
Main findings
We analyzed physician claims submitted by Alberta specialists for inpatient and emergency department services and found that physicians reimbursed by alternative payments plans submitted significantly fewer medical service claims than physicians reimbursed by fee-for-service plans; however, most services that were provided by both groups had claims submitted, and the accuracy of the diagnostic coding of these claims was high.
There was a slight decrease in the overall rate of submission after implementation of an alternative payment plan; however, our results show that physicians using alternative payment plans submitted over 90% of their claims that were reimbursed by alternative payment (i.e., less than 10% of claims were either not submitted or lost), and this was a smaller proportion of claims lost than was expected. Given the hypothesis that physicians using alternative payment plans may not be submitting their shadow billing claims, our findings do not support the original hypothesis. The potential for data loss with the implementation of alternative payment plans seems to be minimal and does not appear to have affected the overall completeness and accuracy of claims being submitted in Alberta. We had hypothesized that submission rates would be close to 100% for FFS physicians reimbursed by fee for services plans; however, the rates were closer to 90%. Possible explanations that could account for the missing billing submissions include physicians or administrative staff forgetting to submit a claim, rejected claims and inaccurate data linkage.
Comparison with other studies
Alberta is one of several Canadian provinces that require physicians reimbursed by alternative payment plans to submit shadow bills to account for the services they provide (Alberta Health Services: unpublished data, 2011). In addition, many provincial alternative payment plans based at teaching hospitals use incentive-based programs to motivate physicians to submit billings. For example, in some divisions or departments in Ontario and Alberta, physicians reimbursed by alternative payment plans who do not submit the recommended quota of shadow billings based on their expected annual patient workload face a potential withholding of a portion (e.g., 15%) of their annual earnings.18,19 Internal reviews compare physicians' shadow billing submissions for the year with other physicians within their specialty with a similar expected workload as part of this process.20 Similarly in Nova Scotia, to ensure complete and accurate submission of shadow billing information, the values of shadow billing data are periodically compared with total payments under the physician's alternative payment plan contract.20 Unfortunately, departmental or provincial incentives have received relatively little evaluation.21
Monitoring of shadow billing has the potential to improve the quality and completeness of claims submitted by physicians. Canadian health agencies and provincial governments are beginning to realize the possible risk of data erosion with the switch to alternative payment plans and are implementing policies to adjust for the possible undersubmissions of shadow bills.
Limitations
Our study has several limitations. We surveyed a small number of specialists in intensive care, internal medicine, neurology, neurosurgery, pediatrics, psychiatry and general surgery practising at 1 or more of 4 adult and pediatric acute care hospitals in Calgary between 2002 and 2009. Our findings may not be generalizable outside of this specific setting. We surveyed physicians in urban areas, not rural-based physicians. However, only a small proportion of specialists are practising in rural areas. We focused on inpatient and emergency department claims records and did not analyze outpatient records. Physicians at private or outpatient clinics may display different billing behaviours. Our results may not be generalizable to countries with different health care systems or different payment models.
There was a potential for the introduction of response bias because of the differential response rates by payment program. Although were unable to identify the payment program of nonresponders, the proportion of survey respondents reimbursed through alternative payment plans is similar to the proportion of physician specialists in Alberta reimbursed through alternative payment plans.
We calculated only the positive predictive value and did not report statistics, such as sensitivity, specificity and negative predictive value. In this instance, it is important to note that the positive predictive value does not necessarily reflect whether the patient truly had the condition or disease in question when they visited their physician; it reflects only what was recorded in the physician billing claim or what the physician wrote down as the patient's reason for the visit. It shows that the proportion of services provided by physicians was billed. We did not assess predictors of nonsubmission because of a lack of statistical power. Models that predict or screen physicians who are less likely to submit shadow billing should be developed in the future. The model will be helpful for promoting the submission of billing claims among targeted physicians.
Conclusion and implications for practice and future research
Our analyses showed that overall claims submission rates for physicians reimbursed by both methods were high, as was the validity of diagnostic coding. This suggests that, contrary to popular belief, implementation of alternative payment plan programs in Alberta may not have an impact on the quantity or frequency of physician claims submitted. Because Alberta uses shadow-billing incentive programs, future research is needed to determine if incentive programs should be considered in other provinces or nationally to preserve the overall quality of physician claims data. In addition, there should be continuing vigilance with respect to completeness and frequency of physician claims submitted, regardless of the payment system.
Supplemental information
For reviewer comments and the original submission of this manuscript, please see www.cmajopen.ca/content/3/4/E406/suppl/DC1
Footnotes
Competing interests: None declared.
Contributors: Ceara Cunningham, Nathalie Jetté, Hude Quan, Bing Li, Brenda Hemmelgarn, Tom Noseworthy, Cynthia Beck, Elijah Dixon, Susan Samuel, William Ghali and Carolyn Decoster acquired the data. Ceara Cunningham, Bing Li, Ravneet Robyn Dhanoa and Hude Quan performed the analyses. Ceara Cunningham, Hude Quan and Nathalie Jetté drafted the article, and Bing Li, Brenda Hemmelgarn, Tom Noseworthy, Cynthia Beck, Elijah Dixon, Susan Samuel, William Ghali and Carolyn Decoster revised it critically for important intellectual content. All of the authors contributed substantially to conception and design, interpreted the data, approved the final version to be published and agreed to act as guarantors of the work.
References
- Copyright 2015, 8872147 Canada Inc.
In this issue
Article tools
Related Articles
Cited By...
- No citing articles found.