Chapter 83: Can Intensivist Performance Be Measured?

The US health system is the most expensive system in the world, and yet numerous developing countries outperform us.¹ Patients with critical illnesses account for the majority of healthcare expenditures per capita. According to a 2009 study by the Medicare Payment Advisory Commission (MedPAC), the top 5% of spenders account for 50% of all healthcare spending, totaling $623 billion, or nearly $41,000 per patient.² Since the population is aging and medical technology is advancing, this group of patients is growing rapidly. Managing patients with critical illnesses presents considerable challenges: high-cost care, frequent readmissions, and dissatisfaction of care among them.³

ICU care is labor- and resource-intensive. The focus on quality performance and maintaining quality processes is vital to critical care. In the United States, the major players monitoring ICU quality performance include the Agency for Healthcare Research and Quality, the National Quality Forum (NQF), the Volunteer Hospital Association, the Institute for Healthcare Improvement, the Leapfrog Group, and the Joint Commission. In 2010, the NQF endorsed hospital mortality and intensive-care-unit (ICU) length of stay (LOS) as quality indicators.⁴ Usually when the NQF measure is endorsed, an exploration of policy around these measures is anticipated. Also, the growing emphasis on processes of care and outcomes is pushing providers to broaden the focus of ICU metrics beyond LOS and mortality to include readmission rates, core measures, and patient satisfaction.

However, can performance indicators such as ICULOS, ICU mortality, and readmission rates be used to measure intensivist performance? If so, what about intensivists working in safety-net hospitals with a high percentage of patients who are uninsured? On the other hand, how can accountability be spread among physicians since each ICU patient is likely to be treated by multiple intensivists and other physicians? What would be the financial implications and consequences of using such measures? The aim of this chapter is to appraise whether these performance measures are appropriate and unbiased in measuring intensivist performance.

The ACA

Most of the health systems are now facing the same fundamental challenge: how to deliver broad access to health services while improving quality of care and controlling costs. Greater competition has often been proposed as a solution that addresses each element of this challenge. The performance failures of our healthcare system are largely invisible and will continue to be invisible as long as we do not have systems that allow us to track quality performance of the providers that are the central suppliers of our health care.

The landscape is changing. CMS launched Physician Compare in December of 2010, and it expanded to include information on quality of physicians’ care in 2013. Measuring and reporting on the performance of doctors represents an effort to move to a more transparent healthcare system. In the near future, Physician Compare will provide information on physicians whether they provide recommended care to patients or not. New recommendations from the Joint Commission and CMS mandate the tracking of individual provider’s competency through the following methods: Ongoing Professional Practice Evaluation on appropriateness of privileges, procedural volume, patient satisfaction, and professional interactions.

The Institute of Medicine (IOM) recognized that payment influences provider behavior. The strategy the IOM recommended to improve quality of care was to pay for performance (P4P) or financial incentives to transform behaviors to achieve greater value.^5,6 Physician Quality Reporting System (PQRS) was a first step in this direction.⁷ Value-based purchasing (VBP) is a strategy that links payments to value of care over the entire continuum of patient treatment and hinges on recognizing and rewarding shared accountability among providers.⁸ CMS is working to transform the Medicare program from a passive payer to an active purchaser of high-quality healthcare services by linking payment to the value of services delivered.⁹

Quality Indices to Measure ICU Care

The aims of using performance indicators are to inform policy making or strategy at a regional and/or national level, to improve the quality of care, to monitor performance of healthcare funders, to identify poor performers, and to protect public safety. The main issue is which performance indicators are able to differentiate a genuine difference in quality. Process and outcome measures are frequently used in assessing quality care in the ICU. Process measures are more highly sensitive to differences in the quality of care and are easier to interpret. One advantage of outcome measurement, for example, mortality rate, is that it is a measure that is important on its own, even if the differences have nothing to do with the quality of care.¹⁰ It has been used frequently in the critically ill population, but it is important to bear in mind that outcome measures are not a direct measure of quality of care. On the other hand, a process measure is only of value if it is assumed to have a link to a meaningful outcome. By itself, it has little intrinsic value.¹⁰ Thus, robust case-mix adjustment systems are needed before a raw number is evaluated. While it is tempting to use mortality to identify poor performers, its appropriateness needs to be considered for each specific disease and operation.

Quality Performance Tied to Reimbursement

Starting October 1, 2012, the Centers for Medicare and Medicaid Services (CMS) began reducing payments to hospitals with excess 30-day, all-cause, risk-adjusted, hospital readmission rates for pneumonia, acute myocardial infarction, and heart failure. According to MedPAC in 2007: “Readmission is generally more likely the more severely ill a patient is—even within the same diagnosis-related group (DRG).” Severely ill patients whose LOS exceeded the applicable DRG by a factor of two were 26.6% more likely to be readmitted.¹¹ Patients with a number of co-morbidities are at an increased risk for readmission. Also, patients discharged to an SNF or long-term care facility had twice the risk of 30-day hospital readmission.¹¹

Physician Attribution

One of the main issues of measuring intensivist performance is physician attribution. Each episode of care would involve multiple intensivists and other physicians. How should accountability be spread among all specialists? This issue is especially germane to the ICU population because a majority of patients have multiple conditions and may be treated by multiple physicians. Moreover, nearly three-quarters of the care by intensivists in the United States is delivered in what is considered an “open” or “low-intensity” ICU staffing model: An intensivist makes treatment recommendations but has no authority over patient care.¹² Only in a small percentage of ICUs—mostly medical ICUs and ICUs in teaching hospitals—is critical care provided in a “high-intensity” or “closed” staffing pattern, in which treatment decisions are cohesively managed under the guidance of one intensivist.^12,13

Therefore, intensivists in the open system more often treat episodes with multiple physicians and to the extent that efficiency varies among the physicians. Intensivists could be unfairly penalized or rewarded if the other physicians are more inefficient or more efficient. Also, multiple attributions could increase the number of episodes attributed (at least partially) to physicians, increasing the statistical precision of performance measures for them.

ICU LOS

ICU LOS is a process measure that can be independent of quality and is easily manipulated. Encouraging earlier transfer out of the ICU could increase the risk of patient harm and increase readmission rate. On the other hand, in states that have access to LTACs, patients can be transferred to LTACs early in their course of treatment. Thus, without another measure, looking at ICU readmission, there may be pressure for clinicians to discharge ICU patients prematurely. There is potential, however, for adverse consequences that may harm patients and ultimately increase healthcare costs. Furthermore, ICU LOS should always account for ICU mortality and ICU readmission. The measures together balance concerns regarding transferring patients faster because, while the LOS measure may improve, the mortality measure is unlikely to improve and may even worsen.^14,15

There is the potential that hospitals will be rewarded unfairly by transferring a large number of patients to LTACs and encouraging the overuse of post-acute care facilities, which would drive up overall costs. At the same time, the safety-net hospitals will be penalized because the uninsured have no other option for care. Thus, in states without LTACs, both safety-net and non-safety-net hospitals would have increased ICU LOS.

Another issue is coordination of care. In the United States, critical care and palliative care are mutually exclusive entities.¹⁶ After failing a prolonged treatment in the ICU, intensivists are often the first to discuss the goals of care with patients who have reached their end-of-life and their caregivers. Using ICU LOS as a process measure would discourage intensivists from providing time-consuming, yet important, end-of-life care for ICU patients, leading to more fragmentation of care. Having the potential penalty in mind, a goals-of-care conversation is often difficult; intensivists may find it easier simply to transfer the dying patient out of the ICU or into the LTACs.

Hospital Mortality

Even though risk adjustment applies, using hospital mortality as a quality-outcome measure would not account for the impact of palliative care and the ability to transfer to LTACs. Mortality, in general, is higher in safety-net hospitals. Deaths from medical errors and deaths resulting from the decision not to pursue aggressive care are very different things. More than 90% of deaths are unrelated to unsafe care. Most other publicly available quality measures refer only to the in-patient mortality, creating an incentive to move patients to LTACs and other facilities when an end-of-life circumstance arises, an option that is not feasible in a safety-net hospital. Indeed, prior research shows that benchmarking-based in-hospital mortality simply delays death or shifts the site of death to an LTAC, without actually reducing overall mortality.¹⁵

Readmission Rates

Hospitals that serve economically disadvantaged populations, which presumably have less access to care in the community and lower levels of self-efficacy in navigating a complex, fragmented healthcare system, are going to be penalized the same as hospitals serving populations that do not struggle with these complexities. On the one hand, the safety-net hospitals cannot afford any reduction in resources. Generally speaking, states with LTACs would have lower readmission rates compared to states without LTACs. These measures reflect a process of care that is independent of quality and that can be misleading.

Physician Attribution

Most stakeholders recognize that even with perfect information it would be difficult to equitably divide responsibility for complicated mixtures of resource utilization among multiple physicians treating a single patient in an episode. Therefore, the best way is to try to arrive at a reasonable approximation to reality in these situations. Instead of using single attribution, multiple attribution acknowledges that the decision maker, if there is one, has incomplete control over treatment by intensivists and other physicians, even if the decision maker referred the patient to those other physicians.

ICU LOS

One benefit of performance measurement is to encourage changes—particularly systemic changes—that improve patient outcomes. Public reporting of ICU outcomes can provide the stimulus for needed system changes and appropriate use of resources. The ICU mortality measure has been publicly reported in California since 2007. Developers and implementers of the measure have not received feedback from hospitals identifying unintended consequences. According to MedPAC in March 2011, since reporting of the measure began, patient risk profiles are basically unchanged, but ICU mortality has declined by 0.5%—a statistically significant difference.¹⁷ There has also been excellent engagement in benchmarking and quality-improvement strategies across the ICUs in California as a result of the public reporting of the mortality measure.

The noisy, active ICU is probably not the best environment for end-of-life care, and transfer to other units may be appropriate and desirable. The transfer will only affect the LOS measure since an in-hospital death is still included in the mortality measure.

Hospital Mortality

The suggestion that a 30-day mortality measure would be an alternative to avoid discharge bias has been studied.¹⁸ Comparing in-hospital and 30-day mortality, there was little change in performance, particularly at the high- and low-performance levels. The data from the National Death Index required for 30-day morality measures is expensive, and its availability is delayed by at least 2 years. On balance, the in-hospital measure provides good data in a timely manner.

Readmission Rates

CMS has gone on record as stating that there are several safety-net hospitals that do not show evidence of higher-than-expected readmissions rates. This implies that it is not an unaccomplished act that these hospitals are unable to make improvements in care delivery and services to reduce avoidable rehospitalization.

For patients with critical illnesses, improving care coordination does not necessarily mean the index hospitalization would have a shorter LOS and a lower mortality. Whichever variables are chosen must satisfy several preconditions. They must be relevant to deciding whether high-quality care is delivered. Mortality and LOS are simple to measure, but they are only one piece of a larger puzzle. Measuring one process without understanding other processes may be misleading. The current system has not yet been made to link the fragmented entities caring for these patients with critical illnesses around accountability for value. Implementing performance measures such as ICU LOS would only widen the gap between safety and non-safety-net hospitals.

ICU LOS and hospital mortality may be good surrogates for an adverse event: medical errors and hospital-acquired infections; however, there are certain conditions that require longer LOS with better patient outcomes that may lead to reduced cost overall. The more appropriate indicators should include the total cost of care, outcomes of each chronic disease management separately, management of adverse events, as well as patient and caregiver satisfaction.

To improve coordination and decrease fragmentation of care, intensivists should be encouraged to broaden care beyond the acute episode and to weigh in on quality of life post-ICU care.¹⁶ There should be greater responsibility and accountability placed on physicians to discuss the prognosis and goals of care. Using ICU LOS as a process measure would adversely affect the quality of care in patients with critical and terminal illnesses. On the other hand, if a hospital has a lower mortality rate, it could mean a greater proportion of its discharged patients are eligible for readmission. To some extent, a higher readmission rate may indicate successful care.¹⁹

To improve service productivity, measuring and monitoring performance and its variance is a fundamental requirement for identifying efficiencies and best practices and for spreading them throughout the system or organization. Although some variance is inevitable, much of it can be controlled if each system properly accounts for differences in the type of patients they serve and then defines and collects data uniformly across different patient populations.²⁰ To do so, the intensivists in each critical-care service need to compare themselves against their own performance rather than against poorly defined external benchmarks, which only compounds the difficulties for accurate assessment.

Advancing performance measurement at the physician level is the vital strategy on the policy agenda when considerable unexplained variation exists in practices that lead to poor quality, inefficient care delivery, and waste of resources. To move the physician measurement and reporting agenda forward, there is a need for continued development of evidence-based quality measures that can be applied to gauge individual physician performance on a deeper array of medical conditions and specialties. There is a need for research to assess the optimal ways to construct reproducible and unwavering performance scores (eg, levels of aggregation) to assess the optimal ways to provide feedback to physicians and patients to facilitate their understanding and use of the information.²¹