Bereft of both quality improvement and professional accountability

In 1979 California housed 22,632 inmates in 12 prisons [13]; by 1999 the inmate population had grown to 160,687 and the number of prisons to 33 [14], the majority now in isolated desert locations. By many measures, the system was in shambles. A class action lawsuit filed in federal court in April 2001 argued that inmates were "not receiving constitutionally adequate medical care as required by the Eighth Amendment to the U.S. Constitution and that defendants are not complying with the Americans with Disabilities Act (ADA) and § 504 of the Rehabilitation Act" [15]. Following extended courtroom negotiations regarding medical care quality, the state and inmate plaintiffs agreed in 2002 to a remedial plan to be monitored by inspections and audits. That plan failed, however, and in 2005 the state acknowledged that “an inmate in one of California’s prisons dies needlessly every six to seven days due to constitutional deficiencies in the … medical delivery system” [16]. In consequence Thelton Henderson, the presiding federal judge, placed the prison medical system in receivership and ordered a reduction in the prison population [17], which from 2006 through 2008 exceeded 170,000 inmates.

In addition to glaring system dysfunctions such as medical records “either in a shambles or non-existent” and pharmacy “in almost complete disarray,” these deficiencies included “a prevailing lack of accountability” [16]. The state’s own expert witness conceded that “historically the CDCR [California Department of Corrections and Rehabilitation] would hire any doctor who had ‘a license, a pulse, and a pair of shoes’” [16]. Many states have a documented history of collaboration between medical boards and correctional systems to send physicians with restricted licenses into prison medical care [18, 19]. In California’s prison medical care system, the presence of physicians under medical board discipline was rampant [20]. While Judge Henderson acknowledged the presence of a cadre of highly qualified and dedicated clinicians, he concluded that even these clinicians were working ineffectively in a system that was “broken beyond repair.”

Under the terms of the receivership, the court recruited a new leadership team with a mandate to improve mortality and overall quality in the medical delivery system. To guide the transformation efforts, the leadership team relied on the Institute of Medicine’s Crossing the Quality Chasm report [2] and launched a series of interventions to address deficiencies in the structures and processes of care delivery, including human and physical resource deficiencies. Although the receivership continues, this bundle of accountability and quality improvement interventions has proven effective [21]. In the analysis that follows, we focus particularly on the years 2005 through 2007 and the impact of removing problem physicians.

The implementation of physician accountability

Inspection of Fig 1 suggests a modest improvement in mortality trend from 1998 to 2006 and a significant change in 2007. Court oversight, court-mandated improvements in mental health services, and isolated practice improvements [34] may have contributed to the initial modest decline. The federal court takeover and formation of the receivership team occurred in mid-2006. While it may be tempting to link the receivership’s quality improvement interventions with the significant mortality decrease in 2007, our narrative analysis of the years 2004 to 2007 reveals that the reasons for the downturn must have preceded those interventions. The decrease in mortality appears to be associated with implementation of accountability and peer review systems, i.e., with workforce, rather than the receiver’s quality improvement initiatives that came later.

Beginning in 2004, the court focused heavily on death reviews and the inadequate care they revealed. Court-appointed experts brought physician incompetence to light and described inadequate physician credentialing, supervision, peer review, and discipline. Many of the physicians had checkered or criminal pasts, and peer review was “bogus” or “not done at all” [16]. The state then contracted with the Physician Assessment and Clinical Education (PACE) program at the University of California, San Diego, to provide competency testing and remediation, if appropriate, for its primary care physicians. The state agreed to hire only board-certified or “board-eligible” physicians and began to send physicians without time-limited certification to the PACE program for testing. The department’s physician leaders initiated a broad-based peer review process that was centralized statewide.

The death review findings often brought physicians to the attention of the peer review committee, which supplemented that evidence with pattern-of-practice reviews of up to 60 charts, PACE assessments if available, and interviews with the physicians of interest if appropriate [35]. The committee could remove physician privileges if warranted by standard of care deviations and additional evidence. From June 2005 to July 2007, the committee took adverse actions against 56 physicians [35]. By law, peer review privileging actions, civil judgments, settlements, and criminal convictions require reports to the Medical Board of California. Table 2 shows that in fiscal years 2005–2007, the California Department of Corrections and Rehabilitation filed 70 reports, nearly two orders of magnitude more than the statewide average rate. Additional voluntary departures occurred once staff realized that increased scrutiny was inevitable and privileging actions a possibility. An internal memo described the exodus as follows:

“Of the first 53 physicians completing the PACE assessments by May 2006, 18 passed, 3 failed outright, and 32 passed with recommendations for remediation.… Of the 79 physicians who separated from state service from December 2004 to April 2006, only 10 were board certified and in good standing.”

The resulting vacancies were filled largely by physician contractors (not state employees) until the receiver raised salaries, included a differential for time-limited board certification, streamlined the hiring process, and intensified recruitment efforts. These measures led to an increasing number of physician applicants who successfully passed the initial hiring examination: 184 from March 2006 to March 2007 and another 105 in the next two months [36].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Reports to the medical board of California.

https://doi.org/10.1371/journal.pone.0199961.t002

Accountability processes preceded quality improvement initiatives

The receiver also launched a host of initiatives to improve system performance, described at length in plans submitted to the court [37, 38] and illustrated in Fig 2. The receiver’s regular reports to the court have documented successful implementation of virtually all these quality and infrastructure initiatives [21]. What is now glaring, however, and came as a surprise to the first author, is that the decrease in mortality occurred long before these initiatives became relevant. In spite of the receiver’s broad powers and financial resources, program implementation was slow. Both the physical facilities and the bureaucracy could be almost unimaginably inhospitable and intransigent. Connecting stand-alone computers in all the clinics took three years, delaying deployment of an electronic pharmacy system. A small physician leadership team attended its first quality improvement training from the Institute for Healthcare Improvement in July 2007. The first interdisciplinary chronic care collaborative, focused on asthma, began in June 2008 with 6 prisons; the other 27 prisons were engaged in 2009. The newly created quality, utilization management, and public health units installed their first physician directors in 2008. An executive in charge of reforms in laboratory, radiology, and ancillary services began work in late 2009. Accreditation for continuing medical education came in 2009. In summary, implementation of the receiver’s “Quality Chasm” portfolio began in earnest in 2008–2009. Development of statewide systems for professional accountability and peer review, in contrast, began in 2005 and accelerated in 2006–2007. Physician competency testing at the PACE program began in March 2005, and remedial training began in June 2006. These accountability processes and workforce improvements preceded the drop in mortality seen in Fig 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Driver diagram of interventions to improve physician performance in the California Department of Corrections and Rehabilitation (CDCR).

Recruiters emphasized salary, mission, and a practice environment of renewed professionalism. Peer review enabled removal of truly problematic physicians. Although this 2006 diagram lists numerous system improvements designed to boost patient safety and professionalism, the only interventions that were mature in 2006 were the salary increases and processes for physician accountability. CME, continuing medical education; QI, quality improvement; UM, utilization management; PDA, personal digital assistant (mobile device).

https://doi.org/10.1371/journal.pone.0199961.g002

The annual summaries of death reviews provide corroborating evidence of excess mortality linked to physician performance in 2006 followed by improvement in 2007. In 2006, 18 deaths were deemed “preventable” and another 48 considered “possibly preventable” [35]. Physicians made an array of errors leading to 6 asthma deaths, for example, and failed to respond to classic cardiac signs and symptoms in another 6 deaths. One inmate died of a perforated ulcer after presenting multiple times over 5 days; another died of acute pancreatitis after presenting 9 times over 3 days; and another died of incarcerated hernia after physicians delayed 5 weeks before referral to a specialist in spite of recurrent abdominal pain, vomiting, and known inguinal hernias. In 2007 the physician leadership refined the process for determining whether deaths were preventable; they also developed a taxonomy for identifying lapses in care at both the individual and system level, as described above. The most common serious lapse was the “failure to recognize or evaluate important signs and symptoms,” consistent with the importance of diagnostic errors as described by patient safety advocates [39]. Reflecting death data through 2007 and peer review data through July 2008, the annual summary published in November 2008 concluded:

“The major clinical impact felt throughout the system … has been from improvements in the number and caliber of healthcare professionals. … Most of the Receiver’s other interventions that aim to improve the quality and cost-effectiveness of healthcare services are still in early, pilot, or planning stages. … Death rates in the CDCR are significantly decreasing in part because the high-quality … peer review process has resulted in the replacement of 85 potentially dangerous providers with new well-qualified providers” [33].

As described above, the receiver’s quality initiatives took shape in 2008 and gained traction gradually over the next several years. In 2012 the outside reviewer could finally point to these as responsible for an overall decline in serious lapses in care [40].

Recovering professionalism

In addition to the direct harms inflicted by problem physicians, the court’s uncontested 2005 findings document pointed to harms induced by “a culture of non-accountability and non-professionalism.” The court’s message to qualified and dedicated physicians “who have been struggling to provide quality care in dire circumstances” was that “California is about to embark on a dramatic transformation of its prison medical system” [16]. We have correlated the mortality decrease with implementation of workforce accountability processes and replacement of problem physicians by qualified physicians, but it would be helpful to know more about changes in social dynamics and norms in the years 2005 through 2007. The archival documents offer little direct and systematic evidence regarding staff morale and professional norms, but there is indirect evidence of transformation.

The baseline was grim. Court experts in 2006 cited the frustration of managers with their inability to discipline staff and the ploys used by poor performers, which were “demoralizing to both managers and staff who are working hard” [41]. Plaintiff attorneys routinely inspected prisons and wrote detailed, structured reports about medical facilities, staffing, and processes. A review of 31 such reports revealed significant changes in local medical staffs from 2006 to 2007. In 2006 the reports commonly used language such as “still in crisis” and “mired in the same problems.” In one prison, some physicians were routinely working less than a full day, but “management seemed at a loss,” afraid that “if individuals were disciplined, they would not be able to fill the positions at all.” Only two of the 13 reports from 2006 used positive descriptors, e.g., in October 2006 managers and medical staff in one prison were “excited about the improvements that have occurred.” In contrast, 13 of 18 reports in 2007 used positive language about diverse prison medical staffs such as “strong commitment,” “steady improvement,” “proud of the achievements they had made, and with good reason.” At another prison, “the hiring landscape has dramatically changed for the better, with contract physicians now applying for permanent state jobs.”

Similarly, the receiver’s June 2007 report about San Quentin cited the improved stability of the medical staff, the recruitment of recent graduates from the University of California, San Francisco, and the new interest from contract physicians in becoming state employees [36]. A declaration submitted by the first author in July 2007 with regard to other prisons asserted, “Bit by bit, the local prison staff are becoming enthusiastic about the realistic possibilities for significant change” [42]. In a 2008 declaration he could more confidently assert, “there has been a change in the attitude of healthcare staff, many of whom are voicing newfound empowerment and recommitment to their original motivations as healthcare professionals” [43].

These various reports suggest aggregate improvement in morale and restoration of professional norms; they also allude to the potential downsides of accountability implementation. As noted in the introduction, peer review in stable systems has been characterized at times as a heads-on-platter exercise. In this unstable system there was widespread concern that the court’s focus on weeding out problem physicians, particularly in the context of death investigations, would lead to unfair blame of individuals, sometimes referred to as “collateral damage.” The following description from the receiver’s September 2007 report [44] is terse but nevertheless conveys the gravity of these processes:

The committee met 14 times from June 2007 to September 2007 and reviewed 33 initial allegations of clinical misconduct or neglect and the findings of 23 peer review investigations. The committee acted by suspending the privileges of 12 practitioners and restricting the privileges of an additional three physicians. They also restored the privileges of two physicians.

In 2006 and 2007, the receiver’s team revised the death review process to incorporate safety principles and developed a taxonomy of lapses in care to assist with identification of system issues contributing to adverse outcomes [33]. In order to promote fairness and allay rising staff anxiety, the receiver’s team introduced “just culture” principles and promoted use of James Reason’s decision tree for determining culpability of unsafe acts [45]. The team also hastened to deploy professional development programs for all physicians, as illustrated in Fig 2, in addition to remediation opportunities for marginal physicians. Given that good physicians were often “isolated among those considerably less competent and/or less dedicated than themselves” in a system with “few support and safety mechanisms in place,” these changes helped mitigate medical staff concerns that heightened accountability processes could indiscriminately sweep up marginal or even fully competent physicians [37].

The physician union repeatedly expressed concern about physicians being unfairly blamed for bad outcomes. There were also complaints of discrimination against minority physicians; given evidence of continued discrimination and associated career dissatisfaction in medicine at large [46], this constituted another major concern. It is noteworthy, however, that the union repeatedly expressed support for efforts by the court and receiver to restore professionalism and in particular for a thoughtful peer review process independent of monitoring and control by non-physician state employees [36]. In spite of the evident anxiety and consternation caused by implementation of accountability processes and the potential negative effects of this anxiety and consternation upon staff morale and relationships, it appears that norms improved, and the fact remains that mortality decreased.

Professional self-regulation within organizations

Given the long-standing, oft-cited problems with peer review [47], its achievements in this system were noteworthy. The peer review committee reviewed patterns of practice, not just single incidents, and solicited input from multiple sources, including the physician under review. With input and eventual support from the physician union, the governing body developed a due process procedure that withstood legal challenge [48]. As within most organizations, many of the same physicians led both the peer review and quality improvement efforts.

A recent primer for creating a “thoughtful, fair, systematic, and organized approach” to peer review [49] noted the surprising lack of literature on the topic. We share the concern about scholarly neglect of this domain. Most discussion of peer review and professional self-regulation fails to account for the evolving organizational forms of our current delivery systems, as if hospitals and medical boards were the only loci of activity. Health plans and many medical groups, clinics, and ambulatory surgery centers perform formal peer review, as does the occasional post-acute or long-term care provider, all overlaid with a now-bewildering array of contractually-accountable entities that also review individual physician performance. Peer review systems can operate across an entire state, as in this case study, or even across the nation [50].

Macro-level policy discussion about professional self-regulation, e.g., maintenance of certification, reveals antagonistic divisions between the profession’s guardians of competency and the rank-and-file [51]. At the micro-level within organizations and delivery systems, however, the available evidence suggests that these contentious debates do not necessarily reflect physicians’ everyday attitudes and behaviors. One survey found that physicians are “disheartened” when their efforts to discipline fellow physicians are thwarted [52]. Another study has described how perceived violations of clinical standards across specialties within a hospital can trigger moral emotions that mobilize individual and/or collective action, including a range of corrections and “other-condemning” penalties directed toward the offending physicians [53]. This qualitative study found that rank-and-file physician behavior was explicitly motivated by concern for patients and respect for professional values. The study did not document physicians’ reactions to overt dyscompetence, which presumably would trigger yet stronger emotions and actions.

The comprehensive and methodical nature of the peer review processes deployed in our setting may have mitigated against the perception of arbitrary injustice and assisted in winning physician support. While we have alluded to anxieties induced by peer review, the physician leaders and union all came to support it. We also found indirect evidence of improvement in professional norms. The perception of procedural justice has been associated with improved clinical performance [54], and evidence from social science suggests that use of appropriate sanctions with marginal and/or difficult employees can elevate group norms and improve motivation [55]. As shown in Fig 3, this dynamic may have contributed to the reduction in population mortality. Beyond the direct harms done by dyscompetent professionals, these individuals can have significant negative impacts on group performance, e.g., lowering group morale, trust, resilience and creativity, thus contributing to burnout. Individuals who are skilled but unconscientious in their work have disproportionately negative effects compared to individuals who are well-meaning but lacking in skills [56].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Drivers of mortality reduction.

https://doi.org/10.1371/journal.pone.0199961.g003

The reluctance of organizations to discipline physicians, even in “egregious” cases [50], is widely known, as is their reluctance to report physicians who are disciplined. A 2017 study of five Veterans Administration Medical Centers found that only one of nine disciplined physicians were reported to appropriate boards. Most of the physicians not appropriately reported were “likely” practicing elsewhere; one of these subsequently had privileges revoked by another hospital for the same cause [57]. A study commissioned by the California legislature found that organizations often pressure physicians to resign in ways that do not trigger formal reporting to the medical board: “The costs of [reporting] are prohibitive, and entities and physicians use all possible means to avoid the time and money that are involved in the lengthy, contentious processes” [52]. This study found that non-hospital entities were even more reluctant to investigate dyscompetence and pursue discipline: “Health plans and medical groups are generally passive and depend upon hospitals and medical boards to bear the heavy burden of pursuing physician discipline.” The steep costs of physician discipline, however, must be weighed against an organization’s direct and indirect costs of physician dyscompetence. An academic medical center reported that in a 5-year period a single neurosurgeon was responsible for 12 of the 23 lawsuits in which its neurosurgeons were named. This neurosurgeon “departed from the [department] during the study period” [58].

The experience of peer review here also challenges Berwick’s sharp 1989 distinction between “the truly avaricious and the dangerously incompetent” from “the rest of us.” The threshold of adequate performance is not clear-cut. Setting a threshold for competent performance is a very complex task, even when limited to narrowly defined clinical competence [59]. Remediation for physicians above the threshold for formal reporting is often possible and appropriate at the organizational level without triggering costly and cumbersome high-stakes due-process procedures. Some medical centers have recently developed novel processes to address performance issues that are clustered above the thresholds appropriate for peer review [60, 61].

Furthermore, the distribution of performance outside of standardized testing settings is not likely to be a normal, bell-shaped curve, as assumed by both critics and proponents of peer review. Such curves are a product of standardized test design [62]. Stephen Jay Gould’s examination of performance in baseball helps illuminate the challenge in medicine. As baseball improved in the twentieth century, variation among the better batters decreased, clustering against a “right wall” of high performance such that the right tail was truncated [63]. The left skew of poor performance, on the other hand, was relegated to the minor leagues. Most measures of real-life performance in Olympic sports [64], boating [65], and medicine [66, 67] also yield competence curves in which good and very good performance clusters against a right wall. Simply put, “the distance between the average and peak performance narrows over time” [68]. Medicine’s unresolved problem is the long left skew of dyscompetent performance. The left tail’s heterogeneity was evident in the system described here; it has also been reported from the PACE experience [69]. A study of physicians disciplined by the Medical Board of California described those physicians as “a subset of a poorly defined universe of physicians with significant practice deficiencies” [70].

Self-regulation within the profession

The data in this case study address Brennan’s observation about the lack of evidence linking the removal of “bad apples” with improvement in overall quality of care. Similar improvements in overall mortality might obtain only in settings with high concentrations of physician dyscompetence and high-risk patients dependent upon those physicians. Although our results are not generalizable to the population at large, they do raise questions about the harms of dyscompetence. The exact magnitudes of physician dyscompetence and consequent patient harms are understudied and uncertain. The 1990 Institute of Medicine report cited earlier estimated that at any point in time, 5–10% of U.S. physicians are incompetent, impaired, or otherwise routinely perform below the standard of care [3]. More recent reviews concur [6, 11], suggesting that licensure alone is inadequate to assure acceptable levels of competence and performance. Canadian provinces, the United Kingdom, and Australia have profession-led or governmental systems for screening the performance of all physicians, screening a random sample, and/or targeting certain higher-risk categories, e.g., physicians over a certain age [71]. Physician accountability in the U.S. is non-systematic, dependent largely upon malpractice and the market [72]. On its face, maintenance of certification has limited impact since 20% of U.S. physicians are not certified by a specialty board [73].

The great bulk of problem physicians congregate where oversight is weak or non-existent, such as solo practice [74]. Hospital medical staff committees no longer have line of sight into the performance of most ambulatory care physicians. Oversight in nursing facilities and other long-term care settings has always been rudimentary [75]. These latter settings serve particularly complex patients who are vulnerable to standard of care violations. Variation in physician attentiveness and quality in long-term care settings is well-known [76]; the patient consequences of dyscompetence are unstudied.

Of note, the number of staff terminations in our study far exceeded the subsequent number of licenses lost, so as with the 2017 Veterans Administration study, it is likely that the majority of the problem physicians who left California’s prison system are now practicing elsewhere, with or without adequate oversight. The dilution of these dyscompetent physicians into larger state and national systems diminishes the measurable aggregate impact of their lapses in care, but it does not render negligible the harms they may inflict on patients. The peer review committee in this setting took action against physicians when the committee members concluded that these physicians were likely to continue committing egregious lapses in care and patient harm. Stakeholders may differ as to who is responsible for preventing such harm, whether from these physicians in particular or the broader 5–10% cited above. In its 2005 findings regarding physician incompetence in California prisons, the court identified “a single root cause of this crisis: an historical lack of leadership, planning, and vision by the State’s highest officials” [16]. On the other hand, the public firmly believes that the medical profession bears responsibility for disciplining itself [77].

Organizations such as hospitals and large medical groups can manage physician entry criteria and oversight, including remedial training and discipline as needed. Although most may still be reluctant to bear the cost and controversy of formal expulsion, some have fully integrated these processes with their quality improvement efforts. Respectful and conscientious peer review with ample procedural safeguards should no longer be denigrated as fallacious bad-apple thinking. Our broader challenge in the U.S. context is managing professional self-regulation outside the boundaries and scrutiny of these organizations.

Limitations

We acknowledge that generalizations from this experience merit caution. Our report carries all the limitations of a retrospective case study, marked by an atypical setting, the extraordinary nature of the scandal and the subsequent institutional response, and a high concentration of problem physicians. We have not detailed the CDCR’s specific accountability processes, but good practical accounts of mortality review [78] and peer review [49, 79] are found elsewhere. Problems with the reliability of peer review are well-known [80], but our experience adds credibility to reports that structured peer review can distinguish good quality from bad [81, 82]. We have not elaborated on the workforce turnover among nurses or on the nursing-specific interventions. The sequence of accountability and quality improvement interventions in nursing paralleled those in medicine, but nurses were rarely responsible for lapses leading to deaths. We were not able to investigate the relationship of physician board certification to performance, nor did we have data on physicians’ beliefs and experience, which would enrich our explanations. Professional identity, including one’s commitment to the professional values and professional self-regulation, is multidimensional, varies widely across the profession, and interacts in complex ways with one’s work group and organization [83]. Physician leaders and staff in other settings may respond quite differently when faced with crises of professional legitimacy; their accountability systems, moreover, are likely to be far less robust. In addition to peer review with an enabling governance structure and robust legal support, initiatives here included thorough investigations of all deaths, disciplined root cause analysis, generous use of the PACE program, and investments in physician leadership training. Finally, we did not have data to age-adjust mortality rates beyond 2009 and thus evaluate the impact of then-emerging quality improvement initiatives; although reports cited above suggest they were effective, their impact in the subsequent time frame is beyond the scope of this study.