Three precedents bear consideration as effective systems to select improvement projects. Although two were successful quality improvement systems, they failed to persist and become today’s North American gold standard models. Williamson’s Achievable Benefits Not Achieved (ABNA) system to identify and prioritize potential projects (12) has a remarkable track record among alternatives (13). Similarly, the so-called Denver Connection of the same era is a story of successful amalgamation and reorganization of two hospitals in a way that put quality improvement supported by real-time performance data
analysis as the centerpiece of medical staff departmental meetings and continuing medical education (encouraged by board-level involvement while the usual array of advisory committees was eliminated) (14). Finally, the Institute of Medicine’s (IOM’s) Model Process for semiquantitatively ranking alternative projects is instructive for its mathematical approach (15).

Dr. John W. Williamson developed systems for health accounting and ABNA during an impressive body of work that spans decades on the faculty of the Department of Health Services Administration at the Johns Hopkins University School of Hygiene and Public Health, Medicine and Medical Informatics at the University of Utah School of Medicine, Regional Medical Education Center of the Salt Lake Veterans Affairs Medical Center, and service on government commissions. Health accounting, conceptualized in the early 1960s, is “a management model to integrate continuing education and patient care research into an ongoing cyclic function to systematically improve the quality of medical care” (16). It is an evidence-based outcome-focused approach that selects project priorities through ABNA, a formal, efficient process refined in the 1970s. Although proven effective and cost-effective in a wide range of applied research and demonstration projects at the American National Institutes of Health, Veterans Administration, and elsewhere, Williamson acknowledges that the most successful application of his system to enhance national quality is in the Netherlands (16). The fact that Williamson’s name and work are unfamiliar to so many throughout North American hospitals and healthcare leadership is hauntingly reminiscent of the history of W. Edwards Deming. Deming’s influence took decades to return to North America, heeded by American manufacturers only after Japan capitalized on Deming’s leadership to outperform their American counterparts (and later heeded by health service organizations decades after that!) (17). Williamson stresses the following:

Consistent with concurrent surveillance methods that have become a mainstay of contemporary infection surveillance programs, Williamson recognized long ago that chart-based audits as a basis for quality assurance may be misleading and severely limit the potential impact of programs to improve quality (16). His ABNA process consists of selecting a team (ideally 7-11 persons, including “at least four knowledgeable and respected staff physicians” and representatives of other functional areas and the lay public) and then supporting that team through two 2-hour meetings 3 to 4 weeks apart. The first meeting is a training session simulating the later priority-setting session. A master list of potential topics developed during the training session, together with team recommendations on additional “data, literature, or consultation” required, provides an indication of support materials that will be needed at the second session. Ideally, these are gathered during the 3- to 4-week hiatus. There are seven tasks in the prioritysetting meeting:

Meanwhile, also in the 1970s, radical changes under the amalgamation of the medical staffs at Denver’s Swedish
Medical Center and Porter Memorial Hospital occurred. Radical change was needed because as Dr. William Robinson, Director of Medical Education, noted, “In spite of the hundreds of physician hours devoted to medical staff activities, little actually was accomplished. It was almost impossible to demonstrate that quality of care was in any way influenced by the physicians’ repetitive, duplicative, unrewarding medical staff activities.” The usual litany of committees was reduced to just three (executive, professional activities, and credentials). A reduced number of subcommittees composed of small numbers of individuals, the bulk of whom were not physicians, served these committees, and much of the quality-related work was shifted from committees to medical staff departments supported by the work of subcommittees or research and education department employees. Medical staff members were strongly encouraged to ask questions at their departmental meetings about quality issues, and then make policy decisions based on evidence delivered soon thereafter (answers supplied through real-time research capacity in their own institution) (14).

Although successful into the 1980s, by the turn of the 21st century, the Denver Connection was so far dismantled that it no longer even existed in the institutional memory of Porter Hospital’s present administration! Porter and Swedish, partners of the so-called Denver Connection formed in 1972, went their separate ways in 1992. These two hospitals serving health needs of southeast Denver had remained separate corporate entities, yet collaborated successfully for many years following formal merger of their medical staff organizations. That merger was initiated by the doctors, not by the administrators, for the purpose of improving quality of care. Administrative support grew following demonstrated successes, and the hospitals cooperated in division of complementary health services, instead of duplication of what the other offered solely to compete. That was before big business entered sickness. Ultimately, poor quality of management in emerging, large, managed-care corporations led to unexpected deficits in profitability of operations, and corporate vision shifted to preoccupation with that debt. A grass-roots anticompetitive way of serving the community’s health needs could not sustain itself in the face of powerful market forces and growing business empires awash in corporate debt. Personal ideologies in administrative leadership compounded the difficulty of making effective alliances, and mistrust grew where open communication once thrived. Key participants later interviewed conveyed a sense of loss and regret, a realization they participated in something very unique and beneficial that was lost for illogical reasons (“nobody had any appreciation what we were doing was special or unique … only in retrospect we came to appreciate the specialness of what we were doing.”). The influence of accreditation programs in this saga was noteworthy only for its lack of influence (18). It is tempting to speculate that this visionary effort thrived because it followed characteristic principles that seem to distinguish great companies from others (19) and that the Denver Connection ultimately fell when it strayed from core values and these fundamental principles.

The IOM Model Process to set priorities in health technology assessment addresses similar dimensions as ABNA, but in a more quasinumeric than nominal group consensus manner. The IOM priority score for each technology, instead of being assigned by consensus ranking, is calculated as ΣW₁lnS₁ + W₂lnS₂ + … + W₇lnS₇, where W_i represents the criterion weight and S_i the criterion score for (i = 1-7).

The first three criteria are objective measures; the remaining four are subjective and are addressed by one or more expert panels. Criterion weighting values are arbitrary choices; the process described has an expert panel select one criterion as least important, which then is assigned weight of one. Mean weights given by panel members for each of the remaining criteria, relative to the least important one, then determine the other six weights. After discussion of results to resolve any wide disagreement, results of a second vote are final. Pilot test results with small conventional and mailed response panels are examined in the IOM report, which gives the following values: W₁ = 1.6, W₂ = 2.25, W₃ = 1.5, W₄ = 1.2, W₅ = 2.0, W₆ = 1.5, and W₇ = 1.0. Logarithms of criterion scores are used to make the model multiplicative rather than additive, thus responsive to relative rather than absolute differences in scores (algebraically, the formula can be restated as ΣW_ilnS_i, which equals the product ΠS_iⁱ for i = 1-7). Subjective item scales, therefore, run from a value of one for least likely to five for most likely.

Health accounting provided a philosophy, the Denver Connection a forum, and ABNA a method all consistent with today’s emphasis on evidence-based practice and continuous quality improvement (CQI). What lessons should one take from these all-but-forgotten precedents? Williamson reflects on lessons learned from 25 years of experience (16), naming three premises on which quality assurance or improvement is based and five principles that evolve from it: