Safety in other domains is assessed by the incidence of accidents and injuries; aviation accidents, road accidents, lost time injuries at work and other types of mishap are counted and tabulated by various means. Defining these accidents is relatively, but not completely, straightforward; while a serious crash is unambiguous, there are many lesser road, rail and air incidents that cause minor damage or can be considered as near misses. Ideally, we would like to have a general index of safety, rather like rates of road or rail accidents, so that we could track progress over time and ask more sophisticated questions about the safety of different parts of the system and the factors that increased or degraded safety. However, this reasonable and worthy objective presents a number of problems, which have been well summarized by Peter Pronovost:

A prime challenge in measuring safety is clarifying indicators that can be validly measured as rates. Most safety parameters are difficult or impossible to capture in the form of valid rates for several reasons: (1) events are uncommon (serious medication errors) or rare (wrong-site surgical procedure); (2) few have standardized definitions; (3) surveillance systems generally rely on self-reporting; (4) denominators (the populations at risk) are largely unknown; and (5) the time period for exposure (patient day or device day) is unspecified. All of these may introduce bias. Creating measurement systems that are relatively free of such bias would be costly and complex.
(PRONOVOST, MILLER AND WACHTER, 2006)

Defining harm is a particularly difficult issue in healthcare for a number of reasons. First, in other arenas, establishing cause and effect between accident and injury is reasonably straightforward. In contrast, patients are generally, though not always, sick and separating the harm due to healthcare from that due to illness is often difficult. Second, some treatments given in healthcare are necessarily ‘harmful’ to the patient; radiotherapy and chemotherapy are two obvious examples. Third, harm from healthcare may not immediately be detected or may only gradually become apparent. In fact, a cause celebre of medical error – the chemotherapy overdose of Boston Globe reporter Betsy Lehman – was only discovered on a routine review of research data in the clinical study in which she was a participant. Finally, even if a patient is harmed, this does not necessarily point to any deficiencies in care. One patient may get pneumonia because of a major lapse in basic care; another may receive exemplary care but still succumb to pneumonia.

The issue of denominators is also critical:

Deciding on the best denominator is an added dilemma in the error rate equation. In general, the denominator should quantify exposure to risk for the outcome of interest. For example, when a patient who is hospitalized experiences a narcotic overdose, is the appropriate denominator the patient or patient day, the prescribed or dispensed doses, all administered medication doses, or all administered narcotic doses?
(PRONOVOST, MILLER AND WACHTER, 2006)

If you consider this for a moment, you will see that the choice of denominator makes an enormous difference to the error rate and to the interpretation of the standard of care. Supposing a patient is given 10 different drug doses per day, stays in hospital for ten days and sustains one adverse drug event from an overdose. You could say, well that’s 100 doses over the admission, that’s a rate of 1%. Certainly serious, but it doesn’t look too bad. However, calculate by the day and the rate is 10%, and by the admission the average becomes 100%. Suddenly what looks like a technical issue for statisticians takes on new life.

The basis of the diagram is the classic distinction between the structure, processes and outcomes of healthcare. Structures represent both physical structures (buildings and equipment), but also basic institutional characteristics such as the number and qualifications of staff (Donabedian, 2003). These characteristics can be changed, but generally only slowly, and the link between these factors and patient outcomes is not yet well understood. Some structural factors, such as staffing levels and the organization of intensive care have been linked to the safety and quality of care (Aiken, Sloane and Sochalski, 1998; Pronovost et al., 1999; Main et al., 2007). Human resource practices, which influence staff morale and working environment, have also been shown to relate to patient outcomes, even including hospital death rates (West et al., 2002). Lilford and colleagues suggest that these influences are mediated by a number of intervening variables (discussed below), such as morale, motivation and safety culture, which affect staff attitudes and behaviour which in turn affect the clinical work carried out.

Outcomes are changes in the health status of the patient, covering mortality, morbidity and more subtle changes in quality of life, patient satisfaction with care and changes in health related behaviours (such as giving up smoking). Safety outcomes are certainly top priority for patients and families. While you certainly might be concerned by observing errors in your care, your absolute priority is not to come to any harm, to at least leave hospital or emerge from treatment no worse than you were before. Some of the main adverse outcomes are infections, adverse drug events, pressure ulcers and surgical complications.

Death and surgical complications seem relatively unambiguous outcomes. However, some indicators of morbidity, such as wound infection, anastomotic leak and postpartum haemorrhage are difficult to define with precision (Lilford et al., 2004). Even death can pose difficulties of classification, in the sense that a death in hospital can simply mean the arrival of a terminally ill person who died shortly after admission. A death in those circumstances says nothing at all about the quality or safety of care in that hospital.

Outcomes are determined by a combination of the patient’s underlying condition and the care they actually receive. Any kind of outcome indicator, such as wound infection is only a very indirect reflection of the safety and quality of care provided. Comparing units or institutions on such indicators is therefore problematic, as any differences may simply reflect differences in patient populations as well as other factors, such as data quality and random variation. Case mix adjustment, in which rates or mortality or morbidity are statistically adjusted to allow for differences in patient population, is widely used but there will always be some uncertainty about the validity of comparisons based on such data. This is not to suggest that case mix adjustment is not valid or that comparisons should not be made, only to point out that the differences that emerge need thoughtful interpretation (Bottle and Aylin, 2008).

Issues of case mix adjustment matter much less however, if a unit or institution simply wishes to track its own progress over time and use the mortality or morbidity data as a stimulus and measure of improvement. If one makes the reasonable assumption that the patient population is relatively stable over time, then an organization can certainly use mortality or morbidity data as an indicator (Bottle and Aylin, 2008). Any change does reflect, albeit imperfectly, a corresponding change in safety and quality, though it may be difficult to identify which improvements were critical to the overall success.

Donabedian describes clinical processes as ‘the activities that constitute healthcare – including diagnosis, treatment, rehabilitation, prevention and patient education’. This is basically what healthcare professionals actually do, though it also includes the actions and care provided by patients themselves and their families. It is obviously impossible to capture the quality of fluid, day-to-day clinical work in its entirety. However, it is possible to select and capture specific clinical processes which are clearly indicated, supported by underlying evidence and, ideally, would be agreed as desirable by the clinicians caring for those patients. Examples of such measures would be the use of beta blockers after myocardial infarction and the timing of antibiotics after pneumonia.

When considering safety and quality improvement, process measures have a number of advantages, whether one is comparing organizations or simply monitoring change over time. Richard Lilford and colleagues suggest that monitoring clinical processes have several advantages over outcomes if the primary aim of measurement is to guide efforts to improve performance:

• Process measures focus on violation of agreed evidence or standards, so that deviations are clear cut.
  
• Measurement can be made close to the point of delivery of care, overcoming the delay between intervention and outcome.
  
• They can be applied to all institutions, not just the ‘worst’ 1, 2 or 5%, and therefore offers the hope of improving the average quality of care, yielding far bigger gains to the public health (Lilford et al., 2004).

We should note however that, in practice, it has proved difficult to show that improvements in processes produce improvements in outcomes. For instance, only weak associations have been found between process and outcome for myocardial infarction, a range of acute medical conditions, hip fracture and stroke (Lilford et al., 2004).

As this book unfolds, you will see that multiple factors potentially affect the safety and quality of care delivered to patients. Teamwork, the performance of individuals, the use of technology, the conditions in which people work, the ethos and culture of the organization may all be relevant. These are the ‘intervening variables’ in measurement terms. They may only affect care indirectly, but are also potential reflections of the safety of an organization and also of its potential to improve care in the future. We should note however, that assessing safety by what has happened only tells you how safe asystem has been in the past and does not tell you how dangerous it is now or will be in the future. Looking further ahead, at the possibility of deriving measures which are more reflective of the likelihood of harm, we might wish to assess the levels of hazard, the ability of systems to recover when errors occur and indices such as safety culture or staffing levels which might reflect overall systems safety. We will examine some of this work at a later stage. For now it is sufficient to note that although many of these factors are almost certainly relevant to safety and quality, the precise form of leadership, for instance, and the way it impacts on safety of care in practice remain to be elucidated.

Both measures of harm and assessments of failures in the process of care may reflect overall levels of safety. Failure to give appropriate care may or may not lead to harm, but it certainly seems reasonable to class these failures under the general heading of safety. These process measures however, seem similar if not identical to broader quality measures of effectiveness, reliability and efficiency captured in numerous studies of the quality of care. Does this mean that safety measures are nothing more than quality measures under another name? Not exactly, though when we examine the level of process rather than outcomes, the same measures may reflect both safety (in the sense of potential for harm) and other aspects of quality (efficiency, effectiveness and so on). The reason that this overlap has been slow to emerge is, to my mind, because our concern with safety was initially driven by relatively rare events with serious consequences.

Quality assessments have always been directed at overall standards of care given to populations of patients. In contrast, patient safety initially focused on rarer, often tragic events which had not been captured by traditional assessments of quality. As safety was more systematically studied however, it became clear that the frequency of error and harm were much greater than previously realized and that the safety of all patients needed to be addressed. No longer were we trying to prevent rare events, instead we were facing an epidemic of infection, adverse drug reaction and complications, together with a host of other rarer and less predictable incidents. The gradual rapprochement of these concepts, and the need to maintain focus on both, has been eloquently expressed by Vahe Kazandjian and colleagues (2008) in their paper ‘Safety is a part of quality: a proposal for a continuum in performance measurement.’ This is a long passage but well worth quoting in full:

Indicators of quality assess magnitude (events, frequency of processes, etc.). Through both statistical and clinical decision-making processes, changes in the magnitudes of measurement over time assist organizations in identifying priorities for improvement. For that reason alone, comparative analysis remains essential, be it to an organization’s past performance or the performance of peers (while adjusting for confounding variables, if necessary). In the case of safety indicators, however, the philosophy appears entirely different: adverse events, often described with terms ranging from ‘never events’ to ‘near misses’, may not require comparative data. Indeed, it could even be proposed that for some safety measures one event is too many. Risk management and risk managers are primarily focused on those singular outcomes. For example, while it was not necessary to establish how many wrong doses of chemotherapy drugs were administered to a patient who developed kidney failure, it was sufficient to know that one patient had developed kidney failure because of wrong chemotherapy dosage. It is the very nature of safety measure events to occur with low frequency, although the associated outcomes can be catastrophic.

As the scientific literature has focused increasingly on the importance of near misses, even the potential for errors, a basic reconsideration of the initial distinction between ‘quality’ and ‘safety’ indicators seems in order. Seminal works on errors resulting from the provision of a service in any industry, have well established that errors can occur during any process. Therefore, it appears of much greater importance to understand the environment, structures, processes, as well as the attitudes of the people themselves rather than the outcomes defined as either quantifiable or qualifiable events.

This accounts for the rapprochement between the concepts on the one hand and the mechanics of defining and designing quality indicators on the other. When analysis of a process is required to understand whether best knowledge at the time (evidence-based practice) was followed or whether the process suffered from inherent predispositions to undesirable outcomes (such as errors), the very distinction between ‘quality’ on the one hand and ‘safety’ indicators on the other becomes noticeably blurred.

(REPRODUCED FROM JOURNAL OF EVALUATION IN CLINICAL PRACTICE, KAZANDJIAN ET AL. “SAFETY IS PART OF QUALITY: A PROPOSAL FOR A CONTINUUM IN PERFORMANCE MEASUREMENT”. 14, NO.2, 357–358, 2008.)

We have already discussed record review and reporting of adverse events as methods of assessing adverse events at a particular point in time. We will now briefly consider whether they can also be used routinely to monitor safety over time.

Patient safety is of course underpinned by large-scale studies of adverse events. If we want to monitor progress over time, then surely we should repeat these studies, whether on a local or national level. At a national level though, the simple fact is that no country has had the courage to repeat a study of the incidence of adverse events as a formal comparison; The Netherlands, however, has carried out a major study (Zegers et al., 2009) and a follow-up study is planned to assess progress on patient safety.

Case note review is sometimes viewed as time consuming and comparatively expensive. Nevertheless, with experience and refinement and the development of training packages (Olsen et al., 2007), it can be carried out relatively inexpensively, producing systematic, detailed analyses. A few organizations, such as Royal North Shore in Sydney (Harrison, personal communication) carry out formal, annual case note reviews and use these as the basis of their quality assurance and improvement systems. Record reviews could be repeated over time, and trends studied, particularly as we would now be able to define and monitor specific types of adverse events rather than just assess the overall rates. Reliability and validity of judgement of adverse events is not as good as we would wish but could certainly be improved if specific definitions of particular classes of adverse events were developed.

There is another class of instrument which is sometimes put forward as a measure of safety, namely ‘trigger tools’. Essentially medical records are screened, by a clinician or sometimes electronically, for certain triggers which might indicate that an adverse event has occurred. These might include a return to the operating theatre, a death in hospital or more specifically a low platelet count or the need for renal replacement therapy. Trigger tools have been much used in programmes run by the Institute of Healthcare Improvement, such as the Safer Patients’ Initiative, which will be discussed later. This kind of instrument can certainly be useful in providing a ‘panoramic view of safety’ (Pronovost, Miller and Wachter, 2006) to flag up worrying trends and areas. Whether the trigger tool is a measure of adverse events is not really clear; hospitals might claim ‘we achieved a 50% reduction in adverse events’, when what they mean is that they had 50% less triggers, which is not quite the same thing. Trigger tools are very similar to the Stage 1 of case record review, a screening tool for potential problems. They are certainly useful as a screen, but the subtle shading into the use of triggers as measures is a little disquieting.

Some safety events are rare. Deaths from injecting intravenous drugs into the spinal cord are, thankfully, very rare. These are the most prominent, most disturbing safety events which most closely correspond to the ‘accidents’ of other domains. These events are captured in the list of 28 ‘never events’ drawn up by the National Quality Forum in 2004, and since adopted by many organizations as a safety target. We will never be able to systematically measure ‘never events’ and hopefully will not need to. Identification of these rare but terrible events will always have to rely on reporting, at least until reliable ways of searching electronic medical records emerge.