Fig. 2.1
A twelve-step program: surgical pathology as a production process
Amended Reports and practical errors: Amended reports in surgical pathology are like accident reports . As sources for a taxonomy of defects, amended reports particularly help practitioners study practical errors in surgical pathology. They highlight the sorts of defects that lean production policies and procedures help decrease or eliminate in the surgical pathology production processes .
Information theory and error: In terms of Claude Shannon’s mathematical theory of communication [77], observer variability is variation in signal reception. Shannon’s theory, on which computer programming is based, predicts that getting from antecedent potential message to subsequent actual message always entails making errors [19, 77]. Information theory, as worked out by Shannon and his colleagues, provides a framework within which to think about the making of diagnostic message, the central task of surgical pathology.
Interpretative error in the surgical pathology information flow: Error arises in the information flow (Fig. 2.2) either by commission, not getting the information that is signaled from slides right, or by omission, missing the potential information that the slides have to offer. Practical and interpretative errors are distinct sorts of defects. They are studied differently [39]. In this chapter, we focus on amended surgical pathology reports as the most convenient source of information about practical errors and reviews as the most available source for rates of interpretive errors.
Fig. 2.2
Information flow in surgical pathology
Root causes: Root causes are primary defects that occur earliest, farthest upstream, in the practical production processes. There are more steps in the production process (Fig. 2.1) than there are in cognate information flow (Fig. 2.2). Practical errors are, It follows that practical errors are most often the root causes of errors in surgical pathology; this is particularly true of errors that can be prevented. For this reason, root cause analysis of errors in the surgical production process is the key to developing practical counter measures to improve the process’s performance.
Cognitive errors: Information theory gives the best account of how errors about facts arise in the interrogation of tissues. As presented in reports, surgical pathologists’ mistaken beliefs about matters of fact and classified states are cognitive errors . Nicholas Rescher observes: “specifically cognitive error roots in our human need to resolve issues of thought and action in conditions of imperfect information” [65] or, in the foundational insight of the information age, articulated by Claude Shannon, any sort of information is always imperfect [19, 77].
Information theory maps surgical pathology error: As outlined in Fig. 2.2, surgical pathologists search tissue samples for answers to questions: in the most frequently considered example they question whether or not a malignancy is present, what sort of neoplasm it may be, which features predict its behavior, and whether characteristics are present that indicate a particular therapy. Pathologists’ reports convey information about primary matters of fact: a tissue sample does or does not contain lung cancer; primary matters of classification: a lung cancer is or is not adenocarcinoma; they also inform about secondary matters of fact: an adenocarcinoma does or does not appear within vessels or lymph nodes; and secondary matters of classification: a particular sample of adenocarcinoma of the lung has or lacks specific molecular signatures that indicate susceptibility or resistance to specific chemotherapeutic agents.
The information stream: Shannon discovered that, in the flow of information, a message is selected at an anterior (upstream) point then reproduced at a posterior (downstream) point [77]. This sequence always runs from information sources to messages. In Fig. 2.2, we match the Shannon sequence to surgical pathology terms. From an information source (human tissue) of antecedent, potential information, a transmitter (the tissue sample) selects antecedent message, but the transmitter emits both a signal (anterior, potential information) and noise (mixed-in nonsignal that yields nonmessage). From this mix of signal and noise, receivers (surgical pathologists) select received signals (diagnoses, in Shannon’s terms, subsequent message), which they then pass on as posterior, actual messages (reports).
Interpretive errors and uncertainty: This is a reality beneath interpretative error : any communication system that fits Shannon’s pattern entails uncertainty. Every second, posterior, actual message (every reported diagnostic claim) has some chance of being wrong (for pathologists, either missed diagnoses, wrong diagnoses, or misclassified diagnosis). Quantification of this chance of being wrong calculates greater or lesser likelihood of interpretive error . This is the underlying variation that review of diagnoses aims to define.
Surgical pathology is also an interpretative framework: At this point, it is worth observing that, besides being a production process and a pattern of information flow, surgical pathology is also a conceptual structure. This framework is a group of classifications or taxonomies. The taxonomies aim to transmit the diagnostic, prognostic, and therapeutically relevant information that the production process creates. An act of interpretation places a received signal in a category within a classification. The characteristics of various classifications set limits to the reproducibility of the information. The variable applications of taxonomies also limit the validity, reproducibility, and detail of surgical pathology reports [39]. Taxonomic variability, like intraobserver variability, always lurks in the background, when we think about surgical pathology error.
Validity, reproducibility, and detail: In studies of interpretative diagnostic variability, three properties of measurement—validity, reproducibility, and detail—come into play again and again. Validity is the extent to which measurements correspond to real states of how things are. Increasing validity depends on decreasing systematic differences between observed appearances and real states of being. Reproducibility depends on how often repeated measurements return the same result. Random variation sets limits to reproducibility. Detail depends on the amount of information that measurements provide. The degree of detail determines how much an observer knows about what he has measured after he has measured it. Keeping these three attributes in mind aids orderly study of error in surgical pathology . Importantly, interpretative discrepancies produced by review of surgical pathology diagnoses combine differences in validity, with variability introduced by differences in reproducibility, and variation in matters of detail. In review discrepancies, these three contributing features are usually inseparable.
Surgical pathology is, in addition, a dynamic scientific discipline: The scientific discipline is the larger context that surrounds study of both process and interpretative, error. As a discipline, surgical pathology has assimilated increasingly elaborate techniques that assist in acquiring and processing information. These ancillary techniques find information both on the slide (as most prominently from immunoperoxidase stains) and from handling the sample in different milieux (as most prominently in molecular tests). The information gleaned from samples by converging morphological, quasi-morphological, and molecular techniques yields the explanatory criteria on which the informative classifications base themselves. In particular, sources of information besides histopathological morphology, especially immunohistochemical profiles and molecular motifs, increasingly influence classification. In this wider context, complexity leads to error . As we will emphasize below, increasing practical complexity of process compounds increased complexity of interpretation [48, 49, 50].
Oversimplification: Surgical pathologists always generalize from particular findings on slides to general diagnoses of disease states. As actual message, emerging from the information stream, pathology reports inevitably oversimplify. Another of Claude Shannon’s seminal insights is that informativeness of a message increases in proportion to its vulnerability to disproof [19, 77]. This is the juncture where detail joins validity and reproducibility in the trio of important attributes of surgical pathology information. As they compose reports, pathologists arrange information content. They may reduce complex data presentations to simple ones; they may proliferate qualifications; or they may take away informative detail. In these three ways, they limit, obscure, or decrease the amount of information transferred to clinicians. With these strategies, pathologists try to prevent error by hedging; they trade off informative message for evidential security. This tactic fails when it drains reports of detail, exactness, and precision [68].
Errors of commission and omission: Errors of commission are misleading messages; these diagnostic failures (wrong diagnoses) appear among positive reports. Errors of omission fail to receive anterior diagnostic message. Errors of omission hide among negative reports. To recognize the commission:omission dichotomy, interpretative error detection must combine two different review approaches: (i) review (often redundantly called double review) of positive reports at risk and (ii) review of negative reports in high-risk categories of specimens [69].
Review in search of error and hindsight bias: Review checks the information transfer step in which the pathologist moves from receiving the signal on the slide to composing a report . Important conditions of review are when, where, how, and by whom review is done. Hindsight bias is made up of the systematic differences between looking forward at a new set of facts and looking back at an old set. Six systematical differences between the initial diagnostic event and the review event define various mixes of hindsight bias. The first of these distinctions is between internal and external review. Internal review is carried out within the practice in which the diagnoses under scrutiny were originally rendered. Pathologists in other practices perform external review. The second distinction is between pre-sign-out review and post-sign-out review. Pre-sign-out review takes place before a report is issued. Post-sign-out review happens after reports are released. A third difference is between conference review and non-conference review. Conference reviews are those that surround multispecialty gatherings at which cross-specialty agreement on diagnosis, prognosis, and therapy are sought. A fourth distinction appears between expert and non-expert review. Expert review is by a pathologist with increased knowledge and experience with the sort of diagnoses under review. A fifth pertinent difference is between blinded and non-blinded reviews. Blinded reviews are those reviews by pathologists who possess no more information about a case than the primary pathologist; indeed a blinded reviewer sometimes is given even less case-specific information. The last of these variations in review schemes, but probably not the least important, is that between focused and unfocused reviews. Focused review trains the reviewer’s gaze on specific sorts of specimens or diagnoses. Unfocused reviews either take all comers or check a defined fraction of cases without requiring that they be of specific specimens or types of diagnoses. The variable influences of these half dozen factors together make comparison of review discrepancy rates difficult.
Information sources about surgical pathology error: Because there are two sorts of error in surgical pathology–practical error and interpretative error–two kinds of studies yield useful information about surgical pathology error: classification of errors turned up by amended reports and sorting of discrepancy rates produced by view of surgical pathology diagnoses. The rest of this chapter summarizes observations about surgical pathology error that have emerged from those two approaches.
Amended Reports as a Source for a Taxonomy of Surgical Pathology Defects
Amendments: Because practical errors are more frequent than interpretive errors, root causes of amended reports more often map to the twelve-step production process (Fig. 2.1) than to the six-step information flow (Fig. 2.2). Mapped to either sequence, amended reports offer opportunities to study systematically both surgical pathology errors and the counter measures aimed to decrease them [1, 34, 35, 36, 86].
Amendments vs. addenda: To achieve semantic consistency, the alterations of surgical pathology reports made after they have been issued must be separated into dichotomous groups. One group is composed of amendments: all changes that were not purely additions of information. The other group is made up of addenda: altered reports that include only alterations that purely add information. Adherence to this dichotomy has proven necessary both to detect reports with errors in them and to separate error from other sorts of report variation [34–36].
Taxonomic consistency: Across many institutions, classifiers of altered reports have been able to agree on four defect categories and to sort consistently into these categories [34, 86]. The categories are: misidentifications, specimen defects, misinterpretations, and residual report defects. Report defects are residual because they classify the amendments that are left over after misidentifications, specimen defects, and misinterpretations have been specified.
Misidentifications fail to designate accurately patients, tissues, laterality, or other anatomic localization. Specimen defects include submitted specimens that are lost, those of inadequate sample volume or size, those with absent or discrepant measurements, and those with inadequately representative sampling, as well as, importantly, and less intuitively, those with absent or inappropriate ancillary studies.
Misinterpretations fail to state diagnostic information accurately. They have an internal structure more complex than misidentifications and specimen defects. This complexity divides misinterpretations into three subtypes. The first subtype includes errors of commission; these are false-positive diagnoses, or overcalls. This sort of amendment registers the retraction of wrong information. The second subtype is made up of errors of omission; these are false negatives or undercalls. This second sort of amendment registers either failures to recognize accurate information or initial loss of information that later was found to reside in the sampled tissues. The third subtype is confusion or conflation of relevant, similar, but distinct diagnostic categories. The findings in the third subtype are not over- or underdetermined. Instead, they are misnamed diagnostic designations. The three misinterpretation subtypes, in turn, relate to two levels of diagnostic message: primary level amendments register failures to distinguish positive from negative, malignant from benign; and secondary level amendments mark failures to characterize subordinate diagnostic features appropriately. The subordinate secondary diagnostic features affect clinical context, prognosis, or susceptibility to specific therapies. Most often these secondary characteristics are grade, stage, state of surgical margins, or lymph node status in specimens resected for malignancy.
Report defects: After misidentifications, sample defects, and misinterpretations have been excluded, the residual category in the taxonomy is report defects . Report defects also present themselves in three subtypes: (i) missing or erroneous non-diagnostic information—absent or wrong information about practitioners, procedures, billing codes, etc., (ii) dictation or transcription errors—typographical errors in the strict, proof-reader’s sense, and (iii) failures or aberrations in electronic report formats or transmissions—the miscues colloquially called computer glitches. These report errors are all defects in product, but they have in common that they do not directly affect diagnostic information. Misidentifications, misinterpretations, and specimen defects, in contrast, all directly interfere with the diagnostic message itself. Report defects, however, are not unimportant. Although they fail to muddle message directly, as, they harm the information flow by reducing information redundancy [19]. Redundancy is the informative context in which the text of any message always arrives.
Root causes of amendment types: In the twelve-step production process (Fig. 2.1), the root causes of misidentifications and sample defects appear mostly in the early steps of the surgical pathology process, during specimen collection and sample processing, but, in a minority of instances, they pop up later. The root causes of misinterpretation focus in the middle of the process, when the case is on the pathologist’s desk. Root causes of residual report defects inject themselves into the process at multiple points, but they also tend to cluster at its beginning, before the case reaches the pathologist, and at its end, after the pathologist has settled on diagnostic interpretations.
Application of the amended reports taxonomy: Uniform application of this taxonomy allows consistent monitoring of amended reports among institutions and also within an institution over time. Important to process improvement, when amended rates are followed longitudinally over time, they also evaluate the success or failure of interventions aimed to reduce errors that amendments identify [1, 34–36].
Three characteristics of defect discovery: The amendment taxonomy reveals four characteristics surrounding the discovery of defects. First, the more observers monitor amendments, using the dichotomous definition, the more amendments are identified, usually at the expense of addenda. Second, clinicians discovered most misidentifications; pathologists found most misinterpretations; but discovery of specimen defects were scattered among different observers and discoverers of report defects usually remained anonymous. Third, clinicians most frequently detected misidentifications, and, initially, conference review was the most fruitful mechanism for detecting misinterpretations. Finally, conference review discovered, in various settings, between just over 40 % to a little more than 80 % of all misinterpretations that produced amendments [34].
Effects of lean interventions: In a large surgical practice that accessioned 45–50,000 specimens each year, real time editing of altered reports, undertaken together with changes in process aimed at reducing and preventing the underlying defects, had the following consequences over a 5-year period [34]. Initially, active monitoring caused amendment rates to rise, from approximately 5-amendments/1000 reports to 10/1000 as altered reports were consistently defined as amendments or addenda. Subsequently, as monitoring continued and counter measures were applied, amendment rates fell back to the 5-amendments/1000 reports level. Lean interventions in surgical pathology report production then caused misidentifications to fall from 16 to 9 % of all amended reports. Despite similar interventions, however, the fraction of amendments caused by specimen defects remained at about the same magnitude (< 11 %) and continued to be highly variable from year to year. In contrast, the fraction of misinterpretations fell dramatically, from 18 to 3 % of all amendments. This fall was associated first with introduction of pre-sign-out review of all breast and prostate cases, then, in addition, cases of some gastrointestinal tract lesions. Finally, and reciprocally, as misidentifications and misinterpretations fell, the residual category’s report defects increased its fractional contribution, from 64 to 83 % of all amendments [34].
Lessons from root cause analysis: When case-by-case root cause analysis of amendments assessed success or failure of interventions , three findings emerged: (i) efforts to reduce misidentifications at the specimen collection level (where most of these errors occurred) had a measurable, but modest beneficial effect, (ii) extensive standardization of specimen accession and gross examination reduced specimen defects surrounding ancillary testing, but not specimen defects overall, and (iii) introduction of internal pre-sign-out review of all breast and prostate and some gastrointestinal cases was specifically associated with a reduction in misinterpretations [36, 34].
Amendments vs. addenda: The problem with amendment monitoring caused by misclassification of amendments as addenda continued over time. During active monitoring, 10 % of so-called addenda have consistently turned out to be amendments [35, 36].
Q-PROBES study of amendments using validated taxonomy: In 2011, as part of a College of American Pathologists Q-PROBES study, 73 participating institutions analyzed almost 1700 amendments over a 12-week period [86]. The Q-PROBE study’s salient results are presented here to complete our account of how amendments characterize errors.
The taxonomy-classified amendments effectively across 73 institutions: Using the taxonomy, Q-PROBES subscribers classified 1665 of 1688 amendments (98.6 %). In contrast to our large institutional experience , however, the fractions of misidentifications (13.3 %), specimen defects (13.7 %), and misinterpretations (14.6 %) were about equal [86].
Amendment rates: Median defect rates among Q-PROBES participants hovered around 5-amendments/1000 published reports: the aggregate defect rate was 4.7-amendments/1000 cases and a median participating institution’s defects rate was 5.7/1000. This median amendment rate is similar to the 5/1000 experienced in our single institution monitoring. However, among the 73 Q-PROBES study participants, the range around this median was wide; it extended from 0.9/1000 to 13.5/1000 amendments/reports issued [86].
Misidentifications and sample defects: In the Q-PROBES study, among 225 misidentifications, 31.5 % were of patients, 20.0 % of tissue type, 23.0 % of laterality, and 25.5 % of anatomic localization. Among 231 sample defects, more than three-quarters (77.4 %) involved ancillary testing and the rest mostly involved gross and microscopic sampling [86]. The association of sample-related defects with misdirected or failed ancillary testing is a phenomenon also observed in our single institution’s longitudinal monitoring [36].
Misinterpretations: Analysis of 247 primary and secondary misinterpretation amendments found only 5.7 % false positives and only 11.8 % false negatives. These fractions are dramatically different from our single institution longitudinal experiences . The difference stemmed from very different rates of diagnostic relabeling. In the Q-PROBES cohort, 44.1 % of misinterpretation amendments were attributed to confusion or conflation of similar but distinct diagnoses (misnaming). The Q-PROBES subscribers also produced a different pattern of interpretative errors from that found in the single institution experience. Misinterpretation amendments among the Q-Probes study participants were revised mainly for secondary features in amended reports of malignancy. These amendments usually changed grade or margin status [86].
Residual report defects: Among the Q-PROBES study participants, as in our long-term experience at one institution, the most common causes for amended reports were residual report defects: typographical errors , missing nonidentifying, noninterpretative report attributes, or wrong nondiagnostic report information [34, 86].
Anatomic sites of origin of specimens that produce amended reports: In the Q-PROBES study, the most common tissues of origin for defective reports were the most common sites sampled: the skin, breast, and gastrointestinal tract. Submissions from these sites were about equal defect contributors (18.2, 17.7, and 18.1 %) [86].
Benchmark amendment rates from Q-PROBES study of amendments: The Q-PROBES study of amended reports yielded two benchmarks: First, with a 5/1000-defect rate, the current surgical pathology production process is a ‘three sigma’ production system for surgical pathology reports. Second, median rates of misidentifications and misinterpretations are fairly consistent. These two rates both run below 1/1000 and are about equal: 0.6 amendments for misidentifications/1000 reports and 0.8 amendments for misinterpretations/1000 reports [1].
Defects in the surgical pathology production process as normal accidents [48]: Findings about surgical pathology errors uncovered by root cause analysis of amendments agree with studies of other production processes in different settings [48–50]. From studies in a variety of complex production processes, Charles Perrow defined untoward events, like those which amendments document, as normal accidents. He argued that these events occur in conditions of complexity created by interconnecting subsystems. In surgical pathology, the interconnecting subsystems are the preanalytic , analytic , and postanalytic phases of the report production process. A second error-inducing characteristic, tight coupling, then mediates the connection between subsystem derangement and damage to the final product. A third characteristic is concentration. In surgical pathology laboratories, high volumes of specimens are concentrated by converging from multiple collection sites to enter the production process. Once concentrated in the process, these specimens are also subjected to complex ancillary tests . Computer-enabled communication tightly couples pathologists with pathologist assistants, histologists, and clinicians. Perrow argues persuasively that such concentration, complexity, and tight coupling together inevitably amplify practical error [47, 49].
Eight contributors to normal accidents: All eight features that make systems prone to normal accidents are present in the surgical pathology production process [47]. In the following list we cite, next to each error-promoting feature, examples of its appearance in the surgical pathology setting:
1.
Proximity of components: proximity appears among specimen jars awaiting samples in endoscopy suites and in shopping bags full of many different patients’ skin biopsies arriving at accessioning stations
2.
Common-mode connections: large specimen gross examination stations are common mode connections when pathologist’s assistants examine in succession multiple partial mastectomy and lymph node dissection specimens or multiple colon resections during the same accessioning shift
3.
Interconnected subsystems: subsystems interconnect when prostate biopsies obtained in an ambulatory surgery setting arrive simultaneously at the same accession desk with the products of a radical neck dissection from a frozen section room
4.
Feed-back loops: different feed-back loops cycle simultaneously as telephone calls go back and forth between pathologist’s reviewing slides and pathologist’s assistants returning to fixed specimens to harvest more tissue samples, while, at the same time, pathologists send computer messages to histologists to request additional levels and special stains
5.
Limited substitutions: the constraints due to the different tissue processor cycles limit substitutions of cassette batches depending on run times
6.
Multiple interacting controls: multiple interacting controls appear at accession, in identification of specimens, in the histology laboratory , with the sorting of blocks, and on pathologist’s desks at the arrival of slides
7.
Indirect information transfer: indirect information transfer occurs when clinical features about cases are reported only in shouts over the shoulder of an operating room technician hurrying down a hallway, critical choices in specimen sampling are made only in whispers among residents at specimen processing stations, or vital new clinical information arrives only in muttered remarks from a clinical fellow who has come to look at slides
8.
Limited understanding of the requirements of the process: clinical staff collecting specimens have limited understanding of requirements for histologic diagnosis; pathologists as they interpret slides have limited understanding of what information clinicians imagine reports will contain
Ambivalent effect of electronic information transfer in complex processes: Computerization brings both positive innovations as well as dangers to the complex process that fits Perrow’s description. The positive changes have reduced unwanted variation, standardized data input, and reduced dependence on the variable information transfer media. Computerization has also helped by programming formats like synoptic report checklists and has facilitated automation of routine tasks, like bar-coded logging-in specimens, collating them with bar-coded requisition documents. However, negative changes brought by programmed processes of electronic information transfer require invariant sequences, stipulate one way to perform a component task, allow only limited buffers, and force only designed substitutions [47]. As computer-facilitated standardization has been achieved, former safeguards, redundancies, buffers, and alarms in previous surgical pathology systems have been eliminated. With newer complex systems come tighter couplings. High volume, complex, tightly coupled systems open themselves to untoward events in which two or more failures interact in error-causing ways that process designers and operators have not anticipated. Such event sequences, Perrow and another student of system error, James Reason, find, precipitate disproportionately bad outcomes that Perrow has designated catastrophes [50, 59].
Lessons of lean principles and practices: In these circumstances, sustained practical error reduction , incorporating lean industrial engineering principles and practices, has become a valuable response [3, 7, 11, 82, 94–96]. The lean approach consists of systematic practical error detection , then error reduction, prevention, and amelioration through countermeasures. It addresses all four defect types recorded by amended reports . For the three practical sorts of defects, the analysis makes connections presented in the next three paragraphs .
1.
Misidentification is the practical error with the most devastating potential [82, 96]. To attack it, colleagues who labor upstream in the process must accept forcing functions, labeling standards, and new labeling procedures; the beneficial effects of this apparently extra upstream effort often exert themselves only downstream where those making the changes cannot see their laudable effects. Nevertheless, a trio of worthwhile points has emerged from lean interventions that improve patient and specimen identification. Detecting and preventing misidentification entails: (i) training in labeling standards that extends outside surgical pathology premises, to dermatologists’ offices, endoscopy suites, and operating rooms, (ii) recognizing that batched printing of labels is a recurrent misidentification threat that design of specimen flow must avert it as much as possible, (iii) designing identification checks into multiple steps in the process appears essential, especially at two important checkpoints—(a) arrival of requisitions and specimen containers at accession and (b) reconciliation of requisitions with reports just before reports are released [82, 96].
2.
Specimen defects: Root causes of specimen defects increasingly reveal that ambiguities and delays in potentially decisive ancillary test results, especially those from molecular tests, are a growing cause of specimen defects [82].
3.
Result reporting: In result reporting , the increasing importance of ancillary testing in surgical pathology often now forces a Hobson’s choice. The unattractive decision falls between either issuing an incomplete report liable to later amendment or delaying the report until potentially modifying ancillary information can be combined into a fully integrated report [16].
Stay updated, free articles. Join our Telegram channel
Full access? Get Clinical Tree
