Chapter Contents
A Taxonomy of Clinical Research 2
What Studies Can and Cannot Do 4
Descriptive Study 4
Cross-Sectional Study: A Snapshot in Time 4
Cohort Study: Looking Forward in Time 5
Case-Control Study: Thinking Backwards 5
Variations on These Themes 6
Nonrandomised Trial: Penultimate Design? 7
Randomised Controlled Trial: Gold Standard 7
Measurement of Outcomes 8
Conclusion 10
Many clinicians report that they cannot read the medical literature critically. To address this difficulty, we provide a primer of clinical research for clinicians and researchers alike. Clinical research falls into two general categories: experimental and observational, based on whether the investigator assigns the exposures or not. Experimental trials can also be subdivided into two: randomised and nonrandomised. Observational studies can be either analytical or descriptive. In this book, we distinguish between the two by this criterion: analytical studies feature a comparison (control) group, whereas descriptive studies do not. Within analytical studies, cohort studies track people forward in time from exposure to outcome. By contrast, case-control studies work in reverse, tracing back from outcome to exposure. Cross-sectional studies are like a snapshot, which measures both exposure and outcome at one time point. Descriptive studies cannot examine associations, a fact often forgotten or ignored. Measures of association, such as relative risk or odds ratio, are the preferred way of expressing results of dichotomous outcomes (e.g., sick versus healthy). Confidence intervals around these measures indicate the precision of these results. Measures of association with confidence intervals reveal the strength, direction, and plausible range of an effect, as well as the likelihood of chance occurrence. By contrast, p values address only chance. Testing null hypotheses at a p value of 0.05 has no basis in medicine and should be discouraged.
Clinicians today are in a bind. Increasing demands on their time are squeezing out opportunities to stay abreast of the literature, much less read it critically. PubMed currently catalogues 30,000 journals, and its citations include more than 27 million entries. Results of several studies indicate an inverse relation between quality of care and time since graduation from medical school; older clinicians consistently have a harder time keeping up. In many jurisdictions, attendance at a specified number of hours of continuing medical education (CME) courses is mandatory to maintain a licence to practice. Continuing medical education courses have at best a modest effect on the quality of care, and such courses alone appear unlikely to affect complex clinical behaviours. This deficiency of traditional CME offerings emphasises the importance of self-directed learning through reading. However, many clinicians lack the requisite skills to read the medical literature critically. Scientific illiteracy and innumeracy remain major failings of medical education.
This book on research methods is designed for busy clinicians and active researchers. The needs of clinicians predominate; hopefully, this primer will produce more critical and thoughtful consumers of research, and thus better practitioners. The needs of clinicians overlap with those of researchers throughout the chapters, but that overlap becomes most pronounced in the discussion of randomised controlled trials. For readers to assess randomised trials accurately, they should understand the relevant guidelines on the conduct of trials; these guidelines have been derived from empirical methodological research.
The disproportionate coverage of randomised controlled trials is intentional; randomised controlled trials are the gold standard in clinical research. Randomised controlled trials help to eliminate bias, and research has identified the important methodological elements of trials that minimise bias. Finally, because trials are deemed more credible than observational studies, clinicians might be more likely to act on their results than on those of other study types; hence, investigators need to ensure that trials are done well and reported well. To start, we provide a brief overview of research designs and discuss some of the common measures used.
A Taxonomy of Clinical Research
Analogous to biological taxonomy, a simple hierarchy can be used to categorise most studies ( Panel 1.1 ). To do so, however, the study design must be known. As in biology, anatomy dictates physiology. The anatomy of a study determines what it can and cannot do. A difficulty that readers encounter is that authors sometimes do not report the study type or provide sufficient detail to figure it out. A related problem is that authors sometimes incorrectly label the type of research done. Examples include calling nonrandomised controlled trials randomised, and labelling nonconcurrent cohort studies case-control studies. The adjective ‘case-controlled’ is also sometimes (inappropriately) applied to any study with a comparison group. The media compound the confusion by implying causation (when researchers cautiously reported only an association).
Quality of Evidence
- I
Evidence from at least one properly designed randomised controlled trial.
- II-1
Evidence obtained from well-designed controlled trials without randomisation.
- II-2
Evidence from well-designed cohort or case-control studies, preferably from more than one centre or research group.
- II-3
Evidence from multiple time series with or without the intervention. Important results in uncontrolled experiments (such as the introduction of penicillin treatment in the 1940s) could also be considered as this type of evidence.
- III
Opinions of respected authorities, based on clinical experience, descriptive studies, or reports of expert committees.
Strength of Recommendations
- A
Strongly recommends
- B
Recommends
- C
Makes no recommendation for or against
- D
Recommends against
- I
Concludes evidence insufficient to recommend for or against
Biology has animal and plant kingdoms. Similarly, clinical research has two large kingdoms: experimental and observational research. Fig. 1.1 shows that one can quickly decide the research kingdom by noting whether the investigators assigned the exposures (e.g., treatments) or whether they observed usual clinical practice. At the top of the hierarchy, for experimental studies one needs to distinguish whether the exposures were assigned by a truly random technique (with concealment of the upcoming assignment from those involved) or whether some other allocation scheme was used, such as alternate assignment. Many reports fail to include this critical information.
With observational studies, which dominate the literature, especially in lower-impact journals, the next step is to ascertain whether the study has a comparison or control group. If so, the study is termed analytical. If not, it is a descriptive study (see Fig. 1.1 ). If the study is analytical, the temporal direction needs to be identified. If the study determines both exposures and outcomes at one time point, it is termed cross-sectional. An example would be measurement of serum cholesterol of patients admitted to a hospital with myocardial infarction versus that of their next-door neighbour. This type of study provides a snapshot of the population of sick and well at one time point. A weakness of cross-sectional studies is that the temporal sequence may be unclear: did the exposure precede the outcome? In addition, cross-sectional studies are inappropriate for rare diseases or those that resolve quickly; severe diseases causing early death can mean that those studied are not representative of all those with a given disease.
If the study begins with an exposure (e.g., oral contraceptive use) and follows women for years to measure outcomes (e.g., ovarian cancer), then it is deemed a cohort study. Cohort studies can be either concurrent or nonconcurrent. By contrast, if the analytical study begins with an outcome (e.g., ovarian cancer) and looks back in time for an exposure, such as use of oral contraceptives, then the study is a case-control study.
Studies without comparison groups are called descriptive studies. At the bottom of the research hierarchy is the case report. Indeed, the literature is replete with reported oddities. When more than one patient is described, it becomes a case-series report. Some new diseases first appear in the literature this way.
What Studies Can and Cannot Do
Descriptive study
Starting at the bottom of the research hierarchy, descriptive studies are often the first foray into a new area of medicine, a first ‘toe in the water’. Investigators do descriptive studies to describe the frequency, natural history, and possible determinants of a condition. The results of these studies show how many people develop a disease or condition over time, describe the characteristics of the disease and those affected, and generate hypotheses about the cause of the disease. These hypotheses can be assessed through more rigorous research, such as analytical studies or randomised controlled trials. An example of a descriptive study would be the early reports of legionnaires’ disease and toxic-shock syndrome. An important caveat (often forgotten or intentionally ignored) is that descriptive studies, which do not have a comparison group, do not allow assessment of associations. Only comparative studies (both analytical and experimental) enable assessment of possible causal associations.
Cross-sectional study: a snapshot in time
Sometimes termed a frequency survey or a prevalence study, cross-sectional studies are done to examine the presence or absence of disease and the presence or absence of an exposure at a particular time. Thus prevalence, not incidence, is the focus. As both outcome and exposure are ascertained at the same time ( Fig. 1.2 ), the temporal relation between the two can be unclear. For example, assume that a cross-sectional study finds obesity to be more common among women with arthritis than among those without this condition. Did the extra weight load on joints lead to arthritis, or did women with arthritis become involuntarily inactive and then obese? This ‘chicken-egg’ question is unanswerable in a cross-sectional study.
Cohort study: looking forward in time
Cohort studies proceed in a logical sequence from exposure to outcome (see Fig. 1.2 ). Hence, this type of research is easier to understand than case-control studies. Investigators identify a group with an exposure of interest and another group or groups without the exposure. The investigators then follow the exposed and unexposed groups forward in time to determine outcomes. If the exposed group develops a higher incidence of the outcome than the unexposed, then the exposure is associated with an increased risk of the outcome, and vice versa .
Cohort studies can be prospective, retrospective, or both. In a prospective cohort study, the exposed and unexposed are identified and followed forward in time to outcomes. In a retrospective cohort study, the investigator goes back in time through medical records to identify exposure groups, then tracks them to outcomes through existing medical records. In some disciplines, retrospective cohort studies are more common than prospective cohort studies. As described in Chapter 4 , ambidirectional cohort studies look backward and forward in time.
The cohort study has important strengths and weaknesses. Because exposure is identified at the outset, one can usually determine that the exposure preceded the outcome. Recall bias is less of a concern than in the case-control study. The cohort study enables calculation of true incidence rates, relative risks, and attributable risks. However, for the study of rare events or events that take years to develop, this type of research design can be slow to yield results and thus prohibitively expensive. Nonetheless, several famous, large cohort studies continue to provide important information.
Case-control study: thinking backwards
Case-control studies work backwards. Because thinking in this direction is not intuitive for clinicians, case-control studies are widely misunderstood. Indeed, in one sample from the rehabilitation literature, 97% of research articles designated ‘case-control’ were mislabelled.
Starting with an outcome, such as disease, this type of study looks backward in time for exposures that might be related to the outcome. As shown in Fig. 1.2 , investigators define a group with an outcome (e.g., ovarian cancer) and a group without the outcome (controls). Then, through chart reviews, interviews, electronic medical records, or other means, the investigators ascertain the prevalence (or amount) of exposure to a risk factor (e.g., oral contraceptives, ovulation-induction drugs) in both groups. If the prevalence of the exposure is higher among cases than among controls, then the exposure is associated with an increased risk of the outcome.
Case-control studies are especially useful for outcomes that are rare or that take a long time to develop, prototypes being cardiovascular disease and cancer. These studies often require less time, effort, and money than would cohort studies. The Achilles heel of case-control studies is choosing an appropriate control group, discussed further in Chapter 6 . Controls should be similar to cases in all important respects except for not having the outcome in question. Inappropriate control groups have ruined many case-control studies and caused much harm. Additionally, recall bias (better recollection of exposures among the worried cases than among the healthy controls) is inevitable in studies that rely on memory. Because the case-control study samples on cases and controls, not on exposures, investigators cannot calculate incidence rates, relative risks, or attributable risks. Instead, odds ratios are the measure of association used; when the outcome is uncommon (e.g., most cancers) the odds ratio provides a good proxy for the true relative risk.
Investigations of outbreaks of food-borne diseases often use case-control studies. On a cruise ship, the entire universe of those at risk is known. Those with vomiting and diarrhoea (cases) are asked about food exposures, as are a sample of those not ill (controls). If a higher proportion of those who are ill reports having eaten a specific food than those who are well, the food becomes suspect. In 50 outbreaks associated with cruise ships from 1970 to 2003, the most common culprit was found to be seafood (28% of outbreaks).
Variations on these themes
Nested Case-Control Studies
Here, a group of persons being studied is followed forward in time until outcomes occur. Instead of analysing everyone in the group in usual cohort fashion, those with the outcome are deemed cases, and a sample of those without the disease become controls. Then, the analysis proceeds in typical case-control fashion. The smaller case-control study is ‘nested’ within a larger cohort under scrutiny.
Why analyse a cohort study this way? This approach may be useful when determining the exposure involves an expensive, painful, invasive test or ponderous interview. Determining exposure information for all participants might be impractical or prohibitively expensive. All cases have exposure determined, but only a small sample of all healthy persons (controls) need this done. A study of inflammatory markers and postpartum depression is illustrative. A prospective cohort of women being followed through pregnancy had blood samples collected and banked at specified times. Sixty-three women with depression had a sophisticated protein panel run on their blood sample to examine 92 inflammation markers. The same panel was run on blood from 228 controls without depression. Performing this extensive testing on all women in the cohort would have been costly.
Case-Cohort Studies
In this modification of a case-control study, ‘…controls are drawn from the same cohort as the cases regardless of their disease status. Cases of the disease of interest are identified, and a sample of the entire starting cohort (regardless of their outcomes) forms the controls’ Here, one control group can be used for multiple groups of cases, increasing the efficiency of the exploration. Controls are randomly selected from the population at risk; these controls are called the ‘reference subcohort’. In a traditional case-control study, researchers would need to recruit a different control group for each group of cases being studied; here, one control group can be recycled repeatedly.
A case-cohort study examined risk factors for hospitalisation after dog-bite injury. The cohort consisted of 1384 dog-bite patients ( Fig. 1.3 ) seen at one hospital emergency department; 111 of these who were admitted formed the case group. A simple random sample of the other patients comprised the control subcohort (221 patients). As noted previously, controls were selected independent of their disease status; by chance, 21 patients randomly chosen as controls were themselves cases (admitted to the hospital). Infected bites and bites to multiple sites were most strongly related to the risk of being hospitalised.
