5 Common Research Designs and Issues in Epidemiology
Research is the process of answering a question that can be answered by appropriately collected data. The question may simply be, “What is (or was) the frequency of a disease in a certain place at a certain time?” The answer to this question is descriptive, but contrary to a common misperception, this does not mean that obtaining the answer (descriptive research) is a simple task. All research, whether quantitative or qualitative, is descriptive, and no research is better than the quality of the data obtained. To answer a question correctly, the data must be obtained and described appropriately. The rules that govern the process of collecting and arranging the data for analysis are called research designs.
Another research question may be, “What caused this disease?” Hypothesis generation is the process of developing a list of possible candidates for the causes of the disease and obtaining initial evidence that supports one or more of these candidates. When one or more hypotheses are generated, the hypothesis must be tested (hypothesis testing) by making predictions from the hypotheses and examining new data to determine if the predictions are correct (see Chapters 6 and 10). If a hypothesis is not supported, it should be discarded or modified and tested again. Some research designs are appropriate for hypothesis generation, and some are appropriate for hypothesis testing. Some designs can be used for either, depending on the circumstances. No research design is perfect, however, because each has its advantages and disadvantages.
The basic function of most epidemiologic research designs is either to describe the pattern of health problems accurately or to enable a fair, unbiased comparison to be made between a group with and a group without a risk factor, a disease, or a preventive or therapeutic intervention. A good epidemiologic research design should perform the following functions:
Enable a comparison of a variable (e.g., disease frequency) between two or more groups at one point in time or, in some cases, within one group before and after receiving an intervention or being exposed to a risk factor.
The research designs discussed in this chapter are the primary designs used in epidemiology. Depending on design choice, research designs can assist in developing hypotheses, testing hypotheses, or both. All designs can be used to generate hypotheses; and a few designs can be used to test them—with the caveat that hypothesis development and testing of the same hypothesis can never occur in a single study. Randomized clinical trials or randomized field trials are usually the best designs for testing hypotheses when feasible to perform.
Because some research questions can be answered by more than one type of research design, the choice of design depends on a variety of considerations, including the clinical topic (e.g., whether the disease or condition is rare or common) and the cost and availability of data. Research designs are often described as either observational or experimental.
In observational studies the investigators simply observe groups of study participants to learn about the possible effects of a treatment or risk factor; the assignment of participants to a treatment group or a control group remains outside the investigators’ control. Observational studies can be either descriptive or analytic. In descriptive observational studies, no hypotheses are specified in advance, preexisting data are often used, and associations may or may not be causal. In analytic observational studies, hypotheses are specified in advance, new data are often collected, and differences between groups are measured.
In an experimental study design the investigator has more control over the assignment of participants, often placing them in treatment and control groups (e.g., by using a randomization method before the start of any treatment). Each type of research design has advantages and disadvantages, as discussed subsequently and summarized in Table 5-1 and Figure 5-1.
Qualitative research involves an investigation of clinical issues by using anthropologic techniques such as ethnographic observation, open-ended semistructured interviews, focus groups, and key informant interviews. The investigators attempt to listen to the participants without introducing their own bias as they gather data. They then review the results and identify patterns in the data in a structured and sometimes quantitative form. Results from qualitative research are often invaluable for informing and making sense of quantitative results and providing greater insights into clinical questions and public health problems. The two approaches (quantitative and qualitative) are complementary, with qualitative research providing rich, narrative information that tells a story beyond what reductionist statistics alone might reveal.
A cross-sectional survey is a survey of a population at a single point in time. Surveys may be performed by trained interviewers in people’s homes, by telephone interviewers using random-digit dialing, or by mailed, e-mailed, or Web-based questionnaires. Telephone surveys or e-mail questionnaires are often the quickest, but they typically have many nonresponders and refusals, and some people do not have telephones or e-mail access, or they may block calls or e-mails even if they do. Mailed surveys are also relatively inexpensive, but they usually have poor response rates, often 50% or less, except in the case of the U.S. Census, where response is required by law, and follow-up of all nonresponders is standard.
Cross-sectional surveys have the advantage of being fairly quick and easy to perform. They are useful for determining the prevalence of risk factors and the frequency of prevalent cases of certain diseases for a defined population. They also are useful for measuring current health status and planning for some health services, including setting priorities for disease control. Many surveys have been undertaken to determine the knowledge, attitudes, and health practices of various populations, with the resulting data increasingly being made available to the general public (e.g., healthyamericans.org). A major disadvantage of using cross-sectional surveys is that data on the exposure to risk factors and the presence or absence of disease are collected simultaneously, creating difficulties in determining the temporal relationship of a presumed cause and effect. Another disadvantage is that cross-sectional surveys are biased in favor of longer-lasting and more indolent (mild) cases of diseases. Such cases are more likely to be found by a survey because people live longer with mild cases, enabling larger numbers of affected people to survive and to be interviewed. Severe diseases that tend to be rapidly fatal are less likely to be found by a survey. This phenomenon is often called Neyman bias or late-look bias. It is known as length bias in screening programs, which tend to find (and select for) less aggressive illnesses because patients are more likely to be found by screening (see Chapter 16).
Repeated cross-sectional surveys may be used to determine changes in risk factors and disease frequency in populations over time (but not the nature of the association between risk factors and diseases). Although the data derived from these surveys can be examined for such associations in order to generate hypotheses, cross-sectional surveys are not appropriate for testing the effectiveness of interventions. In such surveys, investigators might find that participants who reported immunization against a disease had fewer cases of the disease. The investigators would not know, however, whether this finding actually meant that people who sought immunization were more concerned about their health and less likely to expose themselves to the disease, known as healthy participant bias. If the investigators randomized the participants into two groups, as in a randomized clinical trial, and immunized only one of the groups, this would exclude self-selection as a possible explanation for the association.
Cross-sectional surveys are of particular value in infectious disease epidemiology, in which the prevalence of antibodies against infectious agents, when analyzed according to age or other variables, may provide evidence about when and in whom an infection has occurred. Proof of a recent acute infection can be obtained by two serum samples separated by a short interval. The first samples, the acute sera, are collected soon after symptoms appear. The second samples, the convalescent sera, are collected 10 to 28 days later. A significant increase in the serum titer of antibodies to a particular infectious agent is regarded as proof of recent infection.
Even if two serum samples are not taken, important inferences can often be drawn on the basis of titers of IgG and IgM, two immunoglobulin classes, in a single serum sample. A high IgG titer without an IgM titer of antibody to a particular infectious agent suggests that the study participant has been infected, but the infection occurred in the distant past. A high IgM titer with a low IgG titer suggests a current or very recent infection. An elevated IgM titer in the presence of a high IgG titer suggests that the infection occurred fairly recently.
Cross-sectional ecological studies relate the frequency with which some characteristic (e.g., smoking) and some outcome of interest (e.g., lung cancer) occur in the same geographic area (e.g., a city, state, or country). In contrast to all other epidemiologic studies, the unit of analysis in ecological studies is populations, not individuals. These studies are often useful for suggesting hypotheses but cannot be used to draw causal conclusions. Ecological studies provide no information as to whether the people who were exposed to the characteristic were the same people who developed the disease, whether the exposure or the onset of disease came first, or whether there are other explanations for the observed association. Concerned citizens are sometimes unaware of these weaknesses (sometimes called the ecological fallacy) and use findings from cross-sectional ecological surveys to make such statements as, “There are high levels of both toxic pollution and cancer in northern New Jersey, so the toxins are causing the cancer.” Although superficially plausible, this conclusion may or may not be correct. For example, what if the individuals in the population who are exposed to the toxins are universally the people not developing cancer? Therefore the toxic pollutants would be exerting a protective effect for individuals despite the ecological evidence that may suggest the opposite conclusion.
In many cases, nevertheless, important hypotheses initially suggested by cross-sectional ecological studies were later supported by other types of studies. The rate of dental caries in children was found to be much higher in areas with low levels of natural fluoridation in the water than in areas with high levels of natural fluoridation.1 Subsequent research established that this association was causal, and the introduction of water fluoridation and fluoride treatment of teeth has been followed by striking reductions in the rate of dental caries.2
Longitudinal ecological studies use ongoing surveillance or frequent repeated cross-sectional survey data to measure trends in disease rates over many years in a defined population. By comparing the trends in disease rates with other changes in the society (e.g., wars, immigration, introduction of a vaccine or antibiotics), epidemiologists attempt to determine the impact of these changes on disease rates.
For example, the introduction of the polio vaccine resulted in a precipitous decrease in the rate of paralytic poliomyelitis in the U.S. population (see Chapter 3 and Fig. 3-9). In this case, because of the large number of people involved in the immunization program and the relatively slow rate of change for other factors in the population, longitudinal ecological studies were useful for determining the impact of this public health intervention. Nevertheless, confounding with other factors can distort the conclusions drawn from ecological studies, so if time is available (i.e., it is not an epidemic situation), investigators should perform field studies, such as randomized controlled field trials (see section II.C.2), before pursuing a new, large-scale public health intervention.
Another example of longitudinal ecological research is the study of rates of malaria in the U.S. population since 1930. As shown in Figure 5-2, the peaks in malaria rates can be readily related to social events, such as wars and immigration. The use of a logarithmic scale in the figure visually minimizes the relative decrease in disease frequency, making it less impressive to the eye, but this scale enables readers to see in detail the changes occurring when rates are low.
(From Centers for Disease Control and Prevention: Summary of notifiable diseases, United States, 1992. MMWR 41:38, 1992.)
Important causal associations have been suggested by longitudinal ecological studies. About 20 years after an increase in the smoking rates in men, the lung cancer rate in the male population began increasing rapidly. Similarly, about 20 years after women began to smoke in large numbers, the lung cancer rate in the female population began to increase. The studies in this example were longitudinal ecological studies in the sense that they used only national data on smoking and lung cancer rates, which did not relate the individual cases of lung cancer to individual smokers. The task of establishing a causal relationship was left to cohort and case-control studies.
A cohort is a clearly identified group of people to be studied. In cohort studies, investigators begin by assembling one or more cohorts, either by choosing persons specifically because they were or were not exposed to one or more risk factors of interest, or by taking a random sample of a given population. Participants are assessed to determine whether or not they develop the diseases of interest, and whether the risk factors predict the diseases that occur. The defining characteristic of cohort studies is that groups are typically defined on the basis of exposure and are followed for outcomes. This is in contrast to case-control studies (see section II.B.2), in which groups are assembled on the basis of outcome status and are queried for exposure status. There are two general types of cohort study, prospective and retrospective; Figure 5-3 shows the time relationships of these two types.
Figure 5-3 Relationship between time of assembling study participants and time of data collection.
Illustration shows prospective cohort study, retrospective cohort study, case-control study, and cross-sectional study.
In a prospective cohort study, the investigator assembles the study groups in the present, collects baseline data on them, and continues to collect data for a period that can last many years. Prospective cohort studies offer three main advantages, as follows:
1. The investigator can control and standardize data collection as the study progresses and can check the outcome events (e.g., diseases and death) carefully when these occur, ensuring the outcomes are correctly classified.