html xmlns=”http://www.w3.org/1999/xhtml” xmlns:mml=”http://www.w3.org/1998/Math/MathML” xmlns:epub=”http://www.idpf.org/2007/ops”>
Generalizability is the ability to draw widely or even universally applicable conclusions from a study. Generalizability is an important goal, because if the conclusions of the study cannot be applied to a broader population, then the results have little or no value to the larger community. Ideally, a study would be generalizable to a large population, but often practical limitations such as time, money, and the population available mean that the study’s generalizability must be reduced to preserve its validity. Validity means that the results of the study are true for the population studied. Every effort must be made to maintain study validity.
11.1 The Population
The participants in a clinical study are assumed to be representative of some population, and when you specify what individuals you will select, you determine which population they represent. For maximum generalizability, the population should be all people who could benefit from the study’s results. However, the more varied the population is, the more individuals you must study to ensure that the effect of interest can be detected despite the variation within the study population. If you limit a study to a specific, narrowly defined group, interpersonal variation is reduced so you can do a smaller study and still be able to detect the effect of interest; however, you then have no information about the effect in other groups. As a rule, the individuals studied will represent a compromise between generalizability and feasibility.
Many patients with major depression also suffer from other related disorders, such as anxiety disorder, which may add more variability to a study’s results. Restricting the target population to patients with no other disorders will reduce the variability among participants and thus improve the ability to detect an effect, but it will exclude a large proportion of patients and thus severely reduce generalizability. If resources are sufficient to study a larger population, investigators may allow the inclusion of participants with other disorders that do not interact with the pharmacological effect of the therapy, thereby increasing the generalizability without adversely affecting validity.
There are practical advantages to either approach. Fewer patients would be required if only participants with no other disorders were included, but because so many patients will have multiple disorders, it may be more difficult to recruit even this smaller number of patients.
Often it can be very difficult to decide what the study population should be, and validity needs to be protected at the cost of generalizability.
Many nephrologists are interested in whether dietary supplements improve nutritional status in individuals who are undergoing renal dialysis. To reduce variability among the participants, such a study might include only individuals who are stable on dialysis. The investigators may also want to exclude individuals with Non-Insulin Dependent Diabetes Mellitus (NIDDM), which may affect nutritional parameters independent of the dialysis. Alternatively, the investigator might want to include only individuals with NIDDM, since they constitute a large fraction of the population on dialysis. In either case, such a study would have validity for the population studied, but not necessarily have generalizability to the total population of individuals on dialysis.
For a study of a rare outcome, a population with a high prevalence of the outcome is necessary so that there will be enough participants with the condition to make conclusions about the effect of therapies or exposures (see Appendix B on sample size). Sometimes it is possible to use a restricted study sample without losing generalizability.
An investigator wants to study whether a high dose of a nutritional supplement during pregnancy reduces the risk of early delivery. The inexpensive supplement has been in general use for many years and has been shown to be nontoxic to the mother and the fetus at the dosage specified. The true target population for this drug would be all pregnant women, since early delivery cannot be ruled out in any pregnancy. Because the rate of early delivery in the general population is low, an impractically large number of participants would have to be studied to detect a reduction. Restricting the study to women at high risk for early delivery makes the study feasible. Positive results in this group would still allow the investigator to recommend use of the supplement for all pregnant women, since the supplement is known to be safe and potentially would reduce the risk of early delivery in low risk women as well. Thus, the investigator’s choice to improve feasibility should not compromise generalizability or validity.
But sometimes generalizability must be sacrificed for feasibility, although validity can never be sacrificed.
Several studies of gestational diabetes and the incidence and timing of metabolic disorders after pregnancy have been conducted in a hospital serving a largely Hispanic population. Hispanics have higher rates of gestational diabetes than do other ethnic groups, as well as higher rates of metabolic diabetes, so fewer participants would need to be enrolled and followed to obtain the needed number of cases. Therefore, these studies were limited to women of Hispanic background. However, the results from these studies may not be applicable to women of other ethnic groups.
Sometimes it is reasonable to apply results in a subgroup to the general population, as in the interventional study in Example 11C. In other circumstances, such as in Example 11D, studies involving other groups are required to confirm that the findings in one specific population are generalizable.
Even if the sample is representative of the total population with the disease, generalizability may be diminished if enrollment is limited only to highly motivated individuals who are very likely to remain in the study and adhere to the protocol. This helps the study progress well and maximizes the chance of detecting an effect, but the results may not apply to the general population, which includes many less motivated individuals.
The Diabetes Control and Complications Trial (DCCT) demonstrated convincingly that intensive monitoring and treatment of Type I diabetes was extremely effective in reducing the risks of diabetic sequelae. But the results are only directly generalizable to Type I diabetics willing to follow the intensive monitoring and treatment plan, requiring participants to test blood glucose levels four or more times a day, adjust their insulin dose based on those measurements, and follow a diet and exercise plan. Therefore, these results may not be generalizable to Type I diabetics who will not or cannot follow such a strict regimen.
Stay updated, free articles. Join our Telegram channel
Full access? Get Clinical Tree
