Exclusions Before Randomisation

Investigators can exclude participants before randomisation. The eventual randomised treatment comparison will remain unbiased (good internal validity), irrespective of whether researchers have well-founded or whimsical reasons for exclusion of particular individuals. However, exclusions at this stage can hurt extrapolation, the generalisability of the results (external validity). For most investigations, we therefore recommend that eligibility criteria be kept to a minimum, in the spirit of the large and simple trial. However, some valid reasons exist for exclusion of certain participants. Individuals could, for example, have a condition for which an intervention is contraindicated, or they could be judged likely to be lost to follow-up. The trial question should guide the approach. Sometimes, however, investigators impose so many eligibility criteria that their trial infers to a population of little apparent interest to anyone, and, in addition, recruitment becomes difficult. If investigators exclude too many participants, or the wrong participants, their results might not represent the people of interest, even though the randomised controlled trial might have been meticulously done (i.e., the results could be true but potentially irrelevant).

Exclusions After Randomisation

Exclusions made after randomisation threaten to bias treatment comparisons. Randomisation itself configures unbiased comparison groups at baseline. Any erosion, however, over the course of the trial from those initially unbiased groups produces bias, unless, of course, that erosion is random, which is unlikely. Consequently, for the primary analysis, methodologists suggest that results for all patients who are randomly assigned should be analysed and, furthermore, should be analysed as part of the group to which they were initially assigned. Trialists refer to such an approach as an intent-to-treat (ITT) analysis. Simply put: once randomised, always analysed as assigned.

ITT principles underlie the primary analysis in a randomised controlled trial to avoid biases associated with nonrandom loss of participants. Investigators can also do secondary analyses, preferably preplanned, based on only those participants, for example, who comply with the trial treatment protocol or who receive the treatment irrespective of randomised assignment (generally referred to as per protocol [PP], on-treatment, or as-treated analysis). Secondary analyses are acceptable as long as researchers label them as secondary and nonrandomised comparisons. Trouble brews, however, when investigators exclude participants and, in effect, present a secondary, nonrandomised comparison as the primary randomised comparison from a trial. In reality, this analysis represents a cohort study masquerading as a randomised controlled trial. Exclusion of participants from an analysis can lead to misleading conclusions ( Panel 15.1 ).

Researchers often do not provide adequate information on excluded participants. Furthermore, in an older review of 249 randomised controlled trials published in major general medical journals in 1997, only 2% (5 of 249) of the reports explicitly stated that all randomly assigned participants were analysed according to the randomised group assignment. About half of the reports (119 of 249) noted an ITT analysis, but many provided no details to support this claim.

More recent data on ITT analyses are not encouraging. In the 50 trials of experimental pain models, none (0%) reported using an ITT analysis. Of the 123 trials of clinical pain, 47% reported using an ITT analysis and 5% reported using a modified ITT (mITT). In another review of 2349 trials, 25% were classified as an ITT analysis, 14% as an mITT analysis, and 61% as not reporting an ITT analysis. Moreover, the mITT classification is a misnomer in that it is decidedly not even close to an ITT analysis. ‘Whatever the definition used in the ITT approach, it is clear that the analysis performed in the mITT trials is a “PP [per-protocol] analysis”. It is important to underline that the authors of mITT studies inappropriately used the term “ITT” because in reality, the analyses they performed were substantially “PP analyses”, which can confuse the reader’. PP analyses, sometimes called ‘as-treated’ or ‘on-treatment’ analyses, represent distinctly non-ITT analyses in that they are nonrandomised comparisons. Indeed, the mITT trials displayed larger treatment effects than ITT trials, which supports the view that mITT trials represent nonrandomised, biased comparisons.

Additionally, researchers frequently do not report anything with respect to exclusions. Left in this information void, many readers deduce that certain trials used ITT principles and had no exclusions. We call this scenario ‘no apparent exclusions’. Readers commonly view trials with no apparent exclusions as less biased, when in fact unreported exclusions probably occurred in many of them. Indeed, trials with no apparent exclusions were methodologically weaker than those reporting at least some exclusions. In other words, some of the more biased trials might be mistakenly interpreted as unbiased, and many of the less biased trials as biased; we call this inconsistency ‘the exclusion paradox’. Until researchers comprehensively report exclusions after randomisation, readers should be aware of this unsettling irony.

Discovery of Participant Ineligibility

In some trials, participants are enrolled and later discovered not to have met the eligibility criteria. Exclusions at this point could seriously bias the results, because discovery is probably not random. For example, participants least responsive to treatment or who have side effects might draw more attention and, therefore, might be more likely to be judged ineligible than other study participants. Alternatively, a physician who had treatment preferences for certain participants might withdraw individuals from the trial if they were randomly assigned to what he or she believes to be the wrong group.

Participants discovered to be ineligible should remain in the trial. An exception could be made if establishment of eligibility criteria is difficult. In such instances, investigators could obtain the same information from each patient at time of randomisation and have it centrally reviewed by an outside source, blinded to the assigned treatment. That source, whether a person or group, could then withdraw patients who did not satisfy the eligibility criteria, presumably in an unbiased way.

Post-Randomisation, Pretreatment Outcome

Researchers sometimes report exclusion of participants on the basis of outcomes that happen before treatment has begun or before the treatment could have had an effect. For example, in a clinical trial of a specific drug’s effect on death rates, investigators withdrew as nonanalysable data on all patients who died after randomisation but before treatment began or before they had received at least 7 days of treatment. This winnowing seems intuitively attractive, because none of the deaths can then be attributable to treatment. But the same argument could be made for excluding data on all patients in a placebo group who died during the entire study interval, because, theoretically, none of these deaths could have been related to treatment. This example illustrates the potential for capriciousness in addressing post-randomisation, pretreatment outcomes.

Randomisation tends to balance the nonattributable deaths in the long run. Any tinkering after randomisation, even if done in the most scientific and impartial manner, cannot improve upon that attribute but can hurt it. More importantly, this meddling sometimes serves as a post hoc rationale for inappropriate exclusions.

Post hoc rationalisation arises when investigators observe the results and then frame rules that favour their hypotheses. Assume that an investigator postulates that a drug used for treatment reduced the death rate associated with a particular condition. After analysis of data, however, the investigator notes that 14 deaths in the treatment group and two deaths in the placebo group arose before treatment had begun or before the drug had been taken for at least 7 days. She then rationalises the deaths as unrelated to treatment and withdraws them from analysis. Such a response would seriously bias her results, even though her reasoning in the report would likely seem logical.

Imposed a priori, such rules only complicate trial implementation; imposed a posteriori, they lead to biased and invalid results. In assessment of randomised controlled trials, identification of when researchers stipulated rules usually proves impossible. We prefer to find, in reports of randomised controlled trials, that investigators did not allow any withdrawal of participants after randomisation. The data of all randomised patients should be analysed. Planned or unplanned, the exclusion of nonanalysable outcomes on grounds of efficiency is not a generally accepted practice in the analysis of a randomised clinical trial.