- to understand the varied methods used to evaluate public health or complex interventions;
- to describe the features of a cluster randomised controlled trial and when it may be appropriate to use this design;
- how we can we use time trend or before/after designs;
- what the role is of natural experiments;
- how ecological data within and between areas can be of value.
What is a complex intervention?
The methods used to evaluate health service interventions have been covered in Chapter 11. Our gold standard is the parallel group randomised controlled trial (RCT) which, if undertaken well, provides very strong evidence as to whether an intervention is causally related to improved health outcomes or not. These are ideally suited to studies of drugs where it is usually possible to blind the subjects and researchers to which intervention has been given. As methods of conducting trials have developed and expanded so have the types of interventions that such trials are used to evaluate – from a single drug to a single nonpharmacological intervention, such as surgery, to a complex intervention such as a stroke unit or a smoking cessation programme.
A complex health care intervention is defined as one that consists of several separate components, each of which is considered essential to the functioning of the intervention as a whole. A smoking cessation programme, for example, might consist of written information, media coverage, physician advice and cognitive behaviour therapy. Unfortunately there are several reasons why it is often very difficult or simply not feasible to conduct a parallel RCT to public health interventions or complex health service research questions. Instead we use different sorts of designs some of which are still randomised (cluster randomised trials) and some that are based on special types of observational studies.
Cluster randomised controlled trials
A cluster randomised trial differs from conventional RCTs in that a cluster rather than an individual is the unit of randomisation. Examples of clusters include (a) hospitals, (b) general practices, (c) geographical areas (d) schools, (e) prisons, (f) workplaces etc. In each cluster all potential subjects receive the same intervention rather than being randomised to different treatment options.
Why do we randomise by cluster?
The usual reason is that it is not practical to randomise by individual. Public health interventions are generally delivered at a group rather than individual level so it is sensible to evaluate them at this level. For example, a work-based health promotion campaign would be unpopular if only some of the workers were offered the intervention since those not receiving it may feel hard done by. On the other hand it would be fine to randomise workplaces to either all workers receiving the intervention or no workers receiving it. One of the main issues is contamination, which is when the intervention also gets delivered to some of the control group. For example, if we tried to randomise individuals to a media campaign that uses local newspapers, it would be difficult to prevent the control group being contaminated by the intervention. In this instance, different geographical areas could be randomised to either receive the media campaign or not.
As noted in Chapter 14, it is usual to seek informed consent prior to randomisation in a parallel group design RCT. This approach is referred to as opt in consent. In a cluster RCT consent to participate has first to be sought at the group level. Randomisation of the clusters follows and consent from the individual is often obtained using opt-out consent an approach which usually involves individuals returning a form if they do not want to receive the allocated intervention or to have their outcome data used in the analysis. Opt out consent is used in population health improvement research because obtaining individual written consent may limit recruitment and introduce bias which may seriously undermine the validity of the research. It is argued that in minimally invasive epidemiological research, individual consent should be waived where (a) the benefits to society are potentially high, (b) the risk to individuals low, and (c) the effort and cost of obtaining individual consent may be prohibitive. Because groups rather than individuals are randomised, cluster trials also require additional design and analysis considerations and the advantages and disadvantages of this approach are explained further in Table 20.1.
Table 20.1 Advantages and disadvantages of cluster randomisation.
| Advantages |
a. Reduces problems of contamination so that either practitioners or subjects in the control arm are not inadvertently exposed to the intervention which may result in benefits to those in the control arm indirectly due to the intervention. This will reduce (attenuate) the chance of showing the intervention is beneficial. b. May be more cost-effective to deliver intervention at a cluster than individual level. c. For public health interventions intended to improve population health, where individual randomisation is not possible, cluster randomisation is the gold standard. |
| Disadvantages |
a. Larger sample sizes are required as are more sophisticated statistical methods that take into account the clustering effect (e.g. multilevel models). In a two-arm trial one needs at least 8 clusters (4 per arm) as a minimum and the more clusters the better so that any differences between clusters are balanced across the trial arms. Stratified randomisation is one method to help ensure balance so that the clusters are grouped into strata based on a characteristic likely to be related to the outcome being studied e.g. teaching hospital versus district general hospital prior to randomisation in a trial comparing day surgery versus surgery including an inpatient stay. b. Recruitment or exclusion bias: individual RCTs first recruit participants and then randomises them avoiding any selection or exclusion bias as the researchers are not aware of which arm the person will be allocated. However in a cluster RCT, if randomisation of the clusters occurs before the participants are recruited then the researchers could influence which participants are included after they know which arm the person has been allocated too which could bias the results. |
Example: Peer led intervention for smoking prevention in teenagers: the ASSIST cluster RCT
Health education lessons in schools have not been shown to be very effective in reducing teenage smoking. However, young people may be more influenced by their peers than teachers. The ASSIST trial studied the smoking behaviour of 10,730 young people aged 12–13 years in 59 schools in England and Wales. Schools were randomised within strata (based on size of school, level of entitlement to free school meals etc.) so that 29 schools received usual health education and 30 schools received the intervention. The intervention required students to identify influential peers in their year group who then went on a 2-day participatory learning course outside school which covered the harms of smoking as well as listening and supportive skills. Peer supporters were then encouraged to discuss smoking during informal conversations with their friends. After two years follow-up, the odds ratio of smoking in the intervention compared to control schools was 0.78 (95% CI 0.64 to 0.96) demonstrating a 22% relative reduction in odds of smoking in the intervention schools (Campbell et al., 2008).
Stepped wedge designs
In some cases it may be hard to recruit clusters if there is a pre-existing belief that the intervention is effective (or a policy decision has been made to implement the intervention even in the absence of evidence) or clusters randomised to the control arm feel little commitment to the study and the necessary data collection given that they will not receive the intervention. In such circumstances, a useful alternative is a stepped wedge design whereby all clusters receive the intervention but some receive it immediately whilst others receive it after a delay so there is a period of time when they act as the control arm. (Such a design can also be used in individual level studies.) This often reflects what happens in the ‘real world’ where a new service cannot be delivered immediately to all areas and has to be introduced to some areas first though often the order is not randomised so that bias may be introduced.
Example: Nutritional supplementation and future cardiovascular risk
There is much scientific interest concerning the role of pre- and postnatal nutrition on later life cardiovascular risk as observational cohort studies have shown that babies born small have more heart disease and diabetes. A national community-based programme to improve the nutrition of children in India was established in the 1980s. This involved providing a cereal based meal with calorie and protein supplementation to pregnant mothers and their children in villages in India. Because the programme had to be implemented in a phased approach, the National Institute of Nutrition in Hyderabad under took a stepped wedge design where 29 villages were selected and 15 were chosen as the intervention and 14 were control villages who all received the intervention after a delay of three years. A follow-up study, conducted when the children were around 16 years of age, showed that children born in the intervention villages were taller, had better measures of insulin metabolism and less stiff arteries though measures of obesity were similar (Kinra et al., 2008).
Why can’t we always do RCTs?
Whilst it may be theoretically possible to imagine how one could do a trial for any evaluation, in the real world it is often simply not possible or not ethical to either do an individual or cluster-based RCT. Table 20.2 highlights possible reasons and examples.
Table 20.2 Reasons why it is difficult or impossible to undertake RCTs.
1. Experimentation unnecessary – effects so dramatic that confounding could not explain results, e.g. insulin therapy for insulin dependent diabetes
 Stay updated, free articles. Join our Telegram channel
Full access? Get Clinical Tree
|