7.3.1 Example: AB/BA cross-over design

We illustrate the AB/BA cross-over design with results from an unpublished study comparing two diuretics in the treatment of mild to moderate heart failure. After initial screening for suitability, there was a period of not less than 1 day and not more than 7 days, where diuretic treatment was withheld. Immediately prior to randomisation to either the AB sequence of treatment or the BA sequence, baseline observations were taken. Each treatment period lasted for 5 days, with an immediate transfer to the second treatment after the first treatment period was completed. As a washout was not employed between treatments, observations made in the first 2 days of each treatment period were not utilised in the analysis of the trial. The primary outcome measures were the frequency of micturition and the subjective assessment of urgency. As neither of these is suitable for illustrating the analysis of normally distributed data, we will instead use a secondary effectiveness variable, namely oedema status, together with diastolic blood pressure (DBP). Oedema status is formed by the sum of the left and right ankle diameters. The DBP was calculated from the mean of three readings. Both of these variables are measured prior to randomisation and at the end of each treatment period.

In total, 101 patients were recruited for the study, but seven withdrew prior to randomisation. Of the remaining 94 patients, only two failed to complete both treatment periods. Therefore, in order to illustrate the alternative methods of analysis, we have systematically removed approximately one in five of the observations from the second period. The structure of the data as analysed is shown in Table 7.1.

For each of our outcome variables, four analyses have been performed. In all of them, a treatment effect and a period effect were included as fixed. In two of the models, the baseline level was also included in the model as a covariate. Whether or not the baseline is included in the model, separate models are considered with the patient effect being fitted either as random or as fixed. The results of the models are summarised in Table 7.2.

Examination of the variance component terms shows that for all models the patient term is larger than the residual term. This indicates that there may have been substantial benefits from employing a cross-over design rather than a parallel group design. We note that this is particularly striking for oedema status. Note also the effect of including the baseline as a covariate in the analysis. This has the effect of reducing the size of the patient variance component term in Model 4. The implications of this are that the benefits of the cross-over are somewhat reduced when a (highly correlated) baseline covariate is available and, conversely, that the use of a mixed model is likely to be most helpful in these circumstances if there are missing values.

We see this in the estimates of the treatment standard errors. Comparison of Models 1 and 2 for the oedema status shows that the standard errors are identical (to the number of digits reported), indicating that the between-subject variation is so large that recovery of between-subject information is ineffective. With inclusion of the baseline level as a covariate, we see that Model 3 gives the same result as Model 1. This result is well known, showing that a single baseline has no effect on a fixed effects analysis. It does, however, produce a small reduction in the treatment standard error when a mixed model is fitted, showing that some between-subject information has been utilised.

The results for DBP show the recovery of between-subject information more clearly because of the relatively smaller between-subject variation. We see a detectable reduction in the treatment standard error, even when baselines are not used, and with the inclusion of baselines, a reduction of about 4% in the standard error is seen with the mixed models approach. This gain is modest but worthwhile.

The greatest advantage of the mixed models approach will unfortunately be gained in situations where a cross-over trial shows little benefit over a parallel group study, that is where the between-subject variance component is small relative to the residual variance component.

Such a situation occurs in a trial reported by Jones and Kenward (1989). In this two-period, cross-over trial, an oral mouthwash was compared with a placebo mouthwash. There were two 6-week treatment periods, with a 3-week washout period separating them. The outcome variable reported was the average plaque score per tooth, with each tooth being assessed on an integer scale from zero to three. Results were presented for the 34 patients with data from both treatment periods. Interestingly, these data arose from a trial in which 41 patients were randomised, and 38 completed the trial. For the purposes of this illustration, we have deleted the second observation from five randomly selected patients from the 34 with complete data.

Two models were fitted to the data using PROC MIXED. In both, a treatment effect and a period effect were included as fixed. In one, the patient effect was fitted as random, and in the other, it was fitted as fixed. The results are shown in Table 7.3.

Examination of the variance component terms shows that the patient term is appreciably smaller than the residual term. This indicates that the benefit of employing a cross-over design rather than a parallel group design may be small. The estimate of the period effect (not shown) is small, but in accord with our recommendation in the previous section, we retain it in the model. The main interest, of course, lies in the estimates of the treatment difference and the associated standard errors. Both analyses demonstrate a clear advantage to using the active mouthwash. For our purposes in comparing the results of the two analytical strategies, it is the standard errors that interest us because it is purely a matter of chance which method gives the larger point estimate of the treatment effect. We see that the use of the random effects model has reduced the standard error of the estimate of the treatment difference by about 6%.

PROC MIXED NOCLPRINT; CLASS treat period patient;

TITLE ‘FIXED EFFECT ANALYSIS WITH BASELINE’;

MODEL oed=treat period patient oedbase;

LSMEANS treat /DIFF PDIFF;

PROC MIXED NOCLPRINT; CLASS treat period patient;

TITLE ‘RANDOM EFFECTS ANALYSIS WITH BASELINE’;

MODEL oed=treat period oedbase / DDFM=KENWARDROGER;

RANDOM patient;

LSMEANS treat /DIFF PDIFF;

We consider the four-period, four-treatment cross-over trial described by Jones and Kenward (1989). Three drugs, A, C and D and a placebo B were compared to assess their effect on cardiac output, measured by the left ventricular ejection time (LVET). Each treatment was given for one week, with a one-week washout period between treatments. Observations were made at the end of each treatment period. Fourteen patients were used in the trial, yielding 56 observations. To demonstrate the use of the mixed models approach, we have arbitrarily set 13 of the 56 observations to be missing. The results of four analyses are presented in Table 7.4, from the combinations of inclusion or exclusion of carry-over effects, and handling the patient effects as fixed or random.

All pairwise comparisons of the carry-over effects were non-significant and will not be considered further or the details presented. We see that between-patient variation is moderate, being about 50% higher than the residual variance component. The random effects analysis without carry-over effects produces an average 1% reduction in the standard errors of the paired treatment comparisons compared with the fixed effects model, a modest but worthwhile gain. Comparing these estimates with the analyses in which carry-over was also fitted, two points are clear. Firsly, the standard errors of the mean treatment differences are larger when carry-over terms are present, irrespective of whether a fixed effects model is fitted or whether the patient term is regarded as random. Secondly, the reduction in the standard error of the mean treatment difference by fitting patient effects as random is larger when carry-over terms are also fitted. This latter result is general, and we will see more dramatic differences in later examples.

/*create dummy variables for carryover effects*/

DATA new; carry=treat; SET mydata; RUN;

DATA new; SET new;

IF period eq 1 THEN carry=4;

* setting a carryover value for the first period.

  it is arbitrary which treatment is selected.

  the choice will only influence the absolute values

  of the fixed effect estimates and not the difference

  between them;

PROC MIXED; CLASS treat period patient;

TITLE ‘Fixed Effect Analysis Without Carryover’;

MODEL lvet=treat patient period;

LSMEANS treat/DIFF PDIFF;

PROC MIXED; CLASS treat period patient;

TITLE ‘Random Effects Analysis Without Carryover’;

MODEL lvet=treat period/DDFM=KENWARDROGER;

RANDOM patient;

LSMEANS treat/DIFF PDIFF;

PROC MIXED; CLASS treat period patient carry;

TITLE ‘Fixed Effect Analysis Including Carryover’;

MODEL lvet=treat patient period carry;

LSMEANS treat carry/DIFF PDIFF;

PROC MIXED; CLASS treat period patient carry;

TITLE ‘Random Effects Analysis Including Carryover’;

MODEL lvet=treat period carry/DDFM=KENWARDROGER;

RANDOM patient;

LSMEANS treat carry/DIFF PDIFF;

7.5.1 Example: Three treatment two-period cross-over trial

Mead (1988) gives results of a two-period cross-over trial to compare three analgesic drugs labelled A, B and C. The trial involved 43 patients in total, and the numbers receiving each treatment combination were as follows: AB 7; BA 5; AC 7; CA 8; BC 8; CB 8. This design is sometimes referred to as ‘Koch’s design’. The effectiveness of each treatment was assessed by the numbers of hours of pain relief provided. The design did not include a washout period between treatments, and so there was a strong possibility of carry-over effects. The model fitted was as follows:

Period, treatment and carry-over effects were taken as fixed. Analyses were carried out with patient effects specified first as fixed in a conventional least squares analysis and then as random in a mixed models analysis.

The estimated variance components and treatments effects are shown in Table 7.5, first omitting and then including carry-over effects. Comparison of the two analyses omitting carry-over effects shows that the average standard error for treatment differences from the combined analysis with recovery of between-patient information is on average 12% less than for the within-patient analysis. The comparison A − B, which has the largest standard error, shows the greatest increase in precision, so that the combined analysis also leads to treatment estimates with a smaller range of standard errors. For this trial, the between-patient component of variance is relatively small, and recovery of between-patient information is clearly worthwhile.

Although the estimated carry-over effects are generally small relative to their standard errors, the results from this model could be preferred on the basis of the trial design and the absolute magnitude of the estimated carry-over effects. With this model, we see that the magnitudes of the standard errors of the treatment effects using a mixed model are only half of those obtained from the fixed effects model. The standard errors of the carry-over effects show an even greater separation. Using the mixed models approach, the penalty in fitting carry-over terms is an increase of just over 10% in the standard errors of the treatment differences.

For this trial, the between-patient variance component is small compared with the within-patient component (γ = 0.12), suggesting that there is little advantage in using a cross-over trial for testing these analgesics, even if carry-over effects can be avoided. Indeed, the predicted average standard error for treatment comparisons for a parallel group trial with the same number of patient sessions per treatment is 0.92, compared with 0.89 and 0.99 for the cross-over analysis ignoring and including carry-over effects, respectively.

This is a well-known example initially described by Hunter et al. (1970). The aim was to determine the effect of amantadine (treatment A) on subjects with Parkinsonism. The trial was placebo controlled (treatment B). After a run-in period of 1 week during which baseline information was recorded, there were two 4-weekly treatment periods, without a washout period. Weekly scores (0–4) were recorded for each of 11 physical signs, and the data presented in Table 7.6 give the weekly average total scores in each treatment period. Seventeen patients were randomised, and the data have no missing values.

Table 7.7 presents the results of analyses with and without inclusion of a carry-over term and with patient effects fitted as fixed or random.

An immediate point to note from the two mixed models is the very high patient variance component compared with the residual variance component (Table 7.8). This immediately suggests that little gain in efficiency will accrue from between-patient information. This is confirmed by comparison of the treatment standard errors, where even in the model in which carry-over is fitted, the reduction is only 4%. In most situations, where the between-patient variation is less extreme, the existence of the AA and BB treatment groups would lead us to expect greater benefits from the mixed models approach.

In this study, there is no evidence of any carry-over effect, and most statisticians would choose to report the model that excludes carry-over. However, having chosen a design for its optimal properties in estimating both treatment and carry-over effects, there is a strong case for reporting the fuller model.

The presentation in this example has been restricted to the analysis of the results in the two treatment periods, and the fact that baseline observations were also recorded has been ignored. Jones and Kenward (1989) present additional analyses utilising this baseline data, and, in particular, use interactions with the baseline to test for an effect of the baseline level on the treatment effect, period effect and carry-over effect. Although none of these terms was statistically significant at the 10% level of significance, they found indications that the treatment differences were higher with greater baseline levels. These authors also handled carry-over in a more involved way than we have employed in our analyses. They allowed for the possibility that carry-over would be different in those on the AA or BB sequence from those on the AB or BA sequence, but this term in the analysis of variance was clearly non-significant.

The conclusions from the trial will, in this instance, be qualitatively similar whichever of the previously described analytical methods is used, as long as carry-over is ignored in estimating treatment effects. Amantadine produces a reduction in the physical signs of Parkinson’s disease, which is statistically significant at the 5% level. Note, however, that inclusion of carry-over terms in the model produces a substantial increase in the standard error of the treatment effect, leading to non-significance of the treatment effect.

7.7.2 Structured by treatment

Although structuring by period is the most obvious way of introducing structure, this can also be applied to treatments. In parallel group trials, we have already met situations where we might wish to fit separate variances for each of the treatment groups. There is an exact analogy in the cross-over situation, where the variances may differ for some of the treatments. In addition, there may be good reason to suspect that the results from certain pairs of treatments may be more highly correlated than others if they have a similar mode of action. Thus, a more complicated structure for treatments than the simple compound symmetry may be highly plausible. This type of approach has been found to be particularly useful in the analysis of bioequivalence trials where treatment differences in reproducibility are highly relevant; examples of its application are available on the FDA website (www.fda.gov). We also present an example in Section 8.15.

7.7.3 Example: four-way cross-over trial

We will consider the four-way cross-over trial analysed in Section 7.4, which compared the effects of three drugs A, C and D and a placebo B on blood flow. This time, no values are set to missing, and therefore the study is more balanced (across random effects). Each treatment period lasted a week and was followed by a washout period also lasting a week. An analysis fitting a random effects model (i.e. a compound symmetry pattern) is compared with analyses fitting general covariance matrices. The covariance matrix is first structured according to periods and then according to treatments. Treatment and period effects are fitted as fixed effects in each analysis. In the first instance, we also include a carry-over effect as an additional fixed effect in the analysis.

Examination of the second model in Table 7.8 shows that data are more variable in the first period than in other periods. This is a common occurrence in clinical trials and is one reason why a run-in period is often used. The correlations between periods indicate a tendency for more widely separated periods to have lower correlations than those close together. The third model shows a higher variation for the placebo treatment (B) than for the three active treatments. Interestingly, the correlations involving treatment D are substantially lower than the correlation between treatments A, B and C.

Error model	Period/ treatment	Variances (×102)	Correlations	− 2 log likelihood (no. of parameters)	Estimated treatment differences (model SE)
Compound symmetry	1–4	36		544.6 (2)
General (across periods)				528.2 (10)
General (across treatments)				527.4(10)

Both general covariance models show significant improvements over the compound symmetry model when tested by likelihood ratio tests (, p = 0.03 and , p = 0.007) and might therefore be preferred.

A comparison of the three models with respect to the estimates of treatment differences shows that the estimates vary substantially when treatment C is involved. The same comment applies when the carry-over effects are compared.

Tests of the null hypothesis of equal treatment effects are all highly significant whichever model is used. For the compound symmetry model, the p-value is 0.003 compared with 0.005 for each of structuring by period and structuring by treatment.

The tests for equality of carry-over effects are more dependent on the choice of model, though none is statistically significant at conventional levels. The three models yield p-values of 0.54, 0.35 and 0.19.

All three models would therefore produce similar overall conclusions with respect to the presence of significant treatment differences without evidence of carry-over. Every model estimates the greatest levels of LVET with treatment A, followed by treatments C, B and D. The conclusions in respect of particular pairs of treatments would depend, however, on the model selected. On the basis of the likelihoods, the model employing structuring by treatment might be preferred, leading to a conclusion of similar higher levels of LVET for treatments A and C, with treatments B and D at similar lower levels. One would also conclude that results with treatment D were only weakly correlated with results on other treatments, which in turn are quite highly correlated.

In view of the absence of any significant carry-over in any of the models, it would seem reasonable to go on to consider models in which carry-over is omitted. We would advocate this even if the above model had demonstrated significant carry-over for reasons that are elaborated upon in Section 7.11. We note for the moment that this trial used reasonable washout periods, making physical carry-over of drugs unlikely, and that even if carry-over does occur, the simple carry-over model used may be inappropriate.

The use of the same three covariance structures as before, but without carry-over in the model, is summarised in Table 7.9. Many of the comments made on the previous models still apply. The patterns of covariances are similar, and structuring by period or by treatment produces a significant increase in the likelihoods compared with the compound symmetry pattern. All of the models give similar estimates of the treatment differences, which do not involve treatment C. The estimates involving treatment C are similar for the compound symmetry model and for the structured-by-treatment model but are substantially different with structuring by period. In view of the highest likelihood being obtained when structuring by treatment and the consistency of the estimates when compared with the compound symmetry model, this model might be tentatively accepted.

As noted in the corresponding model when carry-over was included, the correlations involving treatment D are relatively low. This treatment is also the one producing the lowest measurements. This may therefore increase further the plausibility of this model, suggesting as it does a different but less effective form of action.

All of the above results have been obtained using the Kenward–Roger option to correct for the fixed effects standard error bias. In many situations, this bias has been small in comparison with the use of the Satterthwaite option, but in this example, it is occasionally large. In the models in which we structure by periods, the standard errors of the treatment differences are over 30% larger using the Kenward–Roger option. In some other models, the influence is marginal. Further details can be obtained by comparison with the corresponding table in the first edition of this book for which the Kenward–Roger option was unavailable. We believe that use of the Kenward–Roger option is the correct approach, and there is indirect support for this view by consideration of the standard errors across the different models. Their variation is relatively small, and this compares with the substantial variation seen with other approaches. In the first edition, we also reported the ‘empirical’ standard errors (see Section 2.4.3) and raised questions of their validity in this context. For all but one of the 18 treatment comparisons in Table 7.9, the empirical standard errors are less than the Kenward–Roger standard error, and sometimes the inconsistency is gross. For example, the empirical standard error of C − D in the structured-by-period model is only 4.6 compared with 11.2 for the model-based standard error and 14.8 for the Kenward–Roger standard error. We therefore see no place for the use of unadjusted empirical standard errors with normally distributed, repeated measures data in relatively small datasets. The role of empirical standard errors with non-normal data and the adequacy of the corrections to the empirical standard errors available in PROC GLIMMIX are still an under-researched area.

Returning to the consideration of our findings in Tables 7.8 and 7.9, we believe that there is a case for basing our inferences on the model without carry-over, with the covariances being structured by treatment. Choosing the model from which to present findings is not always straightforward, however. Decisions concerning the inclusion of carry-over terms may be based primarily on how the trial was designed and, in particular, on the adequacy of washout periods. The choice between different covariance pattern models may be influenced by consideration of the likelihoods. However, statistical tests need not be the only factor determining model choice. The validity of the assumptions relating to a model is also important. For example, if we believe that periods and treatments are unlikely to have varying correlations based on past experience or if the trial is too small to give precise covariance estimates, then a compound symmetry structure possibly may be the one of choice.

It will always be a cause for concern when different models give qualitatively different conclusions, and this is always a greater danger when the data are relatively sparse. The different conclusions that were reached with different models and different approaches to the calculation of standard errors that we reported previously have, however, been resolved with the correction of the fixed effects standard error bias using the Kenward–Roger method.

This example makes the point that treating a multi-period, cross-over trial as repeated measures data with a covariance pattern that is structured by treatment provides an additional approach to analysis, which can be informative. For analyses that are conducted in the pharmaceutical industry for drug registration purposes, the requirement to specify the analysis plan in the trial protocol may be restrictive. This is likely to cause the compound symmetry model to be the one of choice for a primary analysis. It should be acceptable, though, to specify a secondary analysis that is structured by treatment, so that the interrelationships of responses to different treatments can be explored.

PROC MIXED; CLASS treat period patient carry;

TITLE ‘COMPOUND SYMMETRY’;

MODEL lvet=treat period carry/ DDFM=KENWARDROGER;

REPEATED period/ SUBJECT=patient TYPE=CS RCORR;

LSMEANS treat carry/DIFF PDIFF;

TITLE ‘STRUCTURED BY PERIOD’;

REPEATED period/ SUBJECT=patient TYPE=UN RCORR;

TITLE ‘STRUCTURED BY TREATMENT’;

REPEATED treat/ SUBJECT=patient TYPE=UN RCORR;

SAS code and output

PROC GENMOD;

CLASS treat period patient;

MODEL outcome/one=period treat/ ERROR=B;

REPEATED SUBJECT=patient/ WITHIN=period TYPE=CS MODELSE CORRW;

Algorithm converged.

NOTE: The scale parameter was held fixed.

As in the example considered in Section 6.4, the above output will always be produced, but it is irrelevant.

Algorithm converged.

NOTE: The scale parameter for GEE estimation was computed as the square root of the normalized Pearson’s chi-square.

7.9 Analysis of categorical data

To illustrate techniques, we will again take an example from Jones and Kenward (1989) and delete five observations from the second treatment period. The example is a placebo-controlled trial of a treatment for primary dysmenorrhoea. Thirty patients entered the trial, and in each treatment period, the amount of relief obtained was recorded as none or minimal (1), moderate (2) and complete (3). The data as we analyse them are summarised in Table 7.12.

Taking a fixed effects approach, a test of significance is most readily obtained using methods based on the analysis of an appropriate contingency table. A simple but inefficient way of producing such a contingency table would be to categorise the changes in the outcome variable from the first treatment period to the second as ‘worse’, ‘no change’ and ‘better’ and to tabulate this variable against the treatment sequence. The significance of the treatment effect could then be determined from this 3 × 2 contingency table, as in Prescott’s test. However, this configuration does not use the information that observations of ‘none’ and ‘complete’ in the two treatment periods represent a larger difference than between ‘none’ and ‘moderate’ or ‘moderate’ and ‘complete’. If we arbitrarily assign numbers of 1, 2 and 3 to the outcome categories, we can generate a 5 × 2 contingency table based on the change scores. The ‘obvious’ approach is to then apply a permutation t test (test for trend) to this table to assess the significance of the treatment effect. Application to the change scores presented in Table 12 gives p = 0.005.

In applying this test, it should be appreciated that the scores of − 2, − 1, 0, + 1 and + 2 are arbitrary. They should not be taken to imply that the difference between ‘none’ and ‘complete’ is twice as large as the difference between ‘none’ and ‘moderate’ nor that the difference between ‘none’ and ‘moderate’ is the same as that between ‘moderate’ and ‘complete’. For this reason, some statisticians may wish to replace the change scores with ranks and apply an (exact) Wilcoxon rank sum test. The choice will rarely make any practical difference, but it is clearly good practice to make this choice prior to analysis, rather than reporting the more favourable result. Note that the situation becomes more complicated when there are more than three categories for the outcome variable. Analysis could still be based on the change scores, but there would be an implicit strong assumption about the meaning of the intervals between the categories. Without such strong assumptions, many of the categories of change would be indistinguishable from each other, and a simplified 5 × 2 contingency table would result. Such an example is presented by Senn (2002).

The mixed models approach with random patient effects and fixed period and treatment effects, based on carrying out ordinal logistic regression, is now available through PROC GLIMMIX. The patient variance component is, surprisingly for a cross-over trial, estimated to be negative and therefore set to zero. The coefficient for the treatment effect, on the logistic scale, is 1.92, with a standard error of 0.57 (p = 0.003). On exponentiation, the estimate of the odds ratio is 6.8, with 95% confidence limits of 2.1 and 22.1. The interpretation of the odds ratio in this situation is that the estimated odds of being in a favourable outcome category when treated with the analgesic compared with placebo is 6.8, whether favourable is defined as complete relief or moderate/complete relief.

SAS code and output

PROC GLIMMIX; CLASS patient period treat;

MODEL outc=period treat/DIST=MULT LINK=CLOGIT SOLUTION OR;

RANDOM patient;

Note that, in this example, if the option DDFM = KR is used in the model statement, the denominator degrees of freedom erroneously appear as 1.

7.10 Use of results from random effects models in trial design

7.10.1 Example

We consider the results from the oral mouthwash trial, summarised in Table 7.3. The estimate of the patient variance component is 0.029 and that of the residual variance is 0.066. Thus, the estimated residual variance in a parallel group trial is 0.095, the sum of these two variance components. The expected standard error of the treatment difference in a parallel group trial with 34 patients per group (giving the comparable number of treatment periods to the cross-over trial) is

Only gold members can continue reading. Log In or Register to continue

Share this:

• Click to share on Twitter (Opens in new window)
• Click to share on Facebook (Opens in new window)

Like this:

Like Loading...

Related

Related posts:

1. Software for fitting mixed models
2. Multi-centre trials and meta-analyses
3. Other applications of mixed models
4. Normal mixed models

Stay updated, free articles. Join our Telegram channel

Join