8.1 Purpose

Quality of care studies include either patients or health professionals or occasionally both, with the following general objectives:

• To examine patient satisfaction or experiences with a clinical service
  
• To evaluate a hospital/clinic system, or treatment or care pathway, by examining the experiences of both patients and health professionals
  
• Assessment of delivery of services by health professionals (e.g. nurses and doctors)
  
• To examine the impact of a service on clinical outcomes using the perspectives of both patients and clinicians
  
• To identify problems in a treatment or care pathway, make changes, and then assess the impact of the change

Many of these studies have the ultimate aim of making recommendations for improvement, or sometimes for further studies. Researchers often make the evaluation in their own institution, where the potential for change is greater, rather than regional or national recommendations, though others can use the findings and consider whether they could apply elsewhere. Quality of care studies are a major feature of health services research. Further details, including guidelines on how to report such studies, can be found elsewhere [1–4].

8.2 Design

There is a wide variety of designs, including qualitative studies (see page 12), but perhaps, the most common is the cross-sectional study (Chapter 5). They are often quick to undertake (<1 year) and can be conducted in a single centre or a few centres (usually hospitals or general practice/family physician units).

As with all observational studies, specifying the sampling frame is essential, but this can often be relatively easy to do in quality of care studies, for example, all patients attending an emergency department January to June 2012, or all patients being treated for diabetes on a particular clinic list.

There are several key aspects to the design and conduct (Box 8.1).

8.3 Measuring variables

Unlike the observational studies in previous chapters, which had relatively well-defined exposures and outcomes, in quality of care studies, many factors are psychometric measures, based on perceived experiences, and these can be difficult to assess. They require patients or health professionals to consider many features of the health service of interest, and some of these features are likely to be correlated. Therefore, attempting to find specific reasons for poor satisfaction, or why certain services perform well (or badly), is not usually straightforward.

All measures associated with quality of care studies, except qualitative research, fall into one of the three categories: ‘counting people’, ‘taking measurements on people’, and time-to-event, though the latter is the least common.

Carefully structured questionnaires are essential to the success of quality of care studies and to obtain accurate data. While open-ended questions may be useful in eliciting general comments from the participants or finding information not previously known, it is best to have mostly questions requiring fixed responses. A Likert-type scale is commonly used, where the participant ticks one of several options in response to a statement. For example:

‘I did not have to wait long before being seen by a doctor’

1. Strongly disagree
   
2. Disagree
   
3. Neither agree nor disagree
   
4. Agree
   
5. Strongly agree

Other options for response, depending on the type of statement, could be:

• Not important, somewhat important, very important, or essential
  
• Never, rarely, sometimes, most of the time, or always

This type of outcome can be analysed as ‘counting people’ endpoints (categorical data). Each item can be analysed separately, and the percentage of individuals can be reported in each group. Alternatively, focus could be on the extremes, whereby categories are combined (1 + 2 vs. 3 + 4 + 5, or 1 + 2 + 3 vs. 4 + 5), which creates a binary measure. Responses from several items could be combined (e.g. simply summed) and converted into a score (as with quality of life measures used in Section 5.9) to create a continuous measure. Another type of continuous measure involves asking people to indicate their response on a visual analogue scale (0–1 or 0–100, e.g. 0 = worst experience and 1 = best experience). All of these approaches can be analysed using methods presented in Chapters 2 to 4 if describing a single group of participants, or looking at associations among them.

If developing a study-specific questionnaire, items should not be repetitive, that is, asking the same thing in slightly different ways. Also, the layout and format of a self-completed questionnaire should be attractive, with sufficient space between questions to avoid the text looking cramped on the page. A poorly laid out questionnaire can make it difficult to complete, and so lead to lower response rate or missing data.

8.4 Collecting the data

Data are often obtained directly from the participants (patients or health professionals), using self-completed questionnaires or interviews.

Questionnaires that have already been developed can be obtained from the published literature and the internet, but researchers may also develop their own, to tailor it to their specific institution, and so bring out key features that led to the research project. Self-completed questionnaires have the advantage that they can be sent to a larger number of people and can cover a wider range of topics, but the main limitation is a potentially low response rate.

Alternatively, face-to-face interviews can encourage participation, reduce the non-response rate, and minimise missing data, but require staff to undertake them. This may involve training, which is particularly important when dealing with psychometric measures and the potential for interviewer bias (see Box 1.6). When quality of care studies are conducted in a specific hospital department, patients could be asked to complete the questionnaire while waiting to see the health professional or afterwards.

8.5 Sample size

Study size is often limited by the size of the available sampling frame, for example, the number of stroke patients on an outpatient list. Also, the desired length of the study will have an effect, because if it might take 5 years to accrue a specified study size, the researchers may need to consider whether or not to proceed. However, quality of care studies are often exploratory, so in general, the more participants, the better.

It is possible to specify an important endpoint and calculate a sample size using the same method as for cross-sectional studies (Box 5.8); See Examples 1 and 2.

8.6 Analysing data

The main results of quality of care studies tend to be descriptive, but additional analyses can examine associations (e.g. differences between males and females, or seniority of staff), and the methods of analyses are the same as those in Chapter 3 (effect sizes, 95% confidence intervals (CIs) and p-values).

Although many of the outcomes (e.g. psychometric measures) are different to clinical or physiological outcomes presented in previous chapters, the statistical methods in Chapters 2 and 3 still apply (particularly linear and logistic regression). Other analyses could involve obtaining correlation coefficients between two continuous factors (to quantify the relationship), measures of agreement between people or techniques that measure the same outcome (Bland–Altman statistic or Kappa statistic), or multivariate methods such as principal components or factor analysis (which are often exploratory analyses that aim to reduce a large number of factors to a few dimensions) [5].

8.7 Example 1: patient satisfaction with a service

Box 8.2 [6] is an example of a common type of quality of care study, focussing on a single department within a single clinic (quality of emergency care). The study took only 1 month to complete. The researchers provided a sample size calculation assuming 63% of patients would have ‘favourable experiences’, and the 95% CI width was required to be ±3%, yielding a target sample size of 995 (Box 5.9).