31
Health Research Reporting Guidelines: An Overview
Introduction
This chapter will provide an overview of why health research reporting guidelines have been developed, what they are and why it is important for health researchers to be aware of them.
Need for Health Research Reporting Guidelines
When considering the rationale for the creation of health research reporting guidelines, it is important to first examine the basic principles that underpin the conduct of medical research. These are that medical research should advance scientific knowledge and lead to improvements in the treatment or prevention of disease. It is also important to recognise that a published research report is the only tangible evidence that a piece of research was carried out and, therefore, if research remains unpublished it might as well not have been undertaken.
Transparent Reporting of Health Research
A scientific manuscript should present sufficient data on how a particular piece of research was conducted and what results were found so that a suitably qualified reader is able to fully evaluate the information provided and reach his or her own conclusions about the findings of the research. Given the potential impact of research findings on health care delivery, researchers have a moral obligation to ensure that research is adequately reported. In fact, inadequate reporting borders on unethical practice when it leads to biased results receiving unwarranted credibility.
Poor Reporting of Health Research
The poor reporting of health research is not a new concern as the following quote from Donald Mainland in 1938 illustrates1:
‘…incompleteness of evidence is not merely a failure to satisfy a few highly critical readers. It not infrequently makes the data presented of little or no value’.
Poor reporting of health research can take a number of forms. The following are some common examples:
- Key information about how the research was conducted is missing, incomplete or ambiguous.
- The research is misrepresented; for example by disguising the design of the study or its power or the importance of the study findings.
- The interpretation of the research results is misleading (the authors have added a ‘spin’ to the study results).
- Only parts of the methods or findings are included in the published report (selective reporting). Most commonly, this involves reporting only those arms of the study that had a positive outcome.
There is ample evidence that poor reporting of health research is a widespread and ongoing problem. The well-quoted study of Chan and Altman,2 published in The Lancet in 2005, clearly illustrates the issue.
Chan and Altman conducted an analysis of 519 randomised trials published in Medline-indexed journals in December 2000. The authors found that power calculations, primary outcomes, random sequence generation, allocation concealment and handling of patient attrition were all items that were not adequately described in more than half of publications they reviewed. They also commented that the small sample sizes used in many of the trials were worrying, and that overall poor reporting of the methodological characteristics of the studies would prevent reliable quality assessment of many published trials.
It is worth mentioning at this point that there are other issues with the transparent reporting of health research. It is well known that researchers are less likely to write up studies that have less interesting or negative findings. It is also a fact that journals are less likely to consider such papers for publication even when they are written up and submitted to journals. In addition to this publication bias, outcome reporting bias exists, whereby the results that are reported favour those with statistically significant results.
The impacts of poor reporting are numerous, but the most notable effects are as follows:
- Difficulty in accurately assessing the value of published research because weaknesses have been hidden.
- Bias of the cumulative evidence. In particular, this has a major impact on the findings of meta-analyses, such as those undertaken by the Cochrane Collaboration.
- Difficulty verifying results. Published studies are not reproducible so findings cannot be verified.
What Are Health Research Reporting Guidelines?
Health research reporting guidelines are statements that provide advice on how to report research methods and findings. Given all the issues surrounding the poor reporting of health research discussed above, the aim of the guidelines is to encourage clear and transparent reporting of the research that has been conducted. The most widely recognised guidelines generally comprise a consensus of opinion from a group of experts. Researchers looking to publish papers should note that many biomedical journals make adherence to the relevant guideline a requirement of publication.
Most guidelines consist of a minimum set of items that are deemed necessary to allow assessment of the findings of a study and, if required, reproduction of that study. It is common for the guidelines to be presented as a narrative with an accompanying checklist of key items that should be addressed when reporting the study. The guidelines often also recommend a template flow diagram that can be used to report the progress of participants through the study as it progressed.
Overview of Major Health Research Reporting Guidelines
Randomised Controlled Trials
The first health care research reporting guidelines to emerge were those developed by the CONSORT (Consolidated Standard of Reporting Trials) Group. The CONSORT guidelines cover the reporting of parallel-group randomised-controlled trials. Development of the guidelines began in the 1990s and they were first published in 19963 with a major revision in 2001.4 Extensions of the 2001 CONSORT Guidelines have been developed for other types of study designs, interventions and data:
- Cluster trials5
- Non-inferiority and equivalence trials6
- Herbal medicinal interventions7
- Non-pharmacological treatment interventions8
- Harms9
- Abstracts10
- Pragmatic trials11
The most widely used CONSORT guidelines from 2001 consist of a 22-item checklist covering the content that should be described and reported in the title, abstract, introduction, methods, results and discussion of a written report of a randomised-controlled trial. These particular 22 items were selected by the CONSORT group because empirical evidence indicated that not reporting the information covered by these items was associated with biased estimates of treatment effect or because the information was deemed to be essential to judge the reliability or relevance of a study’s findings.
The guidelines also include a flow diagram that illustrates the passage of participants through a randomised-controlled trial. The flow diagram shows information from the main four stages of a trial (enrolment, intervention allocation, follow-up and analysis). The diagram explicitly shows the number of participants, for each intervention group, included in the primary data analysis. Inclusion of these numbers allows the reader to judge whether the authors have done an intention-to-treat analysis.
For more detail of the CONSORT guidelines and extensions, the reader is directed to the CONSORT website: http://www.consort-statement.org.
Observational Studies
The main guidelines available for observational studies are those developed by the STROBE (Strengthening the Reporting of Observational studies in Epidemiology) initiative. The STROBE statement covers the essential items to report in cohort, case-control and cross-sectional studies.12 The STROBE statement consists of a checklist of 22 items that relate to the title, abstract, introduction, methods, results and discussion sections of the research report. Eighteen items are common to cohort studies, case-control studies and cross-sectional studies and four are specific to each of the three study designs. The aim of the STROBE statement is to provide guidance to authors about how to improve the reporting of observational studies and to facilitate critical appraisal and interpretation of studies by reviewers, journal editors and readers. A flow diagram is also included.
Diagnostic Accuracy Studies
The reporting of diagnostic accuracy studies has been investigated by the STARD (Standards for the Reporting of Diagnostic Accuracy Studies) initiative.13 In common with the other guidelines already discussed, the objective of the STARD initiative is to improve the accuracy and completeness of reporting of studies of diagnostic accuracy. The initiative also aims to help readers of such studies to assess the potential for bias in the study (internal validity) and to evaluate the study’s generalisability (external validity).
The STARD statement consists of a checklist of 25 items and recommends the use of a flow diagram to describe the design of the study and the flow of patients through the study.
Systematic Reviews and Meta-Analyses
Systematic reviews and meta-analyses are becoming increasingly popular methods for reviewing and analysing data pooled from individual studies. Like any other type of research that is undertaken, particularly one that is observational in nature, the meta-analysis of evidence can be flawed. Therefore, the process by which systematic reviews and meta-analyses are carried out has undergone scrutiny.
The Quality of Reporting of Meta-analyses (QUOROM) conference was held in 1996 to address the conduct and quality of meta-analyses of randomised-controlled trials. The conference resulted in the QUOROM statement, which was published in 1999.14 The statement includes a checklist and a flow diagram. The checklist describes how to present the abstract, introduction, methods, results and discussion sections of a report of a meta-analysis. The checklist is organised into 21 headings and subheadings to encourage authors to provide readers with information on searches, selection, validity assessment, data abstraction, study characteristics, quantitative data synthesis and trial flow. Authors are asked to provide a flow diagram providing information about the number of randomised-controlled trials identified, included and excluded, and the reasons for excluding them.
Meta-analyses of observational studies are also undertaken. A workshop was held in the United States in 1997 to examine the reporting of meta-analyses of observational studies and to make recommendations to aid authors, reviewers, editors and readers. The workshop resulted in the publication of the MOOSE (Meta-analysis of Observational Studies in Epidemiology) guidelines15 that include a 35-item checklist containing specifications for what information to include in the background, search strategy, methods, results, discussion and conclusion of meta-analyses of observational studies.
Future of Health Research Reporting Guidelines
To date, over 80 health research reporting guidelines have been constructed. While they cover a broad spectrum of health research, some areas are better covered than others. It is important that the impact of guidelines on the quality of reporting of health research is now assessed and that the guidelines are continually reviewed and updated. Unfortunately, it is currently believed that most of the guidelines have had limited impact on the quality of published studies. There is also concern that little guidance is available on how guidelines should be developed and that not all guidelines are of good quality. Thus, there is a need for a tool to assess the quality of guidelines and for a ‘seal of approval’.
It is in this setting that the EQUATOR network was set up in 2006. The EQUATOR network (www.equator-network.org) grew out of the work of CONSORT. It is an umbrella organisation for health research guidelines. To quote from the EQUATOR network website:
‘The EQUATOR Network seeks to improve the quality of health research literature by promoting transparent and accurate reporting in scientific journals’.
The EQUATOR network provides an internet-based resource centre bringing together all currently available health research reporting guidelines in one place for use by researchers, journals and guideline developers. The EQUATOR network aims, through education and training, to support the development, dissemination and implementation of health research reporting guidelines. They also hope to assess how journals implement the guidelines and audit guideline quality.
What Health Care Research Guidelines Mean to Researchers
As mentioned earlier, researchers looking to publish papers must be aware that, in addition to compliance with a journal’s policies on declaration of research funding, author conflicts of interest and registration of clinical trials on websites such as ClinicalTrials.gov (http://www.clinicaltrials.gov), many biomedical journals make adherence to the relevant guideline an absolute requirement of publication. However, regardless of your intended target journal and its particular requirements, it is critical that, in order to ensure that your published research is of the highest ethical standard from a reporting perspective, the relevant guideline is both consulted and adhered to. Everyone impacted by health research has a right to trust the integrity of a published research report and this integrity is both encouraged and enabled by the use of health research guidelines. Also be aware that for publication in journals that readers trust and respect,‘box ticking compliance’(i.e. saying you have covered something by ticking the item on a checklist) will not be sufficient and is very likely to result in rejection of your paper.
