, Sam Salek2 and Stuart Walker3
(1)
Centre of Regulatory Excellence, Duke-NUS Graduate Medical School, Singapore, Singapore
(2)
Department of Pharmacy, University of Hertfordshire, Hatfield, UK
(3)
Centre for Innovation in Regulatory Sciences, London, UK
Introduction
The assessment of benefits and risks of medicinal products for regulatory approval remains largely a qualitative exercise, although there are ongoing initiatives to introduce a quantitative approach into the review process. Given the current setting, it is important that both the processes and the benefit–risk decisions are transparent and communicated to stakeholders for accountability. Hence, there is a need to find appropriate tools to enhance communication in a manner that it would uphold transparency, consistency, and standards.
Previous studies (Chaps. 3 and 4) showed that the need for effective communication can be carried out through a benefit–risk framework supported by a documentation tool. The UMBRA Template was designed to enhance the communication of decisions in support of the eight-step framework for the assessment of benefits and risks. However, this template was based on the EMA guidance, and the details required may be challenging for emerging regulatory agencies that are currently building up their scientific capabilities and regulatory processes.
The potential use of the BR Summary Template as a stand-alone in the emerging markets was proposed for the purpose of documenting, understanding, and exchanging information on benefit–risk decision with other similar-sized agencies (Chap. 5). Therefore, this study aims to review the usefulness of the BR Summary Template (a collation of relevant conclusions leading to the final benefit–risk decision) in communicating benefit–risk decisions by the Health Sciences Authority (HSA) of Singapore.
Objectives
The objectives of this study were to:
Determine the practicality of documenting benefit–risk assessment for abridged applications in HSA using the BR Summary Template
Examine the potential of the BR Summary Template for communicating benefit–risk balance and conclusions to stakeholders
Assess the effectiveness of the User Manual in guiding a reviewer to complete the BR Summary Template
Methods
This research was conducted as a retrospective and non-comparative study. The study protocol was made available to all the participants of this study.
The UMBRA BR Template was reviewed and the Benefit–Risk Summary section extracted to produce the BR Summary Template (Appendix III). Both the User Manual and the study evaluation tool (as described in Chap. 5) for the BR Template were changed accordingly to support the BR Summary Template. The study package, namely, the study protocol, BR Summary Template, revised User Manual (Appendix IV), and the revised study evaluation tool (Appendix V), were sent to 16 clinical reviewers in HSA (Therapeutic Products Branch) involved in the assessment of benefit–risk balance and the registration of medicines. The reviewers were asked to identify an appropriate product application based on the following criteria:
New drug application which requires a benefit–risk evaluation
An abridged review, applicable to products having obtained a marketing approval in at least one country
Regulatory decision (having received marketing approval or confirmed benefit–risk decision) obtained within the last 3 months
The reviewers transferred the relevant information required for the BR Summary Template from the completed clinical assessment reports (as per current processes in HSA) with the support of the User Manual. Following this transfer, the reviewers completed the study evaluation tool.
All responses were collated into a single group, and the outcomes were presented according to their respective sections in the study evaluation tool. All data were expressed as percentage over number of responders for that item. Free-text comments were collated and presented in appropriate categories when necessary.
This was designed as an exploratory study, and the outcomes were interpreted to provide qualitative inferences relating to the objectives. No statistical analyses were planned or conducted.
Results
A total of 12 responses (75 %) were received by August 2013. Of the four who did not respond, one was transferred to another unit, two did not have applications that met the criteria, and the remaining one did not respond. Most (75 %) of the responders had between 1 and 5 years of working experience in the agency, with one having less than a year and two having more than 5 years. As the reports were written independently, the responses actually represented the evaluation of ten different products reviewed via the abridged route.
The outcomes will be presented in four parts:
Part I: User-friendliness of the BR Summary Template
Part II: Appropriateness (fit for purpose) of the documentation
Part III: Applicability of the BR Summary Template
Part IV: Suggested amendments to the BR Summary Template
Part I: User-Friendliness of the BR Summary Template
The template has two functions to help users navigate the document, namely, the “Go to page” button and page thumbnails to locate a specific page (Fig. 6.1). These are aimed at reducing the effort required to move between different sections.
Fig. 6.1
Navigation functions of the BR Summary Template
The “Go to page” button appeared to be the more useful, as 83 % of reviewers rated it either good or excellent (Fig. 6.2). For the page thumbnails, 58 % indicated it as fair, or it was not used as it was commented that the thumbnail icons were too small to decipher the contents and bookmarks might have been more effective, although none rated the BR Summary Template as not user-friendly. There was a suggestion to include a “Back” button to the content page or another primary page.
Fig. 6.2
Practicality of the navigation functions
The User Manual was provided to guide the reviewer on the steps to complete the template, as well as to clarify the common terms used in the template. The majority of the responders (between 75 and 100 %) rated the clarity, comprehensiveness, and applicability of the User Manual as “good” (Fig. 6.3).
Fig. 6.3
Appropriateness of the User Manual
None rated the manual as poor in any of the three parameters. Comments received included the consideration to provide examples or a case study in the manual to better illustrate the use of the template. An inexperienced reviewer might find the manual insufficiently comprehensive. Even though the User Manual provided instructions with regard to assigning relative importance to benefit and risk parameters, the lack of experience by the reviewers prevented them from effectively completing the BR Summary Template in this aspect.
Part II: Appropriateness (Fit for Purpose) of the Documentation
The appropriateness of a template is dependent on its capability to present the processes leading to the final benefit–risk conclusion in a structured and systematic manner. In documenting the various conclusions, the BR Summary Template was largely thought to be fit for purpose (92–100 %, Fig. 6.4).
Fig. 6.4
Documentation of relevant information supporting the benefit–risk decision
One modification suggested was to clarify the difference between the clinical conclusion section and the overall conclusion for benefit–risk balance, as it might appear redundant if misunderstood. The other modification was to make available more guidance on writing the nonclinical conclusion as some reviewers were not familiar with providing details for this section.
In documenting the benefits and risks for the product being evaluated, 92–100 % of the responders believed the template is able to achieve the purpose (Fig. 6.5). For documenting relevant benefits and risks as identified by the sponsor, one responder was unsure as to the usefulness of this as the reviewer would eventually indicate the benefits and risks that are to be included for assessment and hence rated these two parameters as not fit for purpose. However, the reasons for listing benefits identified by the sponsor and the reasons for inclusion or exclusion by the reviewer are both for transparency and to provide more fully the rationale for the benefit–risk decision. Another responder felt that there must be greater clarity in defining risks in the template as those considered critical to the benefit–risk assessment and as a result rated the documentation of inclusion or exclusion of all risks as being not fit for purpose.
