Implementation of the Benefit–Risk Assessment Template by Mature Agencies

, Sam Salek2 and Stuart Walker3



(1)
Centre of Regulatory Excellence, Duke-NUS Graduate Medical School, Singapore, Singapore

(2)
Department of Pharmacy, University of Hertfordshire, Hatfield, UK

(3)
Centre for Innovation in Regulatory Sciences, London, UK

 




Introduction


The current climate in regulatory science seeks transparency of decision-making and communication to stakeholders for accountability. The results from Chap. 3 showed that both regulatory agencies and pharmaceutical companies believe that a benefit–risk framework would enhance the quality (transparency and consistency) of decision-making; provide documentation for a systematic, structured discussion; and act as a tool for communication. A tool was thus developed (Chap. 4) with inputs from the Consortium (consisting of TGA, Health Canada, Swissmedic, and HSA), and the resulting universal Benefit–Risk (BR) Template was designed to enhance communication and documentation of benefit–risk decisions. This study aims to review the potential value of the BR Template for regulatory agencies.


Objectives


The objectives are to:



  • Examine the value of the BR Template for documenting the benefit–risk assessment decision of new active substances during the review process


  • Evaluate the BR Template as a tool to communicate the benefit–risk decision to other stakeholders in a systematic, structured manner


  • Determine if the BR Summary section of the BR Template is adequate as a stand-alone tool to communicate a benefit–risk decision to stakeholders


Methods


Three similar-sized regulatory agencies agreed to participate in this prospective study which was conducted as non-comparative evaluation. The study package, consisting of the BR Template (which included the Benefit–Risk Summary section) and User Manual described in Chap. 4, were sent to the three agencies. The reviewers in the respective agencies selected a product undergoing active evaluation and completed their own assessment report as well as the BR Template. Following this process, the reviewers were sent a study evaluation tool (Fig. 5.1), which they completed and returned.

A333179_1_En_5_Fig1a_HTML.gifA333179_1_En_5_Fig1b_HTML.gifA333179_1_En_5_Fig1c_HTML.gifA333179_1_En_5_Fig1d_HTML.gifA333179_1_En_5_Fig1e_HTML.gifA333179_1_En_5_Fig1f_HTML.gifA333179_1_En_5_Fig1g_HTML.gifA333179_1_En_5_Fig1h_HTML.gifA333179_1_En_5_Fig1i_HTML.gifA333179_1_En_5_Fig1j_HTML.gifA333179_1_En_5_Fig1k_HTML.gifA333179_1_En_5_Fig1l_HTML.gifA333179_1_En_5_Fig1m_HTML.gifA333179_1_En_5_Fig1n_HTML.gifA333179_1_En_5_Fig1o_HTML.gif


Fig. 5.1
The study evaluation tool

The study evaluation tool was developed as a questionnaire consisting of 56 questions divided into four sections, namely, user-friendliness, documentation, applicability, and general comments. There were three systems of rating:



  • Excellent, good, fair, and poor (comments to be provided for ratings of “fair” and “poor”)


  • Fit for purpose, fit for purpose with modifications required, and not fit for purpose (comments to be provided for the latter two choices)


  • Yes and no (comments to be provided for rating “no”)

Most of the questions had an open field for comments, allowing the participants to provide any issues of concern or relevant points that were not addressed by the questionnaire. The questionnaires were sent via e-mail directly to the participants. Completed responses were received via e-mail, as instructed to the participants. All responses were collated into a single group, and outcomes were presented according to their respective sections in the study evaluation tool. All data were expressed as direct ratings provided by the responders. Free-text comments were collated and presented in appropriate categories.

This was designed as an exploratory study, and the outcomes were interpreted to provide qualitative inferences relating to the objectives. No statistical analyses were planned or conducted.


Results


The outcomes will be presented in four parts:



  • Part I: User-friendliness of the BR Template


  • Part II: Appropriateness (fit for purpose) of the documentation


  • Part III: Applicability of the BR Template


  • Part IV: Usefulness of the BR Template

None of the agencies used visualizations, and hence no outcomes were documented for these items in the survey tools.


Part I: User-Friendliness of the BR Template


The BR Template has three features that assist the user in locating selected pages within the document, namely, the tabs at the top of each page, the “Go to page” button, and the page thumbnails (Fig. 5.2).

A333179_1_En_5_Fig2_HTML.gif


Fig. 5.2
Navigation functions of the BR Template

The thumbnails were not used, while the tabs and “Go to page” buttons were rated either good or excellent (Table 5.1). The agencies suggested that the use of bookmarks for the sections and subsections would be preferable, as well as a search function for identifying keywords within the document.


Table 5.1
Practicality of the navigation functions





























Agency

Tabs at top of page

“Go to page” button

Page thumbnails

A

Excellent

Excellent

Did not use

B

Good

Good

Did not use

C

Good

Good

Did not use

In addition to navigation features, the BR Template incorporates four functions to print, e-mail, view the form (Fig. 5.3), and auto-populate information for fields requiring the same inputs (Fig. 5.4).

A333179_1_En_5_Fig3_HTML.gif


Fig. 5.3
Document support functions of the BR Template


A333179_1_En_5_Fig4_HTML.gif


Fig. 5.4
Auto-populate function of the BR Template

Conclusion on the usefulness of the print, e-mail, and view functions were not provided as Agency A did not use the former two functions, Agency B experienced a technical issue that prevented them from getting back to the document after using these three functions, while Agency C rated these support functions as good. However, the auto-population function was considered useful by all, being rated as good or fair (Table 5.2).


Table 5.2
Usefulness of the document support functions


































Agency

Print function

E-mail function

View full form function

Auto-populate function

A

Did not use

Did not use

Excellent

Good

B

Poor

Poor

Poor

Fair

C

Good

Good

Good

Good

The User Manual was provided as a guide to help the reviewer in completing the BR Template and included a glossary of commonly used terms. Agencies A and C rated the manual as good or fair in terms of clarity, comprehensiveness, and applicability (Table 5.3). Overall, the agencies believed more details are needed, e.g., case studies and examples to improve the usefulness of the User Manual. Agency B would like to have more guidance regarding the intention of the BR Template, the level of details of the outcomes, and the type of information required. Agency C commented on the need for examples to show how weighting and valuing may be carried out as this concept is new to the agency.


Table 5.3
Appropriateness of the User Manual





























Agency

Clarity of instructions

Comprehensiveness

Applicability

A

Good

Fair

Good

B

Poor

Poor

(Not reported)

C

Good

Good

Good

Other comments received on enhancing the technical aspects of the template include:



  • Allow changes in fonts (e.g., size, underlining, italicizing), use of bulleted listings within text boxes, and use of the tab key within a cell in the tables


  • Allow for the use of the tools for commenting and marking up (highlighting and crossing out functions) in Adobe Acrobat Professional as these would be useful for supervisors or managers recommending revisions to the document


  • Ensure that the text copied and pasted from a Word document retains the original formatting (underlining, italicizing, symbols)


Part II: Appropriateness (Fit for Purpose) of the Documentation


The BR Template incorporates five conclusions that are considered important in making a benefit–risk decision, namely, the background (decision context), quality, nonclinical, human pharmacology, and clinical conclusions. Of the five, the agencies believed the clinical conclusion is fit for this purpose (Table 5.4). For the remaining four, the template could allow for more details as the actual benefit–risk assessment was carried out in much greater depth and the sections may not accommodate such a level of information.


Table 5.4
Documentation of relevant information supporting the benefit–risk decision







































Agency

Background information

Quality conclusion

Nonclinical conclusion

Human pharmacology conclusion

Clinical conclusion

A

Fit for purpose

Modifications required

Modifications required

Modifications required

Fit for purpose

B

Fit for purpose

Modifications required

Modifications required

Modifications required

Fit for purpose

C

Modifications required

Fit for purpose

Fit for purpose

Fit for purpose

Fit for purpose

Agency B commented that these were the only sections to discuss the contributions from quality, nonclinical, and human pharmacology in the entire template, whereas the rest of the template is dedicated to clinical benefits and risks. However, Agency B believed that if the intention of the BR Template was to feature only a high-level summary of the significant findings, then it would suffice. It was mentioned that a considerable amount of evaluation was conducted for those aspects for a new active substance and that this section would not be able to accommodate these findings. Without allowing the reviewer to provide details on the relevant studies, it would be difficult to explain the relevance of the reported issues and concerns. For completeness, Agency A recommended the inclusion of subheadings for pharmacokinetics, pharmacodynamics, and drug interactions to further guide the reviewer. Agency C preferred the background information to allow a discussion on related applications and products that may contribute to decision-making.

It should be noted that the BR Template was designed to present and communicate only the significant findings that would affect the benefit–risk decision and that the corresponding details would be expected to be available from the original assessment report for the product.

The template was seen by the agencies as being able to document benefits and risks identified by sponsors and the reasons for including or excluding them (Table 5.5).


Table 5.5
Documentation of benefits and risks




























 
Benefits

Risks

Agency

Reasons for inclusion or exclusion of all benefits

Relevant benefits as identified by sponsor

Selected list of benefits to be included in the benefit–risk assessment

Reasons for inclusion or exclusion of all risks

Relevant risks as identified by sponsor

Selected list of risks to be included in the benefit–risk assessment

A

Fit for purpose

Fit for purpose

Fit for purpose

Only gold members can continue reading. Log In or Register to continue

Stay updated, free articles. Join our Telegram channel

Mar 26, 2017 | Posted by in GENERAL & FAMILY MEDICINE | Comments Off on Implementation of the Benefit–Risk Assessment Template by Mature Agencies

Full access? Get Clinical Tree

Get Clinical Tree app for offline access