NON THEMED
Patricia Henley
Louise Mawer
Lucy Saunderson
COMPLIANCE IN SCIENCE – A JOB FOR MISS MARPLE?
An investigation into the reporting of randomised clinical trials RESEARCH PRACTICE GROUP PROFILE The RQA Research Practice Group (RPG) was put together through volunteers from the RQA as a means of disseminating best practice across the commercial and non-commercial worlds; highlighting differences between the two and learning from the methodologies employed by the different worlds. We held our first kick-off meeting in London in January 2019, alongside a teleconference to one member in the USA. Unfortunately, this was prior to all-consuming work activities overtaking two of our team, reducing us to just three amigos! The relationship, similarities and differences in research approaches between commercial and academic or non-commercial organisations were discussed, with the conclusion that not all differences resulted from resources and costs, but that culture, attitude to risk and the ultimate fate of the data were key in these aspects. The non-commercial world is concerned with looking for the ‘right’ answer to the study question, regardless of the impact
26 | QUASAR | JANUARY 2020
on the product, or the impact on the clinical development area. The commercial world, whilst still looking for the ‘right’ answer, must also answer to shareholders which creates a slightly different focus. Rightly or wrongly, there is a perception that non-commercial organisations follow different rules for the conduct of their trials, which just highlights the disparity between the two worlds. In looking to explore these similarities and differences, the RPG identified differences in the adopted approach between commercial regulatory submissable reports and scientific publication guidelines. This resulted in a group hypothesis that compliance and quality issues were more commonly reported in regulatory submissions than research publications used by healthcare professionals to inform treatment and prescribing guidelines. The RPG proposed to research this hypothesis and then develop potential underlying causes (beyond cost and resource) which may influence the willingness to report on non-compliance. This article forms a summary of the work completed to date.
INTRODUCTION Academic clinical trials are routinely published according to the Consolidated Standards of Reporting Trials (CONSORT) statement, a minimum set of recommendations for reporting randomised trials. In contrast, pharmaceutical and biotech organisations predominantly use the ICH E3 Guideline for registration study reports, as required by the regulatory receiving authorities. What are the similarities and differences between these standards? Does science prevail over compliance issues in research? Do scientific research articles give an accurate snapshot of a clinical trial? The RQA RPG set about investigating this mystery…
RULES AND MORE RULES: ICH E3 AND CONSORT GUIDANCE Auditors and those involved in clinical trial research in the commercial setting will be most familiar with the ICH E31 guidance: Structure and Content of Clinical Study
NON THEMED Reports www.ema.europa.eu/en/documents/ scientific-guideline/ich-e-3-structure-contentclinical-study-reports-step-5_en.pdf It’s a short 48 pages of principles and guidance which, if followed, allows the compilation of a single integrated study report ‘acceptable to all regulatory authorities of the ICH regions’. Although the guideline is mainly aimed at efficacy and safety trials, it can be applied to other kinds of trials resulting in (according to the guidance itself ), ‘the development of a report that is complete, free from ambiguity, well organised and easy to review’. The CONSORT2 statement www.consort-statement.org/ includes 25 items that are recommended for inclusion in publications. Supported by a short guideline and example flow chart (Example Figure 1), CONSORT is accepted as the basis for scientific publications by a wide variety of journals.
CONSORT trial descriptions recommend (amongst other things) inclusion in the publication title that the trial is randomised, the scientific background, rationale and trial objectives alongside the trial design, locations and interventions with sufficient detail to permit replication of the study as well as randomisation details, study results (including a description of the statistical analysis), trial limitations and generalisability of the findings. Whilst a comparison of CONSORT requirements against ICH E3 requirements gave few differences, in looking at the relationship of items for inclusion in reports/publications the other way around, additional content was identified in ICH E3 regarding quality issues, regulatory approvals and protocol deviations. In addition, more details were requested regarding the breakdown of adverse events.
This is not to say these items would not be considered in publications, but journal word restrictions would limit this detail, though it is acknowledged journal websites with the right subscriptions permit readers to access supplementary reference material held alongside the article and full trial protocol.
BEST MADE PLANS… A review of published articles available in popular scientific journals was completed by the RPG to evaluate compliance with the CONSORT recommendations, and in consideration of the extent to which GCP and compliance issues were described. We initially planned to review the number of journals for review of articles in the period January to March 2019 however, with workload, time constraints and a publication deadline looming, we cut our losses, and, in a deviation to our initial plan,
FIGURE 1. SAMPLE CONSORT FLOW DIAGRAM
This number of patients assessed for eligibility (N)
This number of patients progressed to randomisation (n)
Allocated to intervention A (n) • Received allocated intervention (n) • Did not receive with reason X (n) • Did not receive with reason Y (n)
Intervention A patients did not complete the trial (n) • Withdrew (n) • Were withdrawn by the physician (n) • Were lost to follow-up (n)
(n) Completed the study (n) Allocated A and in the study analysis
Intervention A excluded from the analysis (n) • Reason 1 (n) • Reason 2 (n) • Reason 3 (n)
Allocated to intervention B (n) • Received allocated intervention (n) • Did not receive with reason X (n) • Did not receive with reason Y (n)
Intervention B patients did not complete the trial (n) • Withdrew (n) • Were withdrawn by the physician (n) • Were lost to follow-up (n)
(n) Completed the study (n) Allocated B and in the study analysis
Intervention B excluded from the analysis (n) • Reason 1 (n) • Reason 2 (n) • Reason 3 (n)
This number did not enter the next trial phase (n) • Failed inclusion criteria no.1 (n) • Failed inclusion criteria no.3 (n) • Withdrew (n) • Other reasons (n)
Allocated to intervention C (n) • Received allocated intervention (n) • Did not receive with reason X (n) • Did not receive with reason Y (n)
Intervention C patients did not complete the trial (n) • Withdrew (n) • Were withdrawn by the physician (n) • Were lost to follow-up (n)
(n) Completed the study (n) Allocated C and in the study analysis
Intervention C excluded from the analysis (n) • Reason 1 (n) • Reason 2 (n) • Reason 3 (n)
JANUARY 2020 | QUASAR | 27
NON THEMED FIGURE 2. COMPARISON OF ICH E3 REPORTING GUIDANCE AND CONSORT PUBLICATION RECOMMENDATIONS
LIKE • Introduction/Overview • Methods - Design, Participants - Interventions - Outcomes - Randomisation/Blinding • Statistical Methods • Results discussion • Conclusion.
settled on articles identified in the New England Journal for Medicine and British Medical Journal to prepare this interim summary, before extending our review for the development of a paper for publication (wish us luck or get in touch to offer help! Contact the RPG via the RQA office info@therqa.com The issue index on the journal website provided details of all previous issues. All journal issues in the defined period were reviewed for randomised clinical trial articles. All randomised trial articles were selected for review.
SOMEWHAT ALIKE
UNLIKE
•C ompliance Description • Deviation Evaluation • Ethical Considerations.
• Patient Disposition Tables and CONSORT Diagram • Adverse Event Summary Tables and Important Harms.
RESULTS OF THE REVIEW 45 articles of interest were identified from the indexes of selected journals for the period January to March 2019. 34 of the 45 articles related to trials of medicinal products whereas, 11 were non-medicinal trials: five studies compared surgical interventions, three studies compared non-surgical interventions. The remaining three studies did not involve medical or surgical interventions e.g. prescribing. One trial had two papers but was assessed only once as the content was substantially similar.
42 articles referenced consent (without issue), whilst consent was not applicable to one trial, and consent was waived by the concerned Ethics Committees for one further article. Informed consent was not referenced in one article. Only approximately one third of the articles reviewed (12/34) referred to GCP. Six articles referred to protocol deviations: two in respect of eligibility issues, one because of patient refusal to continue treatment, one did not give a reason for deviations, and one because non-compliance resulted in the exclusion of data, as a result
TABLE 1. EXTRACT OF ITEMS REVIEWED FOR CONSORT, GCP AND PROTOCOL COMPLIANCE
ARTICLES RELATING TO RESEARCH FROM 1ST JANUARY 2019 TO 31ST MARCH 2019
NEW ENGLAND JOURNAL OF MEDICINE
BRITISH MEDICAL JOURNAL
Number of studies in medicines
34/40 (85%)
0/5 (0%)
Number of articles referencing patient consent
40/40 (100%)
4/5 (80%)
Number of articles referencing randomisation in the title
8/40 (20%)
5/5 (100%)
Number of articles complying with CONSORT details relating to randomisation (missing 2 or more elements from CONSORT elements 8 to 10)
8/40 (20%)
5/5 (100%)
Number of articles referring to non-compliance, deviations or similar issues
4/40 (10%) (all medicinal studies)
2/5 (40%)
Number of articles referencing GCP
12/40 (35%) (only medicinal studies in sample)
0/5 (0%)
28 | QUASAR | JANUARY 2020
NON THEMED of issues identified in another study, casting doubt on the integrity of the data collected for that study. The last reference related to a study in which additional infusions were given to patients for clinical reasons, the authors indicated ‘but overall, adherence to the protocol was good’. It was also possible in some cases (not enumerated) to determine deviations from the CONSORT flow chart. It was not possible in the timeframe of this review to establish if CSRs in the public domain reflect a similar level of description of protocol deviations. However, deviations should be reported in the relevant appendices according to ICH E3.
DISCUSSION Research teams often find good practice regulations and guidance bureaucratic and a hindrance to daily activities; however, this commonly arises from interpretation or, more often, over-interpretation of guidance and regulation which then goes on to become ‘standard practice’. The ability of an independent reader, healthcare professional or patient to review the results of research in a way which permits those activities which were planned and complied with to be evaluated in context against those unplanned events and issues which arose as a matter of course during the trial, has some importance in determining the overall quality of the research conducted, and statistical analysis of the results. Ethical and regulatory (legal) requirement compliance were not given much page-space in the majority of articles reviewed, with generalised statements relating to the Declaration of Helsinki, regulatory framework including details for informed consent and sometimes (but not often) a cursory nod to ICH GCP. Encouraging the reporting of non-compliance and quality measures which have a meaning to the data and patients (rather than those things which can simply be timed or counted) requires both a culture change for researchers and those assessing compliance; this is true in commercial and non-commercial/academic settings alike.
'It was not possible in the timeframe of this review to establish if CSRs in the public domain reflect a similar level of description of protocol deviations.' CONCLUSIONS From the sample of articles reviewed from the New England Journal of Medicine and British Medical Journal it was established that although the fundamental principle of consent is almost always declared, not all research makes reference to ICH GCP and deviations are rarely commented upon, though the articles form the basis of clinical decision-making, and this is not a requirement of CONSORT publication recommendations. It should be noted that many non-commercial organisations will not routinely look to the ICH guidelines, but prefer to work to regulations, e.g. the Medicines for Human Use (clinical trial) Regulations 2004, as amended. Articles generally complied with the relevant aspects of CONSORT recommendations, though randomisation was commonly omitted from the article title and not presented in the detail recommended by CONSORT guidance, even when used.
FUTURE WORK In 2020 our group hopes to engage with a number of research teams to develop simplified points of consideration or relevant tools and share best practice in the non commercial/academic setting focusing on areas such as laboratories and/or hosting a forum to understand better the ‘acceptable face of quality’ which will support a shift in behaviours to ensure transparency of compliance and non-compliance, beyond a box-ticking exercise, often cited as a cause of frustration for busy research teams struggling for resources and funding.
REFERENCES 1. EMA, 1996. ICH E3 Structure and Content of Clinical Study Reports. 2. Kenneth F Schulz, D. G. A. D. M., 2010. CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. British Medical Journal, 340(7748), p. 332.
PROFILES Patricia is the Head of Research Governance and Integrity at the London School of Hygiene and Tropical Medicine (LSHTM) where she has responsibility for managing LSHTM’s quality management system, regulatory compliance, is the Designated Individual under the Human Tissue Act, and is the primary contact for research integrity. Patricia is a member of the RQA GCP Committee and the RPG working party. Louise is the Director of Mirabilitas Ltd and Chair of the EFGCP Auditors' Working Party. Louise conducts GCP, GLP and PV audits, mock inspections, and provides training in the same GXP areas for a variety of commercial and non-commercial organisations. She is the RQA RPG Working Party Leader and a member of the Northern Regional Forum Committee. Lucy is a Quality Consultant at Headway Quality Evolution and Head of QA at the University of Aberdeen GLP Test Facility (UoAGLPTF). Headway Quality Evolution is a consultancy firm that delivers quality consultancy and auditing services globally to the pharmaceutical and scientific research industries. UoAGLPTF is a bioanalytical CRO, where Lucy is responsible for the development and management of the GLP and GCP audit programme. In addition to being a member of the RPG, Lucy is a member of the Scotland and English Borders Regional Forum Committee.
Should you wish to join us, or provide comment on this article, please contact us via the RQA office info@therqa.com
JANUARY 2020 | QUASAR | 29