L. Sargsyan-Storim, O. Bohnsack, M. Lesch
Learning Objectives
Demonstration of assessment discrepancies between Local Evaluation (LE) and Blinded Independent Central Review (BICR) in clinical trials with required radiological eligibility and disease progression confirmation based on substantial advantages of BICR in comparison with LE:
- Reduced imaging assessment error rate
- Minimized bias in assessments
- Standardized data evaluation.
Background
The use of medical imaging in clinical trials is well established. Regulatory authorities such as the Food and Drug Administration (FDA) cast doubt on validity and reliability of safety and efficacy determinations based only on assessments of sites’ radiologists or also non-radiologists such as investigators. As a consequence, independent imaging contract research organizations (iCROs) are involved in the imaging standardization and review process, performing Blinded Independent Central Review.
As per FDA statement from June 2004: “… offsite image evaluations are image evaluations performed at sites that have not otherwise been involved in the conduct of the study and by readers who have not had contact with patients, investigators, or other individuals involved in the study. We recommend that Phase 3 trials include offsite image evaluations that are performed at a limited number of sites (or preferably at a centralized site). In such offsite evaluations, it is usually easier to control factors that can compromise the integrity of the blinded image evaluations and to ensure that the blinded readers perform their image evaluations independently of other image evaluations”1.
Nowadays, BICR is widely used for clinical trials with imaging related endpoints and in trials with required radiological eligibility confirmation. In many clinical studies presence of any indication specific radiological finding is used as a patient inclusion criterion in the trial. A blinded independent confirmation or rejection of eligibility determination’s correctness prevents possible inclusion errors and increases the reliability of collected data during the study.
Additionally, BICR is often used in oncological trials with progression confirmation. Since in most trials disease progression leads to the subject’s treatment discontinuation, a verification of a radiological progression event by an independent review committee is of great significance. Many studies document higher response rates and earlier progression determination by LE compared to BICR.2
In this paper we demonstrate data about eligibility and progression determination discrepancies between sites and central radiologists, the reasons for the differences between BICR and LE assessments and the advantages of BICR.
Findings
Data of 1020 subjects from seven Phase II and III clinical trials with required eligibility or disease progression confirmation is used for this analysis. The clinical indications of the analyzed studies were breast cancer, non-small cell lung cancer (NSCLC), colorectal cancer and mantel cell lymphoma. The imaging assessments were performed according to RECIST3, RECIST 1.14, IWG NHL 19995 and IWG NHL 20076 tumor evaluation criteria.
Eligibility reads with verification of at least one measurable lesion on CT/MRI according to used imaging criteria assessed by LE were required in three analyzed trials. The numbers of subjects with not confirmed eligibility due to lack of measurable lesions by independent review committee are:
1. 20 from 142 subjects (Breast Cancer I) – 14.0%
2. 18 from 264 subjects (Breast Cancer II) – 6.8%
3. 8 from 445 subjects (Colorectal Cancer) – 1.8%
Summarized, 46 subjects from 851 pronounced as eligible by LE (5.4%) did not meet the imaging inclusion criteria and their erroneous inclusion in the studies may impact the final statistical analysis of the clinical trial.
Radiological disease progression confirmation reads were required in four analyzed studies. The progression was not confirmed during the blinded independent review for the following number of radiological progression events assessed by clinical sites:
1. 12 from 79 events (NSCLC I) – 15.2%
2. 19 from 47 events (NSCLC II) – 40.4%
3. 4 from 15 events (Breast Cancer III) – 26.7%
4. 11 from 28 events (Mantel Cell Lymphoma) – 39.3%
Summarized, 46 radiological progression events from 169 assessed by LE were not confirmed by independent review committee (27.2%).
Additionally, for both NSCLC trials we were able to perform a reverse analysis of the progression events assessments, used for the previous analysis. The disease progression events number assessed by central reviewers was compared with the number of progression events determined by the site radiologists:
1. 30 from 97 events assessed by BICR are not confirmed by LE (NSCLC I) – 30.9%
2. 19 from 47 events assessed by BICR are not confirmed by LE (NSCLC II) – 40.4%
Summarized, 49 radiological progression events from 144 assessed by independent radiologists (34.0%) were not confirmed by site radiologists.
The origin of the discrepancies between assessments made by local site radiologists/ investigators and central independent reviewers can be traced back to the following 10 significant differences between LE and BICR:
1. Clinical status impact: the patient clinical condition is irrelevant for central independent readers as opposed to site radiologists, since blinded independent central review excludes the potential knowledge of subject’s clinical status and treatment assignment.
2. Interest confrontation: financial or scientific interests of site physicians together with hospital administrative constraints may affect the patient inclusion in the study and the following assessments. Due to the BICR design any interests in the patients imaging assessments are foreclosed.
3. Influence of social indications: any personal relationships with the patients or with their relatives can affect physician’s behavior with a following impact on the patient assessments, which is completely excluded by a blinded review.
4. Image reading consistency and standardization: during the independent review all images of a patient at all timepoints are assessed in one reading session by one expert. The “same subject – same radiologist” concept that significantly reduces assessment variability is hardly achievable in clinical sites.
5. Radiologist’s experience: the opportunity of independent reviewer selection from an experienced reviewer pool beneficially distinguishes BICR from LE, when the assessments can be made by any radiologist on duty.
6. Specified Training: detailed and rigorous criteria trainings aligned with study protocol as well as testing and monitoring of central radiology reviewers is required for BICR. The trainings of the site radiologists are often sketchy and universal for different indications.
7. Data Quality: ongoing quality control from image receipt through reviewer assessment to export data is required for BICR in contrast to LE. Furthermore, the common BICR “doublereadwith adjudication” design for submission studies effectively decreases the number of inevitable errors and inaccuracies in final assessments.
8. Equipment variability: while the equipment for LE varies between different sites/countries, central imaging labs use standardized workstations linked with analysis application software for image evaluation, which significantly increases precision of assessments in comparison to LE.
9. Work efficiency: a sophisticated analysis application with criteria-specific edit checks is used for BICR which distinctly increases the independent reader’s work efficiency and accuracy compared to local radiologists.
10. Time: the independent reviewers are able to set their review time free and don’t have to perform the reviews as a part of daily clinical routine in addition to their common tasks like local radiologists.
Conclusion
Discrepancies in radiological eligibility determination and radiological disease progression assessment have been demonstrated between local evaluation and blinded independent central review. There are several important factors leading to such differences between independent review design and local image evaluation in the investigative sites. Using BICR significantly reduces potential bias in imaging assessments compared to LE and provides more standardized radiological data of proven higher quality for the statistical analysis in clinical studies.
UPDATED: JAN. 15 2024
References
1. U.S. Department of Health and Human Services Food and Drug Administration Center for Drug Evaluation and Research (CDER) Center for Biologics Evaluation and Research (CBER) June 2004 Clinical Medical, Guidance for Industry, Developing Medical Imaging Drug and Biological Products Part 3: Design, Analysis, and Interpretation of Clinical Studies.
2. R. Ford, L. Schwartz, J. Dancey, L.E. Dodd, et al, “Lessons learned from independent central review”, Eur J Cancer. 2009; 45(2):268-74.
3. P. Therasse, S.G. Arbuck, E.A. Eisenhauer, et al, „New guidelines to evaluate the response to treatment in solid tumors. European Organization for Research and Treatment of Cancer, National Cancer Institute of the United States, National Cancer Institute of Canada”, J Natl Cancer Inst. 2000; 92(3):205-16.
4. E.A. Eisenhauer, P. Therasse, J. Bogaerts, et al, “New Response Evaluation Criteria in Solid Tumours: Revised RECIST Guideline (Version 1.1),” Eur J Cancer. 2009; 45(2):228-47
5. B.D. Cheson, S.J. Horning, B. Coiffier, et al, “Report of an International Workshop to standardized response criteria for non-Hodgkin’s lymphomas”,J Clin Oncol. 1999; 17:1244-1253.
6. B.D. Cheson, B. Pfistner, M.E. Juweid, et al, (International Harmonization Project on Lymphoma), “Revised response criteria for malignant lymphoma “, J Clin Oncol. 2007; 25(5):579-86.
