Overview

Calyx Medical Imaging is a leading imaging core lab with the experience and ability to provide a wide range of services involving neuroimaging in clinical trials. Calyx Medical Imaging’s CNS group understands the unique and custom requirements required of Multiple Sclerosis trials and have the flexibility, creativity, bandwidth, and understanding already in place to effectively manage and support these clinical trials.

A significant portion of our CNS experience is in Multiple Sclerosis (MS). For MS studies, we recommend conventional MRI sequences (T1, T1 post gadolinium, FLAIR and T2) for MS lesion evaluation (lesion counts and lesion volume). From the T1 sequence brain volume assessments such as total brain volume/brain volume change, grey matter volume, white matter volume, and CSF (cerebrospinal fluid) volume can be derived. Calyx can support further exploratory measures analyzed on Magnetization Transfer Imaging, Diffusion Tensor Imaging and Double Inversion Recovery. As a basis of a robust independent review assessment, standardized image acquisition is key. Thus, site training and imaging acquisition guidelines are critical for consistent and high-quality MRI images.

Case Study

Calyx Medical Imaging played a vital role in a phase 3, double blind, randomized, multicenter trial in patients with relapsing-remitting multiple sclerosis. Based on the centrally assessed imaging data it was shown that the investigational drug significantly reduces MS lesion activity. The superiority of the investigational drug against interferon treatment was also demonstrated on clinical grounds (relapse rate reduction of 52%). The trial was the largest phase 3 clinical trial program ever submitted to the FDA for a new MS drug at the time.

Example of MS lesion count and volume assessment.
FLAIR image with lesion count and volume of new (green ROI) and enlarging (blue ROI) T2 hyperintense lesions.

FLAIR image with lesion count and volume of new (green ROI) and enlarging (blue ROI) T2 hyperintense lesions.

T1 Gadolinium enhanced sequence with lesion count and volume of T1 hyperintense lesions (yellow ROI).

T1 Gadolinium enhanced sequence with lesion count and volume of T1 hyperintense lesions (yellow ROI).

Experience

Calyx Medical Imaging’s experience is drawn from managing over 2600 trials to date which include more than 4.4 million images from roughly 155,000 sites globally. Within this experience is our management of nearly 180 CNS protocols and over 30 MS studies.

Overview

Calyx Medical Imaging is a leading imaging core lab with the experience and ability to provide a wide range of services involving neuroimaging in clinical trials. Calyx Medical Imaging’s CNS group understands the unique and custom requirements required of Multiple Sclerosis trials and have the flexibility, creativity, bandwidth, and understanding already in place to effectively manage and support these clinical trials.

A significant portion of our CNS experience is in Multiple Sclerosis (MS). Apart from standard MS assessments such as lesion counts (including lesion evolution), lesion and brain volumes Calyx also supports advanced MS analyses:

MAGNETIZATION TRANSFER IMAGING (MTI)

Magnetization Transfer Imaging is a method that has been found to correlate with clinical disability in MS patients. This method is sensitive to the extent of myelination and provides a quantitative assessment of myelin integrity. The MT ratio, a parameter derived from MT imaging data, is a useful imaging biomarker for assessing extent of demyelination and remyelination in MS clinical studies.

DIFFUSION TENSOR IMAGING (DTI)

DTI has been found to provide information about tissue microstructure and architecture including size, shape and organization and as a result represents an effective tool for evaluating white matter tissue integrity. The DTI tool can be used to obtain useful information in many ways – evaluation of Gadolinium-enhancing lesions by DTI provides information about tissue changes in active lesions; analysis of normal appearing white matter (NAWM) using the DTI tool may provide some insight into the early changes in the WM in MS. Parameters such as mean diffusivity (MD), fractional anisotropy (FA) and volume ratio (VR) are some of the most important quantitative indices that are extracted from DTI data.

FUNCTIONAL MAGNETIC RESONANCE IMAGING (FMRI)

fMRI can be used to measure brain activity at rest or while performing a task by detecting the changes associated with blood flow. The underlying understanding is that cerebral blood flow and brain activity are directly correlated. Similarly, functional brain connectivity can be deduced using fMRI data. Activation and/or functional connectivity maps and analysis can provide insight into the brain activity and hence possible abnormalities and response to therapeutics.

DOUBLE INVERSION RECOVERY (DIR)

The Double Inversion Recovery sequence (DIR) has been advocated for detecting gray matter (GM) lesions due to its ability to enhance soft tissue contrast in the brain. It combines two inversion pulses in order to simultaneously suppress signals from tissues with different longitudinal relaxation times. In the brain, DIR allows to selectively image grey matter (GM) by nulling the signal from white matter (WM) and cerebrospinal fluid (CSF) at the time of the excitation pulse.

OPHTHALMIC IMAGING

Within Multiple Sclerosis, it is possible to see structural changes in the retina, specifically within the Ganglion Cell Arteritis (GCA) and Nerve Fiber Layer (NFL). A combination of OCT, FP, and FAF could be appropriate for your trial to evaluate the retinal changes. The combination of Calyx’s operational bench strength and global site/image processing capabilities with
the expertise of Calyx’s medical advisor make for an ideal study management structure.

Ophthalmology imaging endpoints have historically been serviced by academic core labs and though they offer the relevant expertise in the modality and indication, they lack the infrastructure, validated quality systems and SOPs to manage a late phase study. Leveraging expertise across each lab, including Calyx’s systems and export capability will reduce the added workarounds by the sponsor.

Case Study

Calyx Medical Imaging played a vital role in a phase 2, double blind, randomized, multicenter trial in patients with relapsing-remitting multiple sclerosis including MTI analysis. During site standardization Calyx supported the installation of the MTI specific software on the MRI scanners. 110 timepoint MTI datasets of 12 patients have been acquired, collected and processed. Structural MR image processing encompassed image pre-processing, linear spatial normalization, and voxel classification. Lesion-based mean Magnetization Transfer Ratios (MTR) and voxel-based gray matter and white matter MTR histograms were derived for the MTR analysis.

Experience

Calyx Medical Imaging’s experience is drawn from managing over 2600 trials to date which include more than 4.4 million images from roughly 155,000 sites globally. Within this experience is our management of nearly 180 CNS protocols and over 30 MS studies, of which 4 included advanced MRI.

Calyx experience in implementing advanced MRI techniques in Multiple Sclerosis studies:

Copyright 2022 Society of Photo‑Optical Instrumentation Engineers (SPIE). One print or electronic copy may be made for personal use only. Systematic reproduction and distribution, duplication of any material in this publication for a fee or for commercial purposes, and modification of the contents of the publication are prohibited.

Copyright 2022 Society of Photo‑Optical Instrumentation Engineers (SPIE). One print or electronic copy may be made for personal use only. Systematic reproduction and distribution, duplication of any material in this publication for a fee or for commercial purposes, and modification of the contents of the publication are prohibited.

Oliver Bohnsack, MD, PhD, MBA, Manuela Lesch, RT, David Griffiths, MSc, Nicholas Enus, BS

A summary of recommendations from Calyx Medical Imaging

1. THE CHALLENGE OF IMAGING PATIENTS WITH PROSTATE CANCER

Historically, the design and interpretation of clinical trials to assess drug effects in prostate cancer has remained challenging in comparison to other types of solid tumors. In prostate cancer, changes in disease status over time are often difficult to assess because metastases are localized primarily within bones.

The majority of metastases involve the spine, with the lumbar spine affected more frequently than the cervical spine, presumably due to the route of dissemination. Other common metastatic sites in prostate cancer include lymph nodes and visceral organs. As a result, the assessment of prostate cancer with imaging requires the evaluation of bone lesions as well as soft tissue disease. The currently validated and accepted imaging modalities for these disease compartments are radionuclide-based bone scans and Computed Tomography (CT)/Magnetic Resonance Imaging (MRI) scans. Positron Emission Tomography (PET) with 18F -Sodium Fluoride (NaF) or 18F-Fluorodeoxyglucose FDG and other modalities used to image bone are not yet considered validated.

Until 2016, the 2008 publication by Scher, et al. on the “Design and End Points of Clinical trials for Patients with Progressive Prostate Cancer and Castrate Levels of testosterone: Recommendations of the Prostate Cancer Clinical trials working Group 1” defined the standard for the PCWG21: assessment of advanced and metastatic prostate cancer using imaging in clinical trials. Published before the Response Evaluation Criteria in Solid Tumor (RECIST) v1.12 Guidelines, the Prostate Cancer Clinical Trials Working Group 2 (PCWG2) Guidelines reference the original RECIST 1.0 publication from 1999. However, most current prostate cancer clinical trials utilize the clarifications from the updated 2009 RECIST 1.1 guidelines. In 2016, Scher, et al. published the PCWG3 criteria in response to a changing therapeutic landscape and advances in the understanding of prostate cancer biology. Building on the PCWG2 framework, the PCWG3 criteria aim to bridge the gap between radiological profiling and clinical decision-making by encouraging a comprehensive characterization of the disease at baseline as well as post-treatment.

This whitepaper highlights Calyx’s expertise with using and adapting the PCWG3 criteria3 to address imaging-based objectives and endpoints in prostate cancer clinical trials. This discussion is limited to the imaging-related components of bone disease and soft tissue disease. Certain clinical aspects of the corresponding criteria including histology, blood-based assays, biomarker assessments, and patient-reported outcomes require further clarification and are therefore outside the scope of this whitepaper.

2. OVERVIEW OF PCWG3 RECOMMENDATIONS

The overall objective of the PCWG3 criteria is to maximize the benefit to patients with prostate cancer who are being treated with a new therapeutic agent. To achieve this, the criteria aim to establish a series of pre-treatment and post-treatment recommendations to:

a. identify good and poor responders to therapy

b. include Mechanism of Action (MOA) as a key consideration while preparing the dosing schedule

c. suggest clinically relevant intermediate endpoints and patient-reported outcomes-focused toward weighing clinical benefit to the patient against radiologic evidence of progression

d. assess pre- and post-treatment disease pathology

The PCWG3 criteria encourage a more comprehensive characterization of prostate cancer by reporting all lines of therapy. For instance, reporting patient response to prior therapy (as assessed via imaging), radiation and radioisotope therapy is strongly encouraged. In addition, the criteria also suggest that progression via Prostate Specific Antigen (PSA), radiologic imaging, and Symptomatic Skeletal Events (SSEs) be reported To address the use of new agents yet to be clinically characterized, the criteria provide new guidance for choosing the most appropriate imaging intervals. In addition, PCWG3 differentiates between metastatic and non-metastatic prostate cancer states and provides new recommendations for assessing nonmetastatic prostate cancer.

3. EVALUATION OF RESPONSE AND PROGRESSION

RECIST 1.1 is widely used to evaluate response and progression in solid tumor clinical trials. In prostate cancer evaluation, RECIST 1.1 may be used to derive overall responses based on improvements – Complete Response (CR) and Partial Response (PR) – in addition to evaluating Stable Disease (SD) and Progressive Disease (PD). In evaluating bone scans by PCWG3, there are no corresponding response outcomes for CR or PR or even SD. Instead, as in PCWG2, the PCWG3 recommendation for the evaluation of disease on bone scans seeks to rule out progression. PCWG3 recommends approaching bone scan assessment in terms of bone lesions that do not meet the definition of progressive disease (Non-PD) and progressive metastatic bone disease requiring radiologic confirmation (PD).

3.1 Confirmation Criteria, Bone Lesions and ‘Flare’:

The phenomenon of disease flare may be observed when changes occur on a bone scan immediately after starting systemic therapy and is characterized by a transient increase in size or uptake intensity in known lesions and/or the appearance of new, previously occult lesions. These changes are presumably due to active osteoblastic bone deposition (a healing response) as opposed to metastatic progression. An assessment of PD may be made too early if lesions seen on a bone scan are attributable to flare rather than to actual radiological progression. Consequently, there is a concern that the treating oncologist may discontinue the drug prematurely and switch therapy before the drug has had a chance to take effect. For this reason, all subsequent bone scans must be compared to the 1st post-treatment bone scan rather than to the baseline bone scan.

As a general rule, a baseline CT and bone scan are recommended at or close to treatment initiation. The PCWG3 guidance for Castrate Resistant Prostate Cancer (CRPC) trials calls for assessments at an interval of 8- to 9-weeks for the first six months from treatment initiation and an interval of 12-weeks thereafter. For non-metastatic CRPC (nmCRPC) trials, an assessment interval of 16 weeks is recommended.

There is a risk associated with early scanning (<12 or <16 week mark, as applicable) in that pseudo progression on bone scans, as evidenced by the appearance of new lesions, or an increase in the uptake intensity of existing lesions, may be observed. Technetium bone scans rely on the radiopharmaceutical uptake within osteoblasts. Therefore, unlike PET imaging, the technique is
sensitive to osteoblastic activity (flare) rather than to tumor metabolism. Flare due to osteoblastic activity may be induced or sustained by systemic therapies.

To address flare, the PCWG3 criteria designates the 1st post-treatment bone scan as the new baseline when using the 2+2 rule. Per this rule, if ≥ 2 new lesions are seen on the 1st post-treatment bone scan, this may be considered flare. If no additional new lesions are detected on the 2nd posttreatment bone scan, the subject is not considered to have progressive disease. In contrast, if two or more additional lesions are present on the 2nd post-treatment bone scan, however, and the two previously recorded lesions persist, this is considered progression. The date of the scan where the new lesions were first recorded (i.e., 1st post-treatment) is the date of progression.

Clinical study protocols typically require the 1st posttreatment scan to be performed within the flare timeframe (e.g., week 8), followed by the 2nd posttreatment scan at an 8-week interval, within week 16. However, Calyx has noted in a few exceptional cases that the patients had both the 1st and 2nd post-treatment scans within the first 12 weeks.

In accordance with the PCWG3 publication (Scher et al., 2016) CALYX recommends application of the defined lesion counting rules for assessing PD as based solely on the sequence of imaging acquired post-treatment start, rather than dismissing both scans within the flare window for PD confirmation.

Due to the confirmation requirement for bone lesion persistence at a subsequent time point, determining progression status is challenging within a strict timepoint-by-timepoint workflow. Therefore, many sponsors of prostate cancer clinical trials have implemented a confirmation requirement for all bone scan visits, including a persistence rule for those lesions first seen following the 1st post-treatment bone scan. As a result, there are two distinct sets of bone scan lesion counting rules for progression determinations: the 1st post-treatment Progression Rule and the 2nd post-treatment Progression Rule.

1st Post-Treatment Progression Rule (2 + 2 Rule):

At least 2 new lesions are identified on the 1st posttreatment bone scan and at least 2 of these new lesions persist into the subsequent bone scan, which also shows a minimum of 2 additional new lesions (Figure 1).

Schematics developed by:
Maureen Li, Senior Scientist, Scientific and Medical Services, Calyx

Lesion(s) at Baseline

New lesion(s) at 1st Post-treatment

New lesion(s) at 2nd Post-treatment

2nd Post-Treatment Progression Rule:

At least 2 new lesions are identified on the 2nd posttreatment bone scan. For metastatic CRPC studies, progression must be confirmed by the persistence of at least 2 of those lesions at a subsequent bone scan (Figure 2). In this particular example, if the bone lesions identified at the 2nd post-treatment visit are not confirmed at the 3rd post-treatment visit, the assessment will be Non-PD since the requirement for PD has not been met.

Figure 1: Lesions visualized at Baseline (BL), 1st post-treatment and 2nd posttreatment scans. Existing lesions at baseline do not count toward progression. The appearance of two new lesions first seen at 1st posttreatment count toward progression only when they persist into the 2nd post-treatment scan, i.e. when the 2 + 2 rule is fulfilled.
Figure 2: Lesions visualized at Baseline (BL), 1st post-treatment, 2nd posttreatment and 3rd post-treatment scans. Lesions at baseline do not count toward progression; a single new lesion at 1st post-treatment does not count toward progression either. 2nd post-treatment visit is assigned the date of progression since progression (ie, two new lesions at 2nd post-treatment relative to 1st post BL) is confirmed at the subsequent scan per the 2nd posttreatment progression rule.

Note: In the example in Figure 2, PD is not assigned to the 1st post-treatment visit since only one new lesion is identified at this visit. For non-metastatic CRPC studies, any unequivocal new bone lesion which appears subsequent to the 1st post-treatment bone scan need not be confirmed and may immediately result in radiographic progression. Calyx’s recommendation to sponsors is to clarify at study start-up whether one or two unequivocal new bone lesions meet the requirement to confirm progression for non-metastatic CRPC studies.

3.2 SOFT TISSUE DISEASE:

While the PCWG3 criteria refer to RECIST 1.1 for the interpretation of soft tissue disease, specific modifications to RECIST 1.1 are recommended for the derivation of radiologic PFS. For these trials, if soft tissue progression is observed during the flare timeframe, Calyx recommends that clinical trial sites confirm progression via a second scan. As the flare phenomenon in soft tissue disease may be less common and is less well understood, not all sponsors adopt this recommendation. Calyx does not recommend confirmation of soft tissue disease at timepoints beyond flare. Thus, any evidence of soft tissue disease progression beyond the first scheduled post-treatment scan would be considered immediate evidence of progression.

3.3 TIMING OF BONE SCAN IMAGING:

In clinical trials, decisions centered on the frequency of bone scan imaging will need to be weighed between necessity, inconvenience/ potential radiation burden to the patient, and cost to the sponsor. Both patients and investigators must be comfortable with the bone scan imaging schedule and must understand the reasoning behind it. Unlike in its predecessor, PCWG3 discourages the use of CT/MRI images to assess bone disease, as bone lesion assessments must be made using bone scans alone. This emphasizes the importance of properly timed imaging visits for bone scans. Further, it may not be clear to patients and investigators why they should remain on the current treatment until the progression confirmation requirement has been met despite an increase in bone lesion metastases count. Therefore, clearly explaining the treatment plan in the clinical protocol, in particular the timing of image acquisition and educating both investigators and patients, is essential.

The introduction of the No Longer Clinically Benefiting (NLCB) endpoint in PCWG3 encourages patients and investigators to make collective decisions on continuing versus discontinuing therapy. However, as a general rule, the patient should remain on the treatment long enough to allow for a treatment effect.

PCWG3 recommends the first follow-up imaging bone scan assessment to be scheduled 8–9 weeks after treatment initiation for metastatic CRPC trials and 16 weeks after treatment initiation for non-metastatic CRPC trials. Often the first bone scan assessment (i.e., baseline) is not performed on the date of randomization, or dosing. In some trials, baseline imaging may occur as early as 4 weeks prior to the patient’s first dose. This may introduce a source of confusion for the sites, which may perform the first follow-up assessment at 8–9 weeks or 16 weeks, as applicable, after the baseline/ screening scan rather than at 8–9 weeks or 16 weeks, as applicable, post first dose. The impact of this misunderstanding is substantial because imaging acquired following a schedule calculated based on the baseline/screening scan may cause the first follow-up visit to fall short and the patient may not have been on the trial long enough for the treatment to have shown a potentially favorable effect. To avoid confusion, Calyx strongly recommends that clinical trial protocols include clear instructions on the timing of imaging visits.

3.4 ESTABLISHING CONSISTENT LESION COUNTING RULES:

Establishing clear and consistent lesion counting rules to define how the number of bone lesions may drive the assessment of progression is amongst the most critical components of designing a rigorous prostate cancer clinical trial. Lesion counting is dependent on three essential variables:

1. Whether the indication is metastatic CRPC vs. nonmetastatic CRPC.
2. The confirmation requirements for progression.
3. Whether new lesions identified on the 1st posttreatment bone scan are held in suspension until confirmation at a subsequent timepoint.

Even seemingly minor variances in the 1st posttreatment and 2nd post-treatment lesion counting rules can have a significant impact on progression-based endpoints such as PFS and radiologic PFS (rPFS). A critical determinant in planning how the PCWG3 criteria will be applied in a trial is how new lesions observed on the 1st post-treatment bone scan will be counted at follow-up. PCWG3 addresses this issue by assigning the 1st posttreatment scan as the new baseline, i.e. a ‘clean slate’ approach. On the basis of this new baseline, assessments made at subsequent imaging timepoints will use the 1st post-treatment scan as the basis for comparing lesion counts.

3.5 LESION COUNTING AND FLARE:

Consider a scenario where two “hypothetically” new lesions are observed in the 1st post-treatment scan, but are actually the visualization of pre-existing lesions which were not visualized at baseline, as they had not yet triggered their osteoblastic uptake of the radiopharmaceutical agent. Within the initial weeks of flare, the osteoblastic activity becomes significant enough to be detected via the bone scan. In this scenario, while there are visible changes in hotspots noted on the 1st post-treatment bone scan, the understanding is that there is no change in the lesion count. This line of thinking is consistent with the PCWG3 guidelines which specify that changes in the intensity of radiopharmaceutical uptake alone do not constitute progression.

In fact, the interpretation would be that the radiologist now has a better understanding of the disease burden likely to have been present at baseline. As a result, the patient is considered either stable or potentially improving on treatment. Given this, the treating physician would not anticipate that the lesion count would change further at the subsequent timepoint. However, when the 2+ 2 rule is applied, the presence of two additional new lesions on the 2nd post-treatment scan, when noted alongside the two persisting lesions, would lead the radiologist to conclude that progression had already occurred with the 1st post-treatment scan. In this scenario, the conclusion is neither supported by our understanding of the flare phenomenon, nor is it congruent with patient management as it occurs at the site, since in alignment with PCWG3, the site continues treating until there are at least two new lesions relative to the 1st post-treatment scan confirmed on a subsequent scan.

Holding flare lesions in suspension until later confirmation will typically lead to an earlier date of progression without necessitating additional timepoints to confirm progression. A question
to consider is whether this is the appropriate “conservative” approach, or whether it leads to greater discordance between site assessment of progression and clinical trial endpoint determinations. The answer to this key question lies in the interpretation of the nature of these early lesions and thus leads back to the origin of the flare phenomenon.

A drawback is that one does not know the true nature of these initial lesions. They might represent a nonpathological, therapy-induced phenomenon. On the contrary, they may have indeed been new lesions and the first indicators of progression. However, in the spirit of PCWG3, which is aimed to address blind spots in drug development by bridging the gap between clinical practice and radiologic disease assessment across treatment arms, our recommendation is to apply a clean slate approach.

3.5.1 LESION COUNTING WITH THE CLEAN SLATE RULE:

A clean slate approach has been adopted by many sponsors to prevent 1st post-treatment lesions from being considered prematurely “new” (Figure 3).

Figure 3: Lesions visualized at Baseline (BL), 1st post-treatment, and 2nd post-treatment scans. Lesions observed at the 1st posttreatment scan only count toward progression at that scan date when at least two additional new lesions at the subsequent bone scan, are recorded. If not, the lesion counts are ‘reset’ and any new disease is determined by comparison to the 1st post- treatment scan. The 2nd posttreatment scan is non-PD due to the absence of 2 additional new lesions.

Lesion(s) at Baseline

New lesion(s) at 1st Post-treatment

3.6 VARIATIONS IN THE 2ND POST-TREATMENT PROGRESSION RULE — IS LESION PERSISTENCE REQUIRED?

The bone scan imaging schedule may be influenced by the approach used to assess progression. Consider an example where the 1st post-treatment scan is conducted at week 8, and subsequent follow-up scans are conducted every 8 weeks thereafter. If, in one metastatic CRPC clinical trial, two bone lesions at week 16 meet the criteria for progression the first time they are observed, the average time to progression will not necessarily be different from that in a different trial which requires these two bone lesions to persist into the subsequent (i.e., week 24) bone scan. However, when the persistence rule is applied, an additional scan would be required and as a result, switching such a patient over to a different treatment may be delayed. Also, in all confirmation requirements, there is the inherent risk that the additional scan is not acquired and a potential confirmed PD event is lost. For this reason, Calyx recommends including details in the Statistical Analysis Plan on how to address loss due to follow-up and its significance of unconfirmed progression at the last timepoint. Calyx also recommends detailed criteria training for sites. In addition, central confirmation of progression before patients are taken off study can help reduce site-central discordance in PD confirmation and by extension, improve the accuracy of progression-dependent endpoints such as rPFS.

4. RECOMMENDATIONS FOR THE CONSISTENT APPLICATION OF THE PCWG3 CRITERIA

PCWG3 recommendations for disease assessment are summarized (Table 1). Further, PCWG3 suggests modifications to RECIST 1.1 for evaluating soft tissue disease. Of note is the recommendation to categorize and record disease as bone, nodal, lung, liver, adrenal and central nervous system (CNS) (up to 5 lesions per site of disease).

PCWG3 advises investigators to report the date of progression for all anatomical sites (bone, nonnodal soft tissue, nodes) independently. In addition, it also recommends reporting whether favorable changes are occurring across all target and nontarget lesions, or only in some of them. PCWG3 takes into consideration the link between disease site and its prognostic implication. Therefore, it advises to record separately the location of nodal (pelvic vs. extrapelvic) and soft tissue (lung, liver, adrenal, CNS) disease.

Calyx recommends assessing soft tissue disease according to standard RECIST 1.1 guidelines only. This allows sponsors to compare overall soft tissue assessments with the outcomes of other
RECIST1.1-based studies and reduces complexity and ambiguity.

VARIABLE/ MEASURE
RECOMMENDED BY PCWG3
METASTATIC CASTRATE RESISTANT PROSTATE CANCER (MCRPC)
TARGET LESION SELECTION
  • Up to 5 target lesions
SOFT TISSUE
  • Imaging by CT and/or MRI
  • Include chest CT1
  • Include prostate MRI if necessary2
  • Imaging frequency: every 8–9 weeks for the first 6 months, then every 12 weeks.
  • Assess soft tissue (lung, liver, adrenal, CNS) lesions per RECIST 1.1.
LYMPH NODES
  • Nodes ≥1.5cm in short axis are pathologic and measurable
  • Assess nodes ≥1.0 but <1.5cm per RECIST 1.1
  • Record pelvic disease as locoregional v/s extrapelvic. Record extrapelvic disease as retroperitoneal, mediastinal, thoracic, or other.
BONE
  • Imaging by CT and/or MRI
  • Include chest CT1
  • Include prostate MRI if necessary2
  • Imaging frequency: every 8–9 weeks for the first 6 months, then every 12 weeks.
  • Assess soft tissue (lung, liver, adrenal, CNS) lesions per RECIST 1.1.
NON-METASTATIC CASTRATE RESISTANT PROSTATE CANCER (NMCRPC)
LYMPH NODES
  • Perform lesion assessments per RECIST 1.1
BONE
  • Report any new unequivocal lesion seen after the 1st post-treatment scan as non-metastatic to metastatic progression
  • For lesions recorded on the 1st post-treatment scan, two additional new lesions must appear at a subsequent scan to meet the requirement for metastatic progression.
  • Imaging frequency: every 16 weeks for first 2 years, then every 24 weeks.

Table 1: PCWG3: Summary of Imaging-related recommendations

[1] Suggested to adequately capture lung metastatic events

[2] Scans acquired on a case-by-case basis for patients with locally persistent or recurring disease

table1_pcwg3_summary_of_imaging_related_recommendations

Table 1: PCWG3: Summary of Imaging-related recommendations

[1] Suggested to adequately capture lung metastatic events

[2] Scans acquired on a case-by-case basis for patients with locally persistent or recurring disease

Recognizing that a major unmet need in the field of advanced prostate cancer research is a reliable surrogate intermediate endpoint to help guide drug development, a recent study asked whether the RECIST response can predict success in phase III mCRPC trials. The study found that a lower response rate in phase II was associated with a greater likelihood of phase III failure in mCRPC patients being treated with cytotoxic or hormonal therapies. However, for alternative agents with other mechanisms of action, such as immunotherapies or cytostatic agents, time-to-event endpoints such as rPFS, metastasis-free survival, or time to symptomatic progression remain more appropriate (Brown et al., 2018).

5. IMAGE QUALITY CONCERNS

In a clinical trial setting, both computerized Tomography (CT) scans and bone scans must be evaluated to determine if radiologic progression has occurred. PCWG3 relies on bone lesion assessments as well as non-nodal soft tissue and nodal disease assessments for determination of progression. CT is utilized to gather any evidence of response to treatment.

It has now become standard practice for sites to acquire CT scans with contrast using a slice interval of 5mm or less and save them in Digital Imaging and Communications in Medicine (DICOM) format. This standard is ideal for central collection and evaluation of CT scans.

5.1 INCONSISTENCIES IN IMAGE ACQUISITION:

When it comes to bone scans, clinical sites do not routinely apply standardized methods for the acquisition or archiving4 of images. The quality of bone scans is at times limited by inadequate scan delay, insufficient scanning time, urine contamination and missing anatomy. Furthermore, bone scans may be saved in various file formats other than DICOM, which degrade the quality of the bone scans and their associated interpretation. A particular image quality concern seen by Calyx is that sites save the bone scan image as a screenshot, which cannot be adjusted, or the windowing be leveled, as one would with a DICOM image to achieve the best possible display intensity for evaluating the presence of bone lesions, particularly if the patient presents with multiple lesions. Sometimes, screenshots are converted back into the DICOM format; however, the converted images are often of insufficient quality to conduct a robust evaluation. To avoid inconsistent bone scan evaluations and “Not Evaluable” assessments, it is important that sites involved in a clinical trial follow standardized image acquisition guidelines for bone scans. Calyx’s image acquisition guidelines, including recommendations for improving scan quality, are summarized (Table 2).

COMMON IMAGE QUALITY ISSUES
RECOMMENDATIONS FOR IMPROVING SCAN QUALITY
IMAGE ACQUISITION
  • Inadequate scan delay (waiting time after injection)
  • Insufficient scanning time/speed (collecting insufficient counts)
  • Poor image resolution
  • Variable tracer dose
  • 3h ± 1h after injection, consistent per patient
  • Normally 10–14cm/min
  • 500k counts or greater
  • Low energy high-resolution collimator
  • 20–30 mCi (740–1110 MBq)
PATIENT POSITION
  • Patient’s head turned to the side/ inconsistent head position baseline to follow-up
  • Missing anatomy/ inadequate spot views
  • No side markers
  • Urine contamination in the pelvis (may be mistaken for osseous metastatic disease)
  • Detector head not in close proximity to the patient
  • Large lead covers (to cover bladder) can obstruct anatomy
  • Use a pillow to support the head and lower back when necessary
  • Detailed instructions on required views and labeling
  • Have patient void immediately before imaging start
  • Use foot band to keep feet close together
  • If the patient has any type of monitor attached, unplug it or move it out of Field of View (FOV), if possible
IMAGE FORMAT
  • Screenshots
  • .jpg files
  • Transfer files in DICOM format
  • Ensure that site personnel are well trained and understands how to transfer the images correctly

Table 2: Recommendations for improving image quality

table2_recommendations_for_improving_image_quality

Table 2: Recommendations for improving image quality

5.2 SUPERSCANS:

A superscan is a bone scan which exhibits diffuse skeletal uptake of the radiopharmaceutical agent with little to no uptake within soft tissue including the kidneys and bladder. The superscan effect presents a technical challenge in the evaluation of lesions on bone scans in prostate cancer clinical trials. This type of scan results in the characteristically high boneto-soft tissue ratio in image intensity. In addition, discrete lesions may not be visible. Patients with extremely diffuse metastatic disease often exhibit a superscan, however, certain patients with extensive disease have scans that may tend toward, but may not meet all the hallmarks of a true superscan. Making an evaluation utilizing the lesion assessment guidelines outlined in PCWG3 in patients with extensive and diffuse disease and with uncountable lesions or in patients with superscans is rather difficult.

The absence of discrete lesions on bone scans due to the superscan effect may be misleading for investigators while determining patient eligibility during enrollment. Further, the accurate and wellstandardized determination of disease status is challenging in cases where there is diffuse disease at baseline and a superscan at follow-up, yet no discrete lesions to count is equally challenging. Also, patients with superscans are often not evaluable at follow-up and as such, impact the PFS analysis in the intent-totreat population. Calyx has successfully implemented rules particularly for the evaluation of these patients where one must differentiate whether such superscan is already present at Baseline or develops during the treatment.

Figure 4: Anterior (left) and posterior (right) projections of a superscan showing intense systemic activity within bones and diminished renal and soft tissue activity on a Tc99m diphosphonate bone scan.

6. CALYX RECOMMENDATIONS FOR CENTRAL REVIEW OF PROSTATE CANCER DATA IN CLINICAL TRIALS

6.1 CENTRAL REVIEW WORKFLOW:

Central evaluation of bone scans and correlative CT/MRI scans for prostate cancer clinical trials is routinely being implemented by many sponsors. Often the central review workflow is an important point of discussion relating to scientific rigor in reporting imaging measures. There are several approaches; therefore, one of the first decisions to make is whether the soft tissue assessments and bone lesion assessments should be performed by the same reader(s) using a composite read model or by a different reader(s) using a decoupled read model. Calyx has capabilities in both read models and makes study-specific recommendations on which read model will be best suited for a particular clinical trial.

6.1.1 DECOUPLED READ MODEL:

With a decoupled read model, central radiologists with expertise in prostate cancer evaluate CT/MRI scans for soft tissue disease according to standard RECIST 1.1, and central nuclear medicine physicians evaluate bone scans separately for bone lesions according to PCWG3. Calyx’s experience has shown that the interpretation of the bone scans is the more challenging aspect of evaluating disease status. Leveraging the expertise of a nuclear medicine physician will improve the quality of the interpretations. The decoupled read model is typically more expensive, with two sets of readers and potentially more complex imaging case report forms utilized to record independent reviewers’ assessments.

This read model also keeps the radiologist blinded to bone lesion progression, thereby reducing bias. Additionally, for some trials, the decoupled read model allows for a better understanding of the effectiveness of a treatment in the different disease compartments. This may preserve endpoint integrity, particularly in cases where there are mixed reactions, by discerning progression events assessed on CT/MRI from those assessed on bone scan. Occasionally, patients may present with a purely lytic bone lesion that will not be apparent on the technetium bone scan. Such lesions are rare and may not be documented in this decoupled reading model.

6.1.2 COMPOSITE READ MODEL:

With a composite read model, dual-certified central readers evaluate both CT/MRI scans and bone scans. CT/MRI scans and bone scans are evaluated for progression or response within the corresponding disease compartments according to RECIST 1.1 and PCWG3 respectively. Generally, the readers performing reads according to a composite read model are radiologists dual board certified in radiology and nuclear medicine. While the composite read model has time and cost benefits, because the same reader can assess both CT/MRI and bone scan in one review session, the identification of readers with sufficient experience reading both in prostate cancer is one potential drawback.

7. CENTRAL INDEPENDENT REVIEW: VALUE OF CT OR MRI SCAN IN BONE SCAN EVALUATION

When a study adopts the decoupled read model, an important discussion point in bone scan by the nuclear medicine readers is the value imparted by having access to correlative regional anatomical scans for all corresponding timepoints. While there is value in establishing bone scans as a standalone surrogate endpoint for the evaluation of bone metastases in prostate cancer, the benefit of having access to the correlating CT or MRI scans can prove critical. For instance, those scans often provide critical diagnostic information on the nature of bone scan findings (e.g., fractures and degenerative changes), and can validate the presence of true metastases.

8. CALYX’S KEY SCIENTIFIC AND OPERATIONAL RECOMMENDATIONS

There are many challenging aspects of assessing prostate cancer in clinical trials, ranging from protocol design, imaging schedules, and image acquisition to the implementation of assessment criteria, lesion counting rules, and read models. Calyx recommends the following design steps for a prostate cancer clinical trial:

  • Clearly define the imaging schedule with its permittable visit window in the clinical protocol.
  • Educate sites on best practices for scan acquisition, transfer format, and archiving, particularly for bone scans.
  • Consider the impact of superscans when developing protocol inclusion and exclusion criteria. If subjects are allowed on study with superscans, or if superscans are to be expected on-study, educate sites on what a superscan is and the downstream impact this may have on the endpoint (i.e. patient may be Non-Evaluable (NE) at follow-up).
  • Use standard RECIST 1.1 for the evaluation of soft tissue disease.
  • Provide sites with clear guidelines on bone scan lesion counting rules, 2 + 2 and 2-persisting.
  • Provide guidance on reporting the date of progression (supported with examples).
  • Additional recommendations for implementing central review:
  • Provide correlative CT/MRI scan to the bone scan reviewer.
  • Provide clear rules on reporting baseline disease and marking new disease at follow-up.
  • Establish a threshold for determining whether a lesion is metastatic, traumatic, or degenerative.
  • Clearly define the process of determining symptomatic skeletal-related events in the clinical protocol as these are typically determined by the site and not included in central review.

8.1 CALYX’S RECOMMENDATIONS FOR ASSESSMENT OF PROSTATE CANCER DATA IN CLINICAL TRIALS:

8.1.1 REPORTING DISEASE:
  • Baseline disease: Calyx recommends that baseline bone lesions be counted and the total number be recorded. Individual lesions need not be marked.
  • New lesions: Calyx recommends that only up to five new lesions (as compared to the prior timepoint) be marked on the associated bone scan and the total number of new lesions be recorded.
  • Bone lesion burden: Define whether the timepoint of nadir (i.e. timepoint with the least number of bone lesions) should be recorded.
  • Missed’ lesions: If a lesion is missed during the initial lesion counting process, Calyx recommends that the event be documented and addressed when making the timepoint disease assessments.
  • Equivocal lesions: If a new lesion was not obvious at a previous follow-up timepoint, determine how to report the lesion when it is considered unequivocal.
  • Image quality: How image quality may impact the ability to identify and report lesions at baseline and how this may limit assessment options at follow-up timepoints complete a separate CRF.
8.1.2 DEFINING CLEAR PROGRESSION RULES:
  • Define how new lesions appearing at the 1st post-treatment timepoint versus those appearing at subsequent timepoints can drive progression.
  • Prevent early or incorrect progression assessments due to differences in the application of PCWG3 criteria.
8.1.3 REQUIREMENTS ON DATE OF PROGRESSION:
  • Define how the date of progression is reported.
  • Typically, the date of progression is the date two new lesions were first seen together, not the confirmatory timepoint.
8.1.4 SKELETAL RELATED EVENTS ENDPOINT CONTRIBUTIONS:
  • Clearly define the determination of skeletal-related events in the protocol. Skeletal related events can present in many ways:
  • Changes in response to antineoplastic therapy and on-study radiation to treat bone pain.
  • Symptomatic fractures.
  • Spinal cord compression.
  • Surgery to bone, which also includes minimal invasive interventions (i.e. vertebroplasty).

These events are most often documented by site investigator teams even when central review is part of the trial design.

8.1.5 TRAIN THE CLINICAL SITES:

Use examples and training cases to demonstrate lesion counting rules and teach site personnel how to apply these rules.

8.2 LESION COUNTING LOGIC FOR CONSISTENT APPLICATION OF THE PCWG3 CRITERIA:

table3_response_assessment_criteria_per_timepoint
Table 3: Response Assessment Criteria per Timepoint

New lesion(s) at 1st Post-treatment

New lesion(s) at 2nd Post-treatment

New lesion(s) at 3rd Post-treatment

Lesions at baseline do not count toward progression

* Note: The number of lesions at BL is only relevant for a PD at 1ST post-treatment

REFERENCE

1. PCWG2: Scher H, Halabi S, Tannock I, et al. “Design and End Points of Clinical Trials for Patients with Progressive Prostate Cancer and Castrate Levels of Testosterone: Recommendations of the Prostate Cancer Clinical Trials Working Group.” J Clin Oncol 26 (2008): 1148-1159

2. RECIST 1.1: Eisenhauer, EAP, Therasse, J, Bogaerts, LH, et al. “New Response Evaluation Criteria in Solid Tumors: Revised RECIST Guideline (Versions 1.1)“ European Journal of Cancer 45 (2009): 228-247

3. PCWG3: Scher H, Morris M, Stadlet W, et al. “Trial Design and Objectives for Castration-Resistant Prostate Cancer: Updated Recommendations From the Prostate Cancer ClinicalTrials Working Group 3” J Clin Oncol 34 (2016): 1148-1159

4. Langdon C. Brown, Guru Sonpavde, Andrew J. Armstrong : “Can RECIST Response Predict Success”

L. Sargsyan-Storim, O. Bohnsack, M. Lesch

Learning Objectives

Demonstration of assessment discrepancies between Local Evaluation (LE) and Blinded Independent Central Review (BICR) in clinical trials with required radiological eligibility and disease progression confirmation based on substantial advantages of BICR in comparison with LE:

  • Reduced imaging assessment error rate
  • Minimized bias in assessments
  • Standardized data evaluation.

Background

The use of medical imaging in clinical trials is well established. Regulatory authorities such as the Food and Drug Administration (FDA) cast doubt on validity and reliability of safety and efficacy determinations based only on assessments of sites’ radiologists or also non-radiologists such as investigators. As a consequence, independent imaging contract research organizations (iCROs) are involved in the imaging standardization and review process, performing Blinded Independent Central Review.

As per FDA statement from June 2004: “… offsite image evaluations are image evaluations performed at sites that have not otherwise been involved in the conduct of the study and by readers who have not had contact with patients, investigators, or other individuals involved in the study. We recommend that Phase 3 trials include offsite image evaluations that are performed at a limited number of sites (or preferably at a centralized site). In such offsite evaluations, it is usually easier to control factors that can compromise the integrity of the blinded image evaluations and to ensure that the blinded readers perform their image evaluations independently of other image evaluations”1.

Nowadays, BICR is widely used for clinical trials with imaging related endpoints and in trials with required radiological eligibility confirmation. In many clinical studies presence of any indication specific radiological finding is used as a patient inclusion criterion in the trial. A blinded independent confirmation or rejection of eligibility determination’s correctness prevents possible inclusion errors and increases the reliability of collected data during the study.

Additionally, BICR is often used in oncological trials with progression confirmation. Since in most trials disease progression leads to the subject’s treatment discontinuation, a verification of a radiological progression event by an independent review committee is of great significance. Many studies document higher response rates and earlier progression determination by LE compared to BICR.2

In this paper we demonstrate data about eligibility and progression determination discrepancies between sites and central radiologists, the reasons for the differences between BICR and LE assessments and the advantages of BICR.

Findings

Data of 1020 subjects from seven Phase II and III clinical trials with required eligibility or disease progression confirmation is used for this analysis. The clinical indications of the analyzed studies were breast cancer, non-small cell lung cancer (NSCLC), colorectal cancer and mantel cell lymphoma. The imaging assessments were performed according to RECIST3, RECIST 1.14, IWG NHL 19995 and IWG NHL 20076 tumor evaluation criteria.

Eligibility reads with verification of at least one measurable lesion on CT/MRI according to used imaging criteria assessed by LE were required in three analyzed trials. The numbers of subjects with not confirmed eligibility due to lack of measurable lesions by independent review committee are:

1. 20 from 142 subjects (Breast Cancer I) – 14.0%

2. 18 from 264 subjects (Breast Cancer II) – 6.8%

3. 8 from 445 subjects (Colorectal Cancer) – 1.8%

Eligibility Verification by BICR
Fig.1: Determination of subject radiological eligibility by investigational sites and independent review committee

Summarized, 46 subjects from 851 pronounced as eligible by LE (5.4%) did not meet the imaging inclusion criteria and their erroneous inclusion in the studies may impact the final statistical analysis of the clinical trial.

Eligibility Determination by BICR
Fig.2: Subject radiological eligibility assessed by independent review committee

Radiological disease progression confirmation reads were required in four analyzed studies. The progression was not confirmed during the blinded independent review for the following number of radiological progression events assessed by clinical sites:

1. 12 from 79 events (NSCLC I) – 15.2%

2. 19 from 47 events (NSCLC II) – 40.4%

3. 4 from 15 events (Breast Cancer III) – 26.7%

4. 11 from 28 events (Mantel Cell Lymphoma) – 39.3%

Progression Confirmation LE versus BICR
Fig.3: Comparison of progression events numbers assessed by investigational sites and independent review committee

Summarized, 46 radiological progression events from 169 assessed by LE were not confirmed by independent review committee (27.2%).

Progression Confirmation by BICR
Fig.4: Numbers of progression events confirmed by central independent reviewers

Additionally, for both NSCLC trials we were able to perform a reverse analysis of the progression events assessments, used for the previous analysis. The disease progression events number assessed by central reviewers was compared with the number of progression events determined by the site radiologists:

1. 30 from 97 events assessed by BICR are not confirmed by LE (NSCLC I) – 30.9%

2. 19 from 47 events assessed by BICR are not confirmed by LE (NSCLC II) – 40.4%

Progression Assessments Discrepancy BICR versus LE
Fig.5: Comparison of progression events numbers assessed by central independent reviewers vs. investigational sites

Summarized, 49 radiological progression events from 144 assessed by independent radiologists (34.0%) were not confirmed by site radiologists.

Progression Confirmation by BICR
Fig.4: Numbers of progression events confirmed by central independent reviewers

The origin of the discrepancies between assessments made by local site radiologists/ investigators and central independent reviewers can be traced back to the following 10 significant differences between LE and BICR:

1. Clinical status impact: the patient clinical condition is irrelevant for central independent readers as opposed to site radiologists, since blinded independent central review excludes the potential knowledge of subject’s clinical status and treatment assignment.

2. Interest confrontation: financial or scientific interests of site physicians together with hospital administrative constraints may affect the patient inclusion in the study and the following assessments. Due to the BICR design any interests in the patients imaging assessments are foreclosed.

3. Influence of social indications: any personal relationships with the patients or with their relatives can affect physician’s behavior with a following impact on the patient assessments, which is completely excluded by a blinded review.

4. Image reading consistency and standardization: during the independent review all images of a patient at all timepoints are assessed in one reading session by one expert. The “same subject – same radiologist” concept that significantly reduces assessment variability is hardly achievable in clinical sites.

5. Radiologist’s experience: the opportunity of independent reviewer selection from an experienced reviewer pool beneficially distinguishes BICR from LE, when the assessments can be made by any radiologist on duty.

6. Specified Training: detailed and rigorous criteria trainings aligned with study protocol as well as testing and monitoring of central radiology reviewers is required for BICR. The trainings of the site radiologists are often sketchy and universal for different indications.

7. Data Quality: ongoing quality control from image receipt through reviewer assessment to export data is required for BICR in contrast to LE. Furthermore, the common BICR “doublereadwith adjudication” design for submission studies effectively decreases the number of inevitable errors and inaccuracies in final assessments.

8. Equipment variability: while the equipment for LE varies between different sites/countries, central imaging labs use standardized workstations linked with analysis application software for image evaluation, which significantly increases precision of assessments in comparison to LE.

9. Work efficiency: a sophisticated analysis application with criteria-specific edit checks is used for BICR which distinctly increases the independent reader’s work efficiency and accuracy compared to local radiologists.

10. Time: the independent reviewers are able to set their review time free and don’t have to perform the reviews as a part of daily clinical routine in addition to their common tasks like local radiologists.

Conclusion

Discrepancies in radiological eligibility determination and radiological disease progression assessment have been demonstrated between local evaluation and blinded independent central review. There are several important factors leading to such differences between independent review design and local image evaluation in the investigative sites. Using BICR significantly reduces potential bias in imaging assessments compared to LE and provides more standardized radiological data of proven higher quality for the statistical analysis in clinical studies.

References

1. U.S. Department of Health and Human Services Food and Drug Administration Center for Drug Evaluation and Research (CDER) Center for Biologics Evaluation and Research (CBER) June 2004 Clinical Medical, Guidance for Industry, Developing Medical Imaging Drug and Biological Products Part 3: Design, Analysis, and Interpretation of Clinical Studies.

2. R. Ford, L. Schwartz, J. Dancey, L.E. Dodd, et al, “Lessons learned from independent central review”, Eur J Cancer. 2009; 45(2):268-74.

3. P. Therasse, S.G. Arbuck, E.A. Eisenhauer, et al, „New guidelines to evaluate the response to treatment in solid tumors. European Organization for Research and Treatment of Cancer, National Cancer Institute of the United States, National Cancer Institute of Canada”, J Natl Cancer Inst. 2000; 92(3):205-16.

4. E.A. Eisenhauer, P. Therasse, J. Bogaerts, et al, “New Response Evaluation Criteria in Solid Tumours: Revised RECIST Guideline (Version 1.1),” Eur J Cancer. 2009; 45(2):228-47

5. B.D. Cheson, S.J. Horning, B. Coiffier, et al, “Report of an International Workshop to standardized response criteria for non-Hodgkin’s lymphomas”,J Clin Oncol. 1999; 17:1244-1253.

6. B.D. Cheson, B. Pfistner, M.E. Juweid, et al, (International Harmonization Project on Lymphoma), “Revised response criteria for malignant lymphoma “, J Clin Oncol. 2007; 25(5):579-86.

Stay Updated

Never miss a beat. Sign up to receive emails covering industry news and useful content to help you advance clinical development.

  • This field is for validation purposes and should be left unchanged.