This is the second of two articles on the consequences that could arise if an interactive response technology (IRT) system isn’t designed and/or implemented correctly and how a trial could quickly go off track based on risks related to randomization, drug allocation, and trial supply.
In a recent webinar, Calyx’s Peter Tarbox shared the unique RTSM factors oncology trial sponsors should consider in their IRT design, focusing on how advanced IRT systems can accommodate changes in centrally and locally sourced medication for optimal clinical trial supply management.
Here he addresses some of the questions that arose during the webinar and explains how Calyx IRT addresses these unique RTSM challenges to keep these important trials on track.
Is the sourcing strategy only by country or can you set it by site level?
A common setup seen is that sourcing is selected at the country level as the suitability for local sourcing is usually consistent across sites within the same country.
However, if there is a specific need to vary the sourcing between different sites within the same country then the switch can be defined down to the site level.
If a site/country is locally sourcing all treatment components, would you advise that the medication assignment visits still be recorded within the IRT?
Typically, we advise caution when registering scheduled visits during the treatment phase within IRT that do not result in the allocation of medication through the IRT. The reason is that issues can arise with site compliance on registering the visit in IRT in a timely fashion.
However, if there is a need for the visit data to be stored within the IRT (for example, if you want to see these visits listed on a report within IRT) then these can be included but we would recommend that sites are made aware of the importance of these being registered accurately to avoid reconciliation issues at the end of the study.
Will the system be able to confirm the volume of the solution withdrawn to dilute in an infusion bag? Also, what about Cisplatin dose, could a calculator be uploaded in the system?
Calculations to support dosing can be included within the IRT and these can be included on the final readouts of the transaction, email notifications, and patient-level reports as required.
There is a decision to be made as to whether the IRT is the best place to perform these calculations – we have seen a mixture of approaches depending on whether the sponsor wants this handled within the IRT or left up to the sites themselves to calculate.
Generally, where a calculation is performed for a dosage, this is included within the programming of the IRT itself, however, we have seen occasions where if there is third-party software to perform the calculation then it may be possible for the IRT to integrate information for the calculation to be performed outside of the IRT with the result returned for storage in the IRT.
How would you handle dose reductions or dose increases for patients within the IRT?
Dose level can be tracked within the IRT – there are various rules that can be managed by the IRT (e.g. the IRT can ensure that a permitted dose adjustment is made in a stepwise fashion where the protocol dictates, or that there are a maximum number of times a patient can be dose reduced, etc.).
We always want to understand how much of a role the sponsor expects the IRT to have as a gatekeeper of the rules within the protocol, in comparison with their being less management of the rules within the IRT and the responsibility on the sites supported by adequate training, etc. to only select suitable dose level adjustments that comply with protocol procedures. But if the IRT is best placed to manage the titration rules, then protocol rules can be incorporated. Calyx’s intent is to identify the medical flexibility needed in the rules for a safety situation when setting up the system. This is so that the system is not made too rigid for unexpected safety cases or individual investigator decisions.
Outside of the mechanism for dose reduction and escalation itself, we would also recommend that the impact of potential dose escalation/reduction is considered to ensure optimal setup for buffer stock and prediction. For example, we have the option to predict enough medication to cover all options available at a visit, or we can rely on buffer stocks for the selection of the least common option.
Regarding expensive ECPs, do you have any experience on how to manage extra central supply at the depot level for a potential switch from local to central sourcing? We have a country that wants to locally source but would like to have central supply as a backup, but this would require us to always keep extra stock at the local depot for a potential switch, which is not desirable from our side.
The functionality described in this webinar would support the ability of the country to start off as locally sourced and provide the opportunity for the switch to central supply.
In terms of how the stocks are maintained at the depot for the chance of running out of locally sourced medication, this is something that perhaps we’ve not been directly told about or involved in discussions on. However, in theory, if the locally sourced medication does not run out, then the contingency stocks at the depot could be used up by utilizing this IRT switch functionality as we approach its expiry.
The timing of registering the switch within the IRT is always important to ensure that once the decision has been made to move to a central supply there is perhaps time for the local depot to be stocked up to support the central supply affected by the switch before the switch has been registered in IRT.
Calyx’s Supply Simulation is a clinical trial supply forecasting service that improves trial efficiencies by helping sponsors estimate study supply requirements based on sophisticated simulation modeling methodology.
By testing a design aspect/assumption, as well as various potential recruitment patterns, Calyx’s in-house expert statistical design and trial supply consultants run a simulation to help sponsors make informed decisions about the quantity of medication to produce for use in clinical development.
Some of the most common use cases for the service include, but are not limited to:
- Evaluating the amount of medication required to start and maintain clinical trial enrolment
- Predicting how long an existing amount of study medication will last
- Determining the optimal site and depot buffer stock quantities required to ensure dispensation while reducing the cost and burden of excessive drug wastage
Calyx’s Supply Simulation service enables trial sponsors to run their clinical trial design as a simulation before patients are screened, medication has been ordered, and packaging designs/protocols have been finalized.
But how does it work? We break it down for you here.
“Simulations investigate several “what if” scenarios – since some parameters are unknown up-front – which is not possible with static forecasting.“
– Malcolm Morrissey, Calyx
Head of Statistics & Product Support Services
Modeling Methodology Summarized
Simulations allow for the comparison of multiple scenarios tailored to the study, from varying supply chain issues to variable patient titration.
Simulation or Forecast?
A forecast is defined as a static prediction or estimate of a future event or trend. A forecasting tool takes data as a point of entry and creates a likely state of the data in the future.
A simulation considers variable factors that could impact the data used as an entry point. In clinical supplies, a simulation considers the likelihood of some events occurring, and how they affect medication consumption.
With Calyx Supply Simulation, averages are obtained on parameters such as overage, number of shipments, and the risk of not being able to supply patients – these estimates are more accurate than forecasting estimates since we consider more factors and real-world variability.
Simulations provide us with ranges (or confidence intervals) that give us a sense of what we can expect to happen in the study with certain degrees of confidence. Therefore, Calyx Supply Simulation can be used to define what the optimal supply strategy should be e.g., how much buffer stock should be held at sites and depots.
Simulations also allow us to investigate several “what if” scenarios – since some parameters will be unknown up-front i.e., educated guesses – which is not possible with static forecasting.
Total Kits Required over Time
Chart 1: By running multiple simulations, an average of the total required kits is obtained, which is more precise than a simple forecasting average
Simulation Run
We refer to a simulation run as a single representation of what might happen in the study, resulting in the estimation of the total kits required over time for study execution.
By running multiple simulations (multiple runs), we can obtain an average for the total kits required, which is more precise than a simple forecasting average (See Chart 1).
Let’s suppose that we decided to base the number of kits required for the study on the simulation average, then:
- If the actual study behaved similarly to any of the lower simulation runs, we would be producing more kits than would be required
- If the actual study behaved similarly to any of the higher simulation runs, we would have insufficient kits produced
It is therefore vital that we not only consider the average but also the information provided across the many simulation runs. This cannot be achieved via a static forecasting approach.
Monte Carlo Simulation Technique
Some of the key elements of a clinical trial are variable, such as subject recruitment. It is important that the modeling approach incorporates variability into the study simulation so a full range of outcomes can be studied. Monte Carlo simulation allows such an approach, sampling values for key variables from underlying specified distributions of some of the input parameters.
Monte Carlo Simulation Applied to Patient Recruitment
While the average study recruitment for a group of sites may be e.g., two patients per site per month, there is a chance that an individual site may recruit zero patients or as many as six or seven patients in any month.
Monte Carlo simulations account for this variability by using a random number generator to determine each site’s recruitment for each month of the simulation. The random number generator is weighted to return numbers in the proportions that underlie the theoretical distribution. This process is like rolling a die to mimic real-world variability.
Benefits of a Service
The Calyx Statistics and Product Support Services team (SPSS) provides advice and consultancy on randomization and medication management as a standard for all the studies supported by Calyx IRT. Why not utilize that experience to help set up and refine your simulation to meet the requirements for your specific trial?
For example, from their experience of setting lookahead windows, trigger and resupply levels, and other IRT supply chain settings, the SPSS team can minimize kit wastage in a simulation to the point where the chances of a failed visit are at an acceptable level. The acceptable level is discussed with the sponsor and is study-specific. The further amount of medication needed to remove all failed visits can also be shown to ensure an appropriate level of overage is chosen.
Conclusion
Don’t fret over the difficult task of estimating how much drug to manufacture and package for your trial, and when. Leverage the expertise of Calyx’s in-house expert statistical design and trial supplies consultants to help you optimize protocol design to balance the cost of shipments and reduce waste, and assess the impact of protocol amendments, by comparing ‘what if’ scenarios via the Calyx Supply Simulation service.
Multiple elements feed into finding the optimal IRT design for each trial. Here’s a glimpse at how Calyx’s expert IRT designers consider the main elements to deliver effective randomization and trial supply management (RTSM) and meet your trial’s needs.
Understanding and Anticipating Risks
Risk should be a driver of IRT design, along with quality, patient safety, and data integrity. At Calyx, our focus is on understanding the protocol and the needs of the study team, then translating those into a solution that reduces risks.
Not just the risks you may expect, such as those related to the randomization algorithm and balance, patient dispensing and dosing, and unintentional unblinding. But also, the risks we anticipate, for example, those related to the potential for future adaptations and amendments, recruitment delays, data reconciliation, and supply chain concerns.
Calyx’s expert team knows what challenges a trial can face because we’ve managed a diverse range of risks across many protocol designs, study needs, and packaging plans in the 30 years we’ve been implementing IRT in clinical trials.
This knowledge and the experience of Calyx’s IRT designers translate into a deep understanding of how to mitigate risk; this serves as the foundation for each trial’s specialized, ongoing risk assessment and every IRT design we recommend.
Seeking to Understand
We start by understanding the protocol, including any unknowns, as well as the study team’s aims, end goals, and concerns. We don’t just ask what you want. We ensure we understand the why behind any requests.
By understanding the why, we can design an optimal solution that’s simplified for investigative site use and makes it easier to avoid data reconciliations and manage inevitable protocol amendments.
Considering the User
Investigative site personnel use many different systems, and their time is precious, so the IRT system for each study must be easy for site personnel to use and understand.
An optimal IRT design considers the consequences for the investigator/site users for the life of the trial, including the need for data reconciliation and (protocol-permitted) flexibility as sites and patients can’t always stick to the visit schedule.
One consideration is if the IRT system is used to register non-dispensing visits following randomization. Except for withdrawal and completion, this has dual risks of data reconciliation and user compliance with registering visits in the IRT system. When the site obtains nothing during an IRT transaction, e.g., a subject number or a kit number, compliance is reduced, so data recorded in the IRT system is poor.
Considering the Life of the Trial
An effective IRT design takes into consideration everything that happens until database lock and study closure. With that in mind, Calyx IRT is designed to be flexible and to minimize future effort.
Calyx IRT designers are responsible for ensuring the sponsor is aware of the real-world implications of decision-making regarding data collection in IRT, which has risks of reconciliation and compliance issues as well as increased effort for site personnel.
For example, if patient data is collected both in the eCRF/EDC and the IRT system, this can lead to data reconciliation issues. An experienced IRT partner should collaborate with the study team to understand the reason behind any request to collect duplicate data in the IRT system if it is not needed to make patient treatment decisions.
Flexible and Future-proof
An optimal IRT system is designed to be flexible to make immediate changes as and when needed – from increasing enrolment caps to reflect changes to recruitment to including new countries and depots, etc. And the system should be future-proof to anticipate possible changes – from immediately adding new doses for new cohorts of patients or closing treatment arms following a safety review.
Teamwork
Every study has a dedicated team advising on all aspects of protocol and study design, including Calyx subject matter experts (SMEs) in randomization, trial supply management, integrations, and reporting.
Each study’s randomization and trial supply expert, as well as its IRT project manager, remain in the dedicated study team until its closure, supporting the inevitable issues that sponsors face during the running of a trial.
For sponsors who work with Calyx across many trials, we can extend the support to define standards that ensure consistency and reduce effort across their development programs.
Focus on Complexity
Calyx’s IRT solution designer leverages Calyx’s core of pre-validated functionality designed to support all trial needs, covering study, site, and subject management, randomization and dosing, and trial supply management from lot release to destruction – all with corresponding reports. We advise which core functionality is required and we customize it to best meet the needs of each protocol, with no limits to the level of complexity we can address.
We start design discussions by presenting what the Calyx SME team has prepared – our understanding of the protocol and patient journey and a design approach based on a review of initial known risks. We can then focus on any complexities, directing time and attention to what’s important for each study.
Conclusion
The objective of Calyx’s IRT team is to design an optimal system that is right for you, and right for the life of the trial, taking into consideration your aims and needs as well as the interests of site personnel and all system users.
This is the first of two articles on the consequences that could arise if an interactive response technology (IRT) system isn’t designed and/or implemented correctly and how a trial could quickly go off track based on risks related to randomization, drug allocation, and trial supply.
It’s well-accepted that unintentional unblinding during clinical trials can have serious ramifications for data integrity. But what’s not as commonly recognized is how data itself can be the catalyst for unblinding, especially during transfers.
In a recent webinar from our series on unintentional unblinding, which includes randomization and drug supply risks, Calyx RTSM experts reviewed scenarios of how study data can lead to unintentional unblinding.
Here we address some of the questions that arose during the webinar related to Calyx’s approach and suggestions for minimizing unblinding risks related to data during clinical trials
Sometimes IRT vendors provide a data dump, so you are sent the whole table or a segregation of the table. Can Calyx create tailored data sets?
Yes. We believe that data dumps in general are not very useful. We can create specific/tailored datasets that enable you to pick and choose what you want to see and how through a data transfer request form.
Part of the risk discussion around creating a data set will be what you need and why – what combination of all the data stored in the IRT system the statistical programmer needs in one report/transfer.
Our SaaS reports team can select data from all the different places the IRT system stores it, bringing it all into one data set and tailoring it to the format and the labeling and transmission process that the sponsor or the statistical programmer needs.
When you’re combining data, we just need to be very careful that when you create that picture in that one data set, you’re not showing those differences and highlighting something that could lead to unblinding.
Can you provide audit trail data?
Yes, absolutely we can provide audit trail data. We have some packages we can create easily and quickly because this is a very common request during regulatory audits, but we can provide any audit trail of what happened to data over time.
Using an example of IMP/kit data, maybe you want to know the status of how a kit went from arriving at the site all the way to being shipped to the return depot for destruction. Sometimes people want to see what happened to that kit all along the way.
We’re able to provide audit trail data for any data point that is collected in the IRT system.
How do you allow auditors access to blinded data during audits?
Usually, a specific request is formed during an inspection and then brought to us for consultation on how to best meet those needs (if the data is not already available in a pre-validated report).
Maybe the site can show data they have in an existing web report, or the sponsor team has access to data that they can display. But sometimes customer requests arise, like audit trail data which require an ad hoc data set; this can be provisioned to the authorized recipients who can determine how they want to deliver that to an auditor as appropriate.
What are the timelines for critical requests?
We do manage critical requests – the delivery timeline is dependent on what you need, when you need it, and the reason for the request e.g., is it for an inspection or an audit happening now?
We will always work together to respond to critical requests as quickly as possible, provide high-quality data, manage any risks (including unblinding), and make sure you have what you need within the timeline you need.
For double-blind trials, our clinical supplies team would like to provide extra expiry dates to sites to help them prepare for relabeling activities. Can you speak to how 2 different lots with different expiry dates can impact the blinded site team members?
Before relabeling starts, we need to create a plan and decide which kits will be relabeled.
If the expiry date is on the label, some kits with the old expiry will be needed to allow patient scheduled dispensing and site shipments to take place while relabeling is being completed.
Whether some kits need to be returned to a central depot should be considered. But it is also usual to see some kits reserved for relabeling at the depot where the relabeling will take place. This can be achieved by updating the status of the chosen kits in the IRT system, to avoid them being shipped to the site or dispensed.
We also need to understand how the kits are going to be relabeled. For example, is it just expiry that’s being relabeled, or is the actual kit number being over-labeled too?
Where relabeling of kits takes new kit numbers from the medication packaging list, this is not usually a concern as the list would typically already have overage built in. Moreover, with a scrambled kit number used in the original list, assigning a new kit number after the relabeling will mean it is indistinguishable from an old kit number.
The relabeling activity at the depot should not allow the new batch release at the depot to affect the blinding of the study. Including overage at the start in the packaging list will avoid a visible separation of numbers across batches.
We always need to consider, “Is there a difference? What difference are we going to see?”
The switch to assigning a new expiry date (when expiry date is visible to blinded staff) for only one treatment group may be a risk. If the newly re-labeled kits are only supplied when the old expiry date kits can no longer be allocated, then there is no risk. But to avoid failed shipments, it is likely that kits with a new expiry date on the label would arrive on-site before the expiry of the old kits. If one medication type is depleted within site stocks from the expiring kits, then the remaining old kits should be removed from site stock to avoid any observable difference. At a low recruiting site, this may not be a concern.
When lot number is unblinding
We do manage studies that have unblinding lot numbers, where the lot only contains one medication type and that’s obviously a big challenge.
In addition to blinding the lot number, if you had one lot/medication type that was not being relabeled and another lot where some of it is relabeled, that new expiry date is only going to relate to one medication type. In this case, we will see some segregation of kit/IMP types.
If we have different expiry dates for active and placebo, for example, one of the options that we propose is using the earliest expiry date for both batches.
You cannot use the longest, but using the earliest expiry date for both has the potential to increase wastage for the medication type that lasts longer. An extra packaging run of the other IMP type later could enable us to match the expiry dates for both types again, or allow the use of a new ‘earliest expiry date’ when released. Ongoing management of the expiry dates could be needed to make the best use of the medication.
Once again, our aim is to try and remove the visible difference, if there is a blinding concern.
When an unblinded pharmacist is the only person who can see the actual original kit type before it’s made up and handed to the person doing the dosing, this certainly simplifies the actions needed to maintain the blind.
This answer covers several scenarios, but it really mimics the sort of IRT/RTSM questions about relabeling we discuss with the sponsor study team. Brainstorming, discussing, and looking to see what we can do in the system to remove differences, to dial out the risk and investigate what is practically achievable.
