Vivarium CSU Multiple Myeloma Registries Phase 2

Abbreviations

Abbreviation	Definition	Note

1.0 Background

This page describes Phase 2 our multiple myeloma simulation project. Phase 1 initially focused on a planned multiple myeloma patient registry, but registry enrollment has been much lower than expected, so it is no longer a current focus of the client. Instead, we will use our microsimulation to help the client answer various business questions about their new drug isatuximab.

• Phase 1 concept model and HUB page

• Phase 2 HUB page

Our Phase 1 multiple myeloma simulation focused on a scale-up of isatuximab as a first-line treatment in the USA. For Phase 2, the client has asked us to expand our simulation in the following ways:

1. Expand the set of mutually exclusive treatment categories beyond the three treatment categories considered in Phase 1.

2. Consider additional patterns of scale-up of Isa. In particular, model an uptake of Isa following Dara as a first-line treatment, after a “washout” period.

3. Expand the modeled locations beyond the USA.

We gave a presentation to the CSU client on January 20, 2022 proposing some potential business questions for Phase 2, along with an expanded set of treatment categories based on treatment guidelines from the NCCN as suggested by Manoj Menon in Sep-Oct 2021.

1.1 Project Overview

1.2 Literature Review

2.0 Modeling Aims and Objectives

3.0 Concept Model Diagram

4.0 Vivarium Model Components

4.1 Cause Models

• Multiple Myeloma

4.2 Risk Exposure Models

• Multiple Myeloma Risk Factor Exposures

4.3 Risk Effects Models

• Multiple Myeloma Risk Factor Effects

4.4 Intervention Models

• Multiple Myeloma Treatment

5.0 Simulation Scenarios

We have four scenarios that differ only in how treatment is assigned.

• Baseline scenario: Sophisticated treatment models with postprocessing rules

• Alternative scenario 1 (Naive treatment): Naive treatment models with the same postprocessing rules as baseline

• Alternative scenario 2 (Isa-after-Dara): Sophisticated treatment models with baseline postprocessing rules modified to allow Isa directly following Dara in Line 2

• Alternative scenario 3 (Isa frontline): Sophisticated treatment models with baseline postprocessing rules modified to allow Isa in Line 1

For the details of sophisticated vs naive treatment models and the postprocessing rules for each scenario, see the treatment documentation.

6.0 Simulation Parameters

6.1 Locations

United States.

6.2 Population and Randomness

Population description:

• Cohort type: Prospective closed cohort of individuals aged 15 years and older. The sim duration is 15 years (see below), so results above age 30 will not be impacted by the open/closed distinction; essentially all multiple myeloma occurs at age 30+.

• Size of largest starting population: 100,000 simulants

• Time span: Jan 1, 2013 to Dec 31, 2027 (Jan 1, 2013 to Jan 1, 2023 is a 10-year long burn-in period)

• Time step: 28 days (final run) or 90 days (intermediate runs) – the only input data that depends on the timestep is the time-varying hazard; we will have a copy of those CSVs for each of the two time step values

6.3 Timeframe and Intervention Start Dates

7.0 Model Builds and Validation Tracking

Model verification and validation tracking

Model	Description	V&V summary
Model 0 (round 1 without age stratification)	Phase 1 Model 9 re-run wihtout age stratification	Found a bug with the treatment observer in which all simulants are not_treated in Line 1. Cannot meaningfully compare RRMM prevalence or prevalence and incidence of MM overall to Phase 1 (or in the case of the latter, GBD) without age stratification. RRMM prevalence does not appear to converge in our burn-in period, accumulating simulants continuously in the fourth and higher relapse state – I have confirmed that this issue was also present in Phase 1, but we did not know it. Not investigating this for now, in the hopes that the new survival curves we plan to use anyway will resolve this problem as well. Treatment effects are unchanged from Phase 1 but they do not look correct – it appears there was some regression between Model 6.5 and Model 9 in Phase 1. Not investigating this for now, in the hopes that the treatment changes we plan to make anyway will resolve this problem as well. Survival curves are unchanged from Phase 1, though they are systematically biased relative to input curves from Braunlin – a limitation we accepted in Phase 1. Before completing the PR (do not have these versions of the notebooks), found a bug with make_results putting information from many columns into the age column – this was quickly fixed.
Model 0 (round 2 with age stratification)	Phase 1 model 9 re-run with age stratification	Same notes as above round one, except… Comparisons to GBD are as expected, deviating more significantly as age increases, similar to phase I
Model 1	Expanded treatment categories and hazard ratios (placeholder values)	Underestimating survival and progression hazard rates, overestimating survival, and overestimating prevalence compared to model 0. Thought to be due to placeholder values for treatment effects hazard ratios and incompatible PAFs that have not yet been updated. Issue found with treatment coverage algorithm - all simulants initialized into PI-IMiD-Dex category for line one at start of sim and retreatment is not allowed, so underestimation of PI-IMiD-Dex treatment category in the early years of the sim. Otherwise, treatment coverage looks as expected and does not vary by year or age. Treatment hazard ratios generaly reflect input values (when compared to PI-IMiD-Dex as reference, systematically overestimated when compared to population rate, suspected to be a result of incompatible PAFs), but with high degree of uncertainty in some cases
Model 2	Use TTNT directly for hazard of relapse, instead of subtracting OS from PFS	Prevalence overestimate increased relative to Model 1. More simulants in NDMM relative to Model 1. New hazard rates appear to be applied correctly. No change in treatment coverage or treatment effects.
Model 3	Sophisticated treatment prediction model as a scenario and business-rule-modified alternative scenarios	After a few rounds of fixes: Base hazard rates match input in interactive sim. Risk factor hazard ratios match input in interactive sim. Due to component structure, unable to verify treatment effects in interactive sim – but those have been verified in Model 2, before the interactive sim approach was necessary. Isa and Dara coverage approximately matches the target values by year and scenario. Due to reduction in stratifications, not able to check for regressions since Model 2.
Model 3 - China location	Change location to China and add China-specific treatment probability postprocessing rules	Ran updated code only in the China location: No regressions from previous Model 3 run. China-specific treatment targets roughly replicated in simulation, are modified by Isa/Dara projections as expected.

8.0 Desired Outputs

8.1 Final Outputs for Client

8.2 Requested Outputs from Vivarium

8.2.1 Treatment output table

Note

This should be similar to the treatment output table from Phase 1, with an added stratification by age.

Treatment observer metrics

Variable	Definition
input_draw	Input draw number. len(input_draw) = 30
scenario	Intervention scenario. Choose from [‘naive’, ‘baseline’, …]
year	Calendar year
treatment_line	Treatment line/disease state a simulant is in. If a simulant is in state multiple_myeloma_{x}, assign this simulant treatment_line {x}. Choose from [1, 2, 3, 4, 5+]
treatment_category	Treatment regimen category a simulant initiated. For example, IMID+PI+Dex.
age	Age group a simulant is in.
value	Count of simulants in age group age who initiated the treatment_category in treatment_line during year.

8.2.2 Survival output table

Note

This is very similar to the survival output table from Phase 1, with an added stratification by treatment category.

Survival observer metrics

Variable	Definition
input_draw	Input draw number. len(input_draw) = 30
scenario	Intervention scenario. Choose from [‘naive’, ‘baseline’, …]
treatment_line	Treatment line/disease state a simulant is in. If a simulant is in state multiple_myeloma_{x}, assign this simulant treatment_line {x}. Choose from [1, 2, 3, 4, 5+]
treatment_category	Treatment regimen category a simulant is in. For example, IMID+PI+Dex.
period	The number of days since the entrance into the treatment_line that the count measures were evaluated on.
alive_at_start	Count of at-risk simulants alive at period - 28 days since they entered treatment_line.
died_by_end	Count of alive_at_start simulants who died between period - 28 and period days since they entered treatment_line.
progressed_by_end	Count of alive_at_start simulants who progressed to next line of treatment/disease state between period - 28 and period days since they entered treatment_line.
sim_end_on	Count of alive_at_start simulants without death or progression at the end of the simulation between period - 28 and period days since they entered treatment_line.

Time frame for survival observer (timestep = 28 days):

1. start_date = 2021-01-01, end_date = 2025-12-31

2. start_date = 2025-01-01, end_date = 2025-12-31

9.0 Back of the Envelope Calculations

10.0 Limitations

11.0 References