Decision analytic modeling for the economic analysis of proton radiotherapy for non-small cell lung cancer

Introduction

Locally advanced non-small cell lung cancer (LA-NSCLC) is a particularly challenging disease to treat with radiation, given the extent of disease, the sensitivity of lung tissue to radiation, and the close proximity to organs at risk (OARs), such as the esophagus and heart. Furthermore, the necessity to treat centrally located mediastinal disease while not exceeding spinal cord tolerance requires photon beam arrangements that deposit entrance and exit dose through healthy lung and esophagus. Previous dosimetric comparisons between proton beam therapy (PBT) and X-ray (photon) therapy for LA-NSCLC have demonstrated reduced dose to these OARs with protons (1,2). With this reduced dose, there is a potential to lower overall rates of toxicities and morbidities of treatment.

The toxicity of therapy for LA-NSCLC can often be quite severe, requiring hospitalizations and aggressive interventions, which have high associated costs of management. One of the main criticisms regarding PBT compared to photons is the higher upfront costs of delivery. However, a reduced toxicity rate could avoid the costs of toxicity treatments, thereby opening up the possibility that PBT may actually lower overall costs. A recent PTCOG consensus statement advocates for an evaluation of each case in order to select patients for proton therapy (3). The question is, under what circumstances can clear distinctions be made between the two modalities with respect to cost?

Cost and cost benefit analysis are applications of decision theory, and are most often used to guide clinical decision making on a population based level, by seeking to minimize cost, maximize benefit, or by utilizing a willingness-to-pay threshold in order to balance these oft-competing aspects of care. These models are most commonly developed only after a technology has emerged as a clinical option for patients, and are applied to rank various treatment options. It is also common for a subset analysis to be performed, as different treatment options typically have differing costs and effectiveness for different groups of patients. These subsets are often based on disease staging, but stratification by other decision variables is possible.

The purpose of the current work is to develop a model to explore the effects of using different classifications of patients on the ultimate costs of treatment for different treatment choices (PBT, 3D conformal X-ray therapy and intensity modulated X-ray therapy). We studied four different methods of classifying patients: (I) rate-adjusted; (II) toxicity-adjusted; (III) dosimetry risk group; and (IV) radiosensitivity. This comprehensive approach provides a detailed understanding of the different factors that affect the costs for each of the treatment choices. These analyses serve different purposes including long-term financial planning, isolating the causes of high costs, and choosing the appropriate therapy for any given patient.

The structure of the model is flexible enough that we have expanded our analysis beyond the traditional cost-benefit analyses to explore the effects of a test for individual radiosensitivity. Historically, a patient’s propensity for radiation-induced toxicity has been unknown and only population-based statistics were available. However, with a greater understanding of its molecular basis, genetic testing for this is becoming more likely (4). Given the critical role that normal tissue damage plays in this analysis, knowledge about a particular patient’s chance of suffering radiation-induced complications would be very important. Such knowledge could be used to determine a particular cohort who would benefit and/or it could be used to help in individual decision making. Using this example, we demonstrate the capabilities of using probabilistic knowledge representation to explore different scenarios and to highlight areas that need more information or more exact predictors.

Literature review

Decision analysis is a systematic and quantitative approach which synthesizes different types of available information such as medical benefits, toxicities, and costs, typically from many sources, in order to develop tools that guide physicians, patients, third-party payers, or health care policy makers to optimal decisions (5,6). It is most commonly applied to selecting an optimal strategy in the form of cost-benefit analysis (7), but can focus on cost (8) or benefit (9) alone. Decision analysis is far more general, however, and can be used to guide nearly any decision that needs to be made such as using value of information analysis (10) to estimate which parameter or groups of parameters contribute most to decision uncertainty and may be candidates for future research (11,12), or determining the optimal sample size for a clinical trial (13).

The most common modeling techniques applied to radiotherapy include decision trees (14) and state-transition models (15). Decision trees are best suited to simple problems with a fixed time horizon and without time-dependent parameters, although the numbers of states in the model can become difficult to manage since each state is represented many times on the tree. State-transition models are able to incorporate time-dependent parameters and, through microsimulation the probabilities can depend on the characteristics of individuals in the simulation, allowing the impact of different parameters to be examined. Influence Diagrams are used in many fields and are useful for integrating knowledge from disparate sources, exploring the effects of different decision variables, and determining information needs in order to make an optimal decision (16). Influence diagrams, which have long been used for decision modeling in health care (17), are gaining in popularity for cost-effectiveness analysis (18) and decision analysis in radiotherapy (19).

No guidelines exist for a review of decision analysis, to the best of our knowledge. Therefore, this review of decision analysis for proton therapy for NSCLC is informed by recent guidelines (20) although our focus is more general. Searches were conducted in Medline and Embase and abstracts from the Particle Therapy Co-Operative Group and Particle Therapy Co-Operative Group of North America. In order to find results that were relevant for cost-benefit analysis and decision analysis in general, search results were structured so that results contained one term from the list, “non-small cell lung”, “non-small cell bronchial”, and one term from the list “NSCLC”, “cancer”, and one term from the list, “cost”, “decision”, and the word “proton”. Results were reviewed by multiple co-authors and references of articles found were manually screened for additional articles.

Only one full publication was found, performing cost-benefit analysis for inoperable and operable stage 1 NSCLC, including proton therapy, carbon-ion therapy, and photon SBRT and conventional radiotherapy (CRT) as treatment strategies (21) in the Dutch health care system. The authors also undertake an extensive cost analysis for delivering radiotherapy (8) and meta-analysis of effectiveness studies (22) and survey patients for utilities for acute health states. They find that SBRT and Carbon-ion therapy dominate protons and CRT for inoperable patients, and SBRT is the most cost-effective for operable patients. In addition they perform probabilistic sensitivity analysis and use value of information analysis to determine that reducing uncertainty in the effectiveness of both carbon-ion therapy and SBRT for this population would have the greatest impact on future decision making, in terms of monetary gains where QALYs have been expressed as euros with a willingness-to-pay-threshold of €80k. They also undertake a scenario-based analysis, using only more recent publications to determine the effectiveness of proton therapy, which show that the effectiveness of proton therapy has been increasing for this population, and determine that continued research into improving the effectiveness may result in proton therapy emerging as a more competitive option.

Two studies focusing on locally advanced NSCLC were found, both in abstract form. One study (23) used a state-transition model with a 10-year time horizon comparing PBT, 3D-conformal (3DCRT), and photon intensity modulated radiation therapy (IMRT) and found proton therapy to be borderline cost effective in the Belgian health care system. Another study (24) used an Influence Diagram as a cost model, including the upfront cost of delivering 3D-conformal, IMRT, and PBT, and the cost to manage acute toxicities. The study did not find a scenario where PBT reduced financial costs, in part due to the uncertainty in which individual patients will develop severe toxicity.

Decision analysis has been utilized to compare the expected gains or losses from waiting to adopt PBT until after a prospective trial has been completed, to the expected gains or losses from adopting PBT before undertaking the trial for stage I NSCLC (25). If the trial turns out to show a sizable outcome advantage then waiting to adopt the technology forgoes gains for current patients, but if the trial result is negative, then large sunken costs in term of facilities built unnecessarily cannot be recovered. This approach is an example of value of information analysis, and probability distributions are used to represent uncertain parameters, with the uncertainty in outcomes for PBT as the dominant uncertainty in the decision. Since the future trial result is unknowable, results are averaged over all possible trial outcomes. For this patient population the authors find that adopting PBT now and conducting a trial has the largest expected net gain.

In summary, decision analytic techniques have been used to perform cost, cost-effectiveness, and value of information analysis to direct future research for PBT for NSCLC. Cost and cost-effectiveness studies are most often performed for decisions when sufficient evidence exists, while value of information techniques are used in areas where evidence is sparse, in order to direct research towards reducing uncertainty in key decision parameters.

In the remainder of the paper we utilize an influence diagram in order to highlight its role in decision analysis, by showing that even in the setting of a simple cost-model that includes monetary costs for the delivery of proton therapy and costs to manage acute toxicities, it can direct future research to study impactful decision variables and also place boundaries on the uncertainty of the research results in order for new information to guide clinicians to optimal decisions.

Methods

Model design

An influence diagram (Figure 1) was created to model the delivery of protons or photons (3D conformal or IMRT) for LA-NSCLC. An influence diagram is a Bayesian network with decision, chance, and utility nodes (4). The basic model structure (yellow nodes) included a single decision node (protons, 3DCRT, or IMRT), several chance nodes (predictors of toxicity and toxicity rates), and a utility node (overall costs of treatment). The model was based on a patient receiving a 30-fraction definitive course of concurrent chemo-radiation for stage III NSCLC. A 6-month time horizon was assumed for toxicity rates.

Figure 1An influence diagram used to model the decisions, chance variables and outcomes of radiation therapy of LA-NSCLC. The yellow-shaded nodes denote the basic model. The rectangular node is a decision node (radiation modality); oval nodes denote chance variables (use of chemotherapy, age, toxicity grades of V20, V60 and mean lung dose), and the diamond-shaped node is the outcome metric (cost). The purple-shaded nodes indicate that part of the influence diagram that represents a potential diagnostic test for patient-specific radiosensitivity, with an exploratory decision node used to describe the clinically useful operating region for an ROC curve. LA-NSCLC, locally advanced non-small cell lung cancer.

The nodes in purple represent an as-yet-undeveloped medical test or ‘biomarker’ which predicts acute esophagitis requiring hospitalization. The biomarker tests for radiosensitivity, the true state of which is only knowable up to the level of sensitivity and specificity of the test. This introduces an exploratory decision node for sensitivity and specificity which is adjustable, and allowed us to set requirements for test accuracy in order to lower costs. In order to calculate results using only currently available information it is possible to disable these nodes in the model, through appropriate values in the conditional probability table.

For photon toxicity rates, we included published predictors of both pneumonitis and esophagitis found to be significant in large meta-analyses (26,27). For pneumonitis, the included predictors were age (≤ vs. >65 years old), chemotherapy type (cisplatin/etoposide/other vs. carboplatin/paclitaxel), the percentage volume of the tumor receiving 20 Gy, V20 (< vs. ≥25%), and mean lung dose, MLD (< vs. ≥10 Gy). For esophagitis, the only significant predictor was the fraction of the esophagus that received 60 Gy, V60 (<0.07% vs. 0.07–17% vs. ≥17%). The dependence of the toxicity rates on the radiation modality is instantiated in the conditional probability tables that are represented by the nodes and links of the influence diagram (Figure 1).

Grades of pneumonitis and esophagitis were defined according to the RTOG/EORTC Acute Radiation Morbidity Scoring Schema. Because the need for hospitalization is not explicitly stated for certain toxicities, we assumed that all grade 4 toxicities required hospitalization and aggressive interventions. Per a previously published study by Shah et al. (28), the management of grade 3 pneumonitis was assumed to occur as an outpatient with continuous oxygen and intermittent steroids. Similarly, although grade 3 esophagitis can be severe and sometimes require dilation, we assumed management as an outpatient requiring aggressive hydration, gastric feeding tube placement, and/or initiation of total parental nutrition. Grade 2 toxicities were less severe and managed as an outpatient with intermittent oxygen and steroids (pneumonitis) and narcotics/liquid diet (esophagitis). Grade 1 toxicities did not require any interventions, given the low severity.

Determination of costs

The utility assessed in this model was the direct overall financial costs of treatment, which included the radiation delivery costs and acute toxicity treatment costs. Radiation costs were calculated from 2014 Medicare non-facility reimbursement rates per current procedural terminology (CPT) code (29), checking a subset of the costs against the 2017 values demonstrated that costs changed negligibly. The costs were divided into the different phases of radiation therapy: simulation, planning, radiation delivery, and consult/weekly management. Costs for grade 0–3 toxicities were based on published estimates by Shah et al. Since this publication does not reference costs for grade 4 toxicities, which require hospitalization and aggressive intervention, we used discharge data from the Nationwide Inpatient Sample (NIS), Healthcare Cost and Utilization Project (HCUP), and Agency for Healthcare Research and Quality based on MS-DRG code (30). For grade 3 and 4 esophagitis management, which can require outpatient or inpatient surgical procedures, we also included estimates based on the ICD-9 code for those specific interventions, using the NIS, HCUP, and AHRQ discharge data.

Types of analyses

We first performed three separate cost analyses: rate-adjusted, toxicity-adjusted, and risk group-adjusted analyses, without including the effects of a possible biomarker. For the rate-adjusted analysis, we determined the average overall costs of treatment per patient for each radiation modality assuming a population with certain rates of toxicities. Rates of population toxicities were obtained from the photon STRIPE toxicity meta-analyses and the phase II proton lung cancer trial by Chang et al. (26,27,31). Since the STRIPE meta-analyses reported overall rates of grade ≥2 pneumonitis/esophagitis, we applied published toxicity distributions for each grade, calculated as the average from three previously published studies, to determine the STRIPE toxicity rates per grade (32-34). Differences in toxicity rates for 3D vs. IMRT for photons were taken from a recent retrospective analysis (35).

For our rate-adjusted analysis, individual dosimetric parameters were not modified (in contrary to the risk group analysis), as published toxicity distributions inherently reflect an ability to meet dosimetric parameters that were not explicitly stated in those publications. We calculated a “rate adjusted cost” based upon the expected rate of each grade of toxicity (pneumonitis and esophagitis) from each modality (3DCRT/IMRT/protons). Toxicity-adjusted analyses were performed by calculating the overall treatment costs for each toxicity grade per patient and comparing them for each modality. For the risk group-adjusted analysis, group criteria from the photon toxicity meta-analyses were applied to the model, generating average overall treatment costs per patient in each group. Risk group classifications incorporated treatment planning parameters, such as V20, MLD, and V60, which were modified to affect the toxicity distributions and determine treatment costs.

The previous analyses were conducted with the effects of the radiosensitivity biomarker disabled in the cost model. With the effects of the biomarker in effect, the sensitivity and specificity were both varied independently from 0 to 1 in a 2-way analysis to determine which values resulted in a cost reduction for proton treatments. This analysis explores the cost trade-offs between false positives which may result in greater upfront costs (protons) in cases for which increased toxicity-avoidance is unnecessary, and false negatives which result in greater overall costs from a photon treatment which also accrues costs to manage toxicities that could have been avoided.

Model assumptions

We assumed that overall survival and local control were equivalent between photons and protons, given the similar target volumes and doses used for both modalities. When exploring dosimetric input parameters, we assumed that protons always resulted in superior dosimetry compared to photons. For cases where photons result in an equivalent or superior dose distribution then it is assumed photons would be the modality of choice. At our institution, protons are considered on a case by case basis, only when they impart some dosimetric advantage. For example, for the intermediate risk group V20 dosimetric parameter, we assumed protons always resulted in a V20 <25% and photons ≥25%. If this parameter was not met with protons, we assumed limited ability for protons to reduce toxicity risks compared to the photons. Lastly, we assumed a third-party payer (Medicare) perspective for calculating costs.

Results

Radiation and toxicity costs

Based on a 30-fraction course of radiotherapy, the upfront cost of delivering PBT, IMRT, and 3DCRT was $41,061.80, $23,893.83, and $16,730.37, respectively. The baseline cost difference between proton and photons was $24,331.43 (3DCRT) and $17,167.97 (IMRT). The phase of radiation which contributed the most to the baseline cost difference was the actual treatment delivery, with a cost of $35,564.40, $17,764.20, and $12,914.01 for PBT, IMRT, and 3DCRT, respectively.

Costs of managing acute pneumonitis and esophagitis are as follows. Grade 0/1 acute toxicities had no associated costs. Grade 2 pneumonitis was defined as requiring narcotic antitussives and intermittent steroids, with an associated cost of $96.50/month. Grade 3 pneumonitis was defined as requiring intermittent oxygen and steroids, with an associated cost of $1,466.27 (one time) and $291/month. Grade 4 pneumonitis was defined as requiring continuous oxygen, assisted ventilation, or at least a 4-day length of stay in the hospital or ICU, with an average cost of $18,414.00 (range, $6,873–$45,023). Grade 2 esophagitis was defined as requiring narcotics, puree, or liquid diet, with a cost of $26.54/month. Grade 3 esophagitis was defined as requiring 2 liters of IV fluid infusion (outpatient), nasogastric tube placement, and/or total parenteral nutrition, with an average cost of $12,687.86 (range of $444.01–$29,332.01). Grade 4 esophagitis was managed with an EGD, or a surgical procedure/repair for perforation or fistula, with an average cost of $33,235.00 (range of $9,109.00–$65,607.00).

Toxicity-adjusted analysis

Overall costs were calculated per grade of pneumonitis and esophagitis and are presented in Figures 2,3, respectively. As expected, as the grade of toxicity increases, the overall costs of treatment increases. We analyzed this data to determine if there is a scenario in which a patient receiving photons experiencing a specific grade of toxicity would have overall costs that exceeded a proton patient. For pneumonitis, a patient treated with IMRT experiencing a grade 4 pneumonitis had overall costs that exceeded a proton patient who experienced grade 0–2 toxicities ($42,307.83 vs. $41,061.80–$41,640.80). The 6-month costs associated with a patient treated with 3DCRT who experienced any grade pneumonitis always were lower than proton patients with any grade pneumonitis. For esophagitis toxicities, an IMRT patient experiencing a grade 4 toxicity had costs exceeding that of a proton patient with grade 0–3 toxicities ($57,129.43 vs. $41,061.80–$53,749.48). Similar results were seen if a 3DCRT patient experienced a grade 4 esophagitis with costs exceeding a proton patient with grade 0–2 esophagitis ($49,965.97 vs. $41,061.80–$41,221.04).

Figure 2 Vertical bars indicate overall costs (cost of radiation plus cost of toxicity management) per grade of pneumonitis. Horizontal lines indicate baseline cost of each modality. Both bars and lines color-coded to legend.

Figure 3 Vertical bars indicate overall costs (cost of radiation plus cost of toxicity management) per grade of esophagitis. Horizontal lines indicate baseline cost of each modality. Both bars and lines color-coded to legend.

Rate-adjusted analysis

Distributions of pneumonitis and esophagitis were applied to the model based on a population of LA-NSCLC patients (26,27,31,35). In general, rates of symptomatic pneumonitis were lower with the use of proton radiation. Grade 4 pneumonitis was a rare toxicity with a photon rate of 1% compared to a proton rate of 0%. Based on the expected rate of each grade of radiation pneumonitis with each of these modalities, the average cost of treatment per patient was $41,200.67 (protons), $24,222.56 (IMRT), and $17,296.83 (3DCRT). Based on the ability of protons to reduce rates of pneumonitis, $427.59 (3DCRT) and $189.86 (IMRT) were recovered from the baseline cost difference with protons. The overall average costs of protons were not found to be lower than photons based on these pneumonitis rates. Summary of these results are presented in Table 1.

Table 1 Population-based rates of pneumonitis for photons and protons and associated rate-adjusted costs. Money recovered is the difference between the population average costs per treatment and the baseline cost difference
Full table

For esophagitis, PBT resulted in lower rates of symptomatic esophagitis (26,27,31). Grade 4 esophagitis was a rare toxicity with a photon rate of 2% and a proton rate of <1%. Based on population esophagitis toxicities, the average cost of treatment per patient was $42,835.68 (protons), $26,746.04 (IMRT), and $19,582.58 (3DCRT). Compared to the baseline cost difference, the use of PBT was able to recover $1,078.33 of that difference through reducing the rates of esophagitis. Overall, PBT was not found to lower the overall costs of treatment based on the rates of esophagitis. Results are summarized in Table 2. Since pneumonitis and esophagitis are not mutually exclusive toxicities, we analyzed costs combining both toxicity distributions. Average cost per patient for this combined analysis was $42,974.55 (protons), $27,074.79 (IMRT), and $20,149.01 (3DCRT). Cost recovered from proton toxicity reduction was $1,505.89 (3DCRT) and $1,268.21 (IMRT).

Table 2 Population-based distribution of esophagitis for photons and protons and associated rate-adjusted costs. Money recovered is the difference between the population overall costs and the baseline cost difference
Full table

Risk group analysis

As part of the STRIPE meta-analysis for pneumonitis, Palma et al. presents different pneumonitis risk groups based on the results of the recursive partitioning analysis. This includes a high risk group, two intermediate risk groups, and two low risk groups (26,27). We used the model to determine the potential cost benefits of protons for each risk group. For the high risk group (age >65 years old, concurrent carboplatin/paclitaxel), the overall cost difference between protons and photons was $22,594.06 for 3DCRT and $15,803.45 for IMRT, with a cost recovered of $1,737.37 (3DCRT) and $1,364.52 (IMRT) from the baseline cost differential. For intermediate risk group 1 (age ≤65 years old, MLD ≥10 Gy, carboplatin/paclitaxel) and intermediate risk group 2 (cisplatin/etoposide/other, V20 ≥25%), the cost recovered using protons was smaller, averaging $1,463.62 (3DCRT) and $1,256.97 (IMRT). Lastly, for low risk group 1 (age ≤65 years old, carboplatin/paclitaxel, MLD <10 Gy) and low risk group 2 (cisplatin/etoposide/other, V20 <25%), the cost recovered with protons averaged $1,282.04 (3DCRT) and $1,152.92 (IMRT).

Effects of esophageal radiosensitivity biomarker on decision-making

Overall costs of treating with photons and managing grade 4 esophagitis exceed the costs of PBT when PBT avoids grade 4 esophagitis, as shown in Figure 3. A perfect test for esophageal radiosensitivity would enable patient selection for protons in cases where protons exhibit a superior dose distribution (lower V60). Since a perfect test is unlikely we establish boundaries for sensitivity and specificity in order for a test to lower overall costs. The gray line in Figure 4 is an isocost line for equal costs, tests with TP and FP to the left and above the line will lower overall costs, tests to the right and below increase them. Increasing distance from the isocost line indicates greater cost advantages or disadvantages, depending on the direction of movement. Generally, decreases in FP rate are more financially valuable than increase in TP rate. The three symmetric ROC curves are for illustrative purposes. The green line in Figure 4 illustrates the isocost line if the cost to deliver protons falls by 15%.

Figure 4Isocost lines for radio sensitivity test for esophagitis. Arrow from the gray line indicate the region where a test can be used to refer patients to protons to reduce overall costs for the treatment of esophagitis in cases when a proton plan shows reduced dose to the esophagus. Arrows from the green short-dashed line indicate the test requirements for the same result if the cost to deliver protons falls by 15%. Symmetric ROC curves (blue) with AUC of 0.9, 0.8, 0.7 are added to guide the eye.

Discussion

We created a practical, multi-parametric cost-benefit model, exploring the potential cost benefits of PBT relative to two types of X-ray therapy (3D conformal, IMRT). This model is unique in that it allows the radiation oncologist to estimate the total treatment costs while comparing different treatment modalities. Our model incorporates patient-specific factors (age, anatomy) and treatment-specific factors (chemo-type, radiation modality, radiation dosimetric parameters) to help estimate the costs. The dosimetric input parameters are clinically useful for radiation oncologists as they perform comparative planning of proton and photon plans. Essentially, one could use this type of model to present the toxicity estimates and the overall treatment costs to patients for more tailored discussions regarding outcomes and costs prior to deciding a treatment course. On the public health level, such a model can be used to help anticipate future resource needs or savings.

Our rate-adjusted analysis results demonstrate that compared to photons, protons are not likely to produce cost-savings based on previously published rates of toxicities for patients receiving concurrent chemo-radiation for LA-NSCLC. Although protons were found to reduce toxicities compared to photons, this was not cost-beneficial as the rates of the higher grade toxicities are relatively low. Larger toxicity reductions with protons are seen with grade 1–3 toxicities for both pneumonitis and esophagitis; however, these toxicities are not as expensive to manage compared to grade 4 toxicities, adding little to overall costs. This suggests that patient populations at highest risk of severe acute toxicity may see greater cost benefits using protons. This was the motivation for computing costs per RPA risk group.

Although protons were not cost equivalent with photons for any of the risk groups, the high risk group had the largest cost recovery using protons. The two intermediate risk groups did not benefit as much, with smaller cost recoveries. The low risk groups had the smallest cost differences compared to the baseline. If one considers using protons for LA-NSCLC in future trials or off-trial, this high risk group likely would experience the largest benefit from toxicity reduction and would result in a smaller cost difference.

The toxicity-adjusted analysis results raise interesting points. It seems that there are situations where protons could produce overall cost savings, analyzing on a cost per toxicity grade basis. For example, an IMRT patient with a grade 4 esophagitis had higher overall costs compared to a proton patient with a grade 0–3 esophagitis. Although a clinically reliable tool does not yet exist to predict patient toxicities, we were able to incorporate the use of such a tool in our decision model and define the sensitivity and specificity requirements for this tool to be useful from a cost perspective.

There are several limitations of our model/analyses to address. Toxicity was assessed at a 6-month time point, representing acute toxicity only. Long-term toxicity can be debilitating for LA-NSCLC patients, including pulmonary fibrosis, cardiac events, and esophageal dysfunction, which we did not model. Additionally, the only toxicities that we examined are pneumonitis and esophagitis, as they are the major dose limiting radiation toxicities. There are limited data to support the possibility that protons reduce the rate of both non-hematologic and hematologic toxicities in patients with lung cancer, but these toxicities were not modeled in this study (36).

Another limitation was the selection of utility. We used direct financial costs only. Non-medical costs associated with toxicities were not considered. For example, grade 4 pneumonitis comes at an increased cost to the patient in terms of hospitalization, needing ventilator support, time lost from work, and further co-morbidities. Although not included in our model, these additional costs may not be negligible. In addition, utilities such as quality of life were not considered even though society may deem them of comparable importance.

We used a third party payer (Medicare) perspective for this study when calculating costs. One limitation when obtaining Medicare cost data is variability between institutions, particularly with inpatient costs. The cost data presented here is an estimate from our own institution and may not be generalizable to other clinics/hospitals. We attempted to minimize cost variability by using inpatient data from the Nationwide Inpatient Sample (NIS), Healthcare Cost and Utilization Project (HCUP), and Agency for Healthcare Research and Quality based on MS-DRG code. This data represents 20% of discharges from all participating community hospitals in the United States (47/50 states). We did compare our institutional Medicare reimbursements with NIS data for hospitalizations, and they were similar.

There is also the possibility that PBT for lung cancer could improve local control rates, although we assumed equal local control rates in our model. The phase II proton trial by Chang et al. (31) demonstrated low rates of isolated local failures (9.1%) with a median follow-up of 19.7 months. This compares favorably to their long-term IMRT results with a 2-year local-regional recurrence rate of 43%. If these encouraging local control rates persist with longer follow-up, we could incorporate them into our model, potentially improving the cost-benefits of protons. This could also reflect the use of protons to dose escalate to >70 Gy, as the prescribed dose in that study was 74 Gy (RBE). Similar to the use of IMRT, PBT can be used to reduce complications for the same rate of local control or the proton dose distribution can be utilized to increase tumor dose (and hopefully local control) while seeking to achieve the same rate of complications. Given that previous photon studies used doses ranging from 60–66 Gy, we elected to base our model on a safe dose of 60 Gy in 30 fractions for both modalities.

Although Decision Trees and Markov models are commonly used for cost and cost-benefit analysis, influence diagrams are gaining in popularity and have been shown to be more computationally efficient than decision trees (18). In particular, decision trees can grow to be very large since each state is represented several times on the various branches of the tree. Any decision tree can be represented as an influence diagram, and for large decision trees provide a much more compact and understandable representations. In this model we used an influence diagram since it allowed for the placement of a threshold on the accuracy required for a test in order to improve a decision.

This type of analysis has potential applications both in determining a cohort which would benefit, and in individual decision making. Although radiotherapy has long used CT scans to develop treatment plans which are personalized to a patient’s anatomy, the current interest in cohort selection or personalization focuses on predicting the therapeutic response to radiation, rates of radiosensitivity for normal tissue, and may include patient preferences for health states. These efforts require contributions from many fields such as genomics (4), radiomics (37,38), machine learning (39-41), and shared decision making (42,43). One of the most commonly used metrics to evaluate a medical test is the area-under-the-curve (AUC).

We find that the AUC alone is not sufficient to determine the clinical usefulness of a test. This is primarily due to the fact that maximizing the area under the ROC places equal weights on false-positive and false-negative results (44,45). By providing the opportunity to model the trade-offs between sensitivity and specificity, the influence diagram was shown to be instrumental in expanding the possibilities for exploring different scenarios. Such a tool can be very helpful to both researchers and clinicians.

Another important consideration is that the overall cost of definitive concurrent chemoradiotherapy for locally advanced NSCLC is likely to change considerably with the publication of the PACIFIC trial (46). This trial demonstrated an improvement in progression-free survival with the addition of 1-year of anti-PD-1 immunotherapy with durvalumab when compared to placebo. As such, this is now a standard-of-care recommendation and included in the NCCN guidelines. With the inclusion of one-year of adjuvant immunotherapy, costing approximately $12,000 per month, for a total annual cost of $144,000 per year, the cost of chemoradiotherapy regardless of modality employed (protons, IMRT, 3DCRT) will likely comprise a small portion of the overall treatment.

Conclusions

We developed a simple cost model using an influence diagram in order to demonstrate that protons offer the possibility to actually lower costs for a subset of patients, by accounting only for the reduction in acute toxicities. The cost reduction for such a group of patients would increase when long term outcomes are taken into account. We then determined sensitivity and specificity requirements for a test for radiosensitivity that would enable practitioners to identify this group of patients. These test requirements go beyond simply maximizing the AUC, and show that, with the assumption of a realistic cost reduction in delivering proton therapy, a clinically useful radiosensitivity test may be achievable.

This result demonstrates that a shift in the methodology by which these tests are developed is possible. Typically researchers develop tools to predict outcomes while maximizing a metric such as the AUC. We show that a decision model can set goals for research tools by defining the accuracy requirements for a test before these tests are developed in order to guide research to develop tools that are likely to be adopted clinically.

Acknowledgements

None.

Footnote

Conflicts of Interest: The authors have no conflicts of interest to declare.

References

1. Sethi RV, Shih HA, Yeap BY, et al. Second nonocular tumors among survivors of retinoblastoma treated with contemporary photon and proton radiotherapy. Cancer 2014;120:126-33. [Crossref] [PubMed]
2. Chang JY, Zhang X, Wang X, et al. Significant reduction of normal tissue dose by proton radiotherapy compared with three-dimensional conformal or intensity-modulated radiation therapy in Stage I or Stage III non-small-cell lung cancer. Int J Radiat Oncol Biol Phys 2006;65:1087-96. [Crossref] [PubMed]
3. Chang JY, Jabbour SK, De Ruysscher D, et al. Consensus Statement on Proton Therapy in Early-Stage and Locally Advanced Non-Small Cell Lung Cancer. Int J Radiat Oncol Biol Phys 2016;95:505-16. [Crossref] [PubMed]
4. Barnett GC, Kerns SL, Noble DJ, et al. Incorporating genetic biomarkers into predictive models of normal tissue toxicity. Clin Oncol (R Coll Radiol) 2015;27:579-87. [Crossref] [PubMed]
5. Briggs AH, Claxton K, Sculpher MJ. Decision modelling for health economic evaluation. Oxford: Oxford University Press, 2006.
6. Hunink MM, Weinstein MC, Wittenberg E, et al. Decision making in health and medicine: integrating evidence and values. Cambridge University Press, 2014.
7. Siebert U. When should decision-analytic modeling be used in the economic evaluation of health care? Eur J Health Econ 2003;4:143-50. [Crossref]
8. Peeters A, Grutters JP, Pijls-Johannesma M, et al. How costly is particle therapy? Cost analysis of external beam radiotherapy with carbon-ions, protons and photons. Radiother Oncol 2010;95:45-53. [Crossref] [PubMed]
9. Smith WP, Kim M, Holdsworth C, et al. Personalized treatment planning with a model of radiation therapy outcomes for use in multiobjective optimization of IMRT plans for prostate cancer. Radiat Oncol 2016;11:38. [Crossref] [PubMed]
10. Claxton K, Sculpher M, Drummond M. A rational framework for decision making by the National Institute for Clinical Excellence (NICE). Lancet 2002;360:711-5. [Crossref] [PubMed]
11. Brennan A, Kharroubi S, O’Hagan A, et al. Calculating partial expected value of perfect information via Monte Carlo sampling algorithms. Med Decis Making 2007;27:448-70. [Crossref] [PubMed]
12. Groot Koerkamp B, Myriam Hunink M, Stijnen T, et al. Identifying key parameters in cost-effectiveness analysis using value of information: a comparison of methods. Health Econ 2006;15:383-92. [Crossref] [PubMed]
13. Willan AR, Eckermann S. Optimal clinical trial design using value of information methods with imperfect implementation. Health Econ 2010;19:549-61. [PubMed]
14. Richard P, Phillips M, Smith W, et al. Cost-effectiveness analysis of intensity modulated radiation therapy versus 3dimensional conformal radiation therapy for preoperative treatment of extremity soft tissue sarcomas. Int J Radiat Oncol Biol Phys 2016;95:999-1008. [Crossref] [PubMed]
15. Lester-Coll NH, Dosoretz AP, James BY. Decision analysis of stereotactic radiation surgery versus stereotactic radiation surgery and whole-brain radiation therapy for 1 to 3 brain metastases. Int J Radiat Oncol Biol Phys 2014;89:563-8. [Crossref] [PubMed]
16. Kjaerulff UB, Madsen AL. Bayesian networks and influence diagrams. New York: Springer, 2008.
17. Nease RF Jr, Owens DK. Use of influence diagrams to structure medical decisions. Med Decis Making 1997;17:263-75. [Crossref] [PubMed]
18. Arias M, D’ıez F, et al. Cost-effectiveness analysis with influence diagrams. Methods Inf Med 2015;54:353-8. [Crossref] [PubMed]
19. Meyer J, Phillips MH, Cho PS, et al. Application of influence diagrams to prostate intensity-modulated radiation therapy plan selection. Phys Med Biol 2004;49:1637-53. [Crossref] [PubMed]
20. Mathes T, Walgenbach M, Antoine SL, et al. Methods for systematic reviews of health economic evaluations: a systematic review, comparison, and synthesis of method literature. Med Decis Making 2014;34:826-40. [Crossref] [PubMed]
21. Grutters JP, Pijls-Johannesma M, De Ruysscher D, et al. The cost-effectiveness of particle therapy in nonsmall cell lung cancer: exploring decision uncertainty and areas for future research. Cancer Treat Rev 2010;36:468-76. [Crossref] [PubMed]
22. Grutters JP, Kessels AG, Pijls-Johannesma M, et al. Comparison of the effectiveness of radiotherapy with photons, protons and carbon-ions for non-small cell lung cancer: a meta-analysis. Radiother Oncol 2010;95:32-40. [Crossref] [PubMed]
23. Lievens Y, Verhaeghe N, De Neve W, et al. Proton radiotherapy for locally-advanced non-small cell lung cancer, a cost-effective alternative to photon radiotherapy in Belgium? J Thorac Oncol 2013;8:S839-40.
24. Richard P, Phillips M, Zeng J, et al. Developing a Cost Effectiveness Model for Comparing Radiation Modalities for Stage III NSCLC;. Presented at: Particle Therapy Co-Operative Group of North America 1st Annual Meeting; October 27-29, 2014; Houston, TX.
25. Grutters JP, Abrams KR, De Ruysscher D, et al. When to wait for more evidence? Real options analysis in proton therapy. Oncologist 2011;16:1752-61. [Crossref] [PubMed]
26. Palma DA, Senan S, Oberije C, et al. Predicting esophagitis after chemoradiation therapy for non-small cell lung cancer: an individual patient data meta-analysis. Int J Radiat Oncol Biol Phys 2013;87:690-6. [Crossref] [PubMed]
27. Palma DA, Senan S, Tsujino K, et al. Predicting radiation pneumonitis after chemoradiation therapy for lung cancer: an international individual patient data meta-analysis. Int J Radiat Oncol Biol Phys 2013;85:444-50. [Crossref] [PubMed]
28. Shah A, Hahn SM, Stetson RL, et al. Cost-effectiveness of stereotactic body radiation therapy versus surgical resection for stage I non-small cell lung cancer. Cancer 2013;119:3123-32. [Crossref] [PubMed]
29. American Medical Association. Code Manager: CPT Code/RVU Online Search. Available online: https://ocm.ama-assn.org/OCM/CPTRelativeValueSearch.do
30. HCUP Nationwide Inpatient Sample (NIS). Healthcare Cost and Utilization Project (HCUP); 2011. Agency for Healthcare Research and Quality, Rockville, MD. Available online: http://hcupnet.ahrq.gov/
31. Chang JY, Komaki R, Lu C, et al. Phase 2 study of high-dose proton therapy with concurrent chemotherapy for unresectable stage III nonsmall cell lung cancer. Cancer 2011;117:4707-13. [Crossref] [PubMed]
32. Curran WJ Jr, Paulus R, Langer CJ, et al. Sequential vs concurrent chemoradiation for stage III non-small cell lung cancer: randomized Phase III trial RTOG 9410. J Natl Cancer Inst 2011;103:1452-60. [Crossref] [PubMed]
33. Vokes EE, Herndon JE, Kelley MJ, et al. Induction chemotherapy followed by chemoradiotherapy compared with chemoradiotherapy alone for regionally advanced unresectable stage III non-small-cell lung cancer: Cancer and Leukemia Group B. J Clin Oncol 2007;25:1698-704. [Crossref] [PubMed]
34. Jiang ZQ, Yang K, Komaki R, et al. Longterm clinical outcome of intensity-modulated radiotherapy for inoperable non-small cell lung cancer: the MD Anderson experience. Int J Radiat Oncol Biol Phys 2012;83:332-9. [Crossref] [PubMed]
35. Chun SG, Hu C, Choy H, Komaki RU, et al. Impact of intensity-modulated radiation therapy technique for locally advanced non-small-cell lung cancer: A secondary analysis of the NRG oncology RTOG 0617 randomized clinical trial. J Clin Oncol 2017;35:56-62. [Crossref] [PubMed]
36. Komaki R, Wei X, Allen P, et al. Protons compared to x-imrt compared to 3d in locally advanced NSCLC. Radiother Oncol 2011;99:S89-90. [Crossref]
37. Cunliffe A, Armato SG, Castillo R, et al. Lung texture in serial thoracic computed tomography scans: correlation of radiomics-based features with radiation therapy dose and radiation pneumonitis development. Int J Radiat Oncol Biol Phys 2015;91:1048-56. [Crossref] [PubMed]
38. Niedzielski JS, Yang J, Stingo F, et al. A Novel Methodology using CT Imaging Biomarkers to Quantify Radiation Sensitivity in the Esophagus with Application to Clinical Trials. Sci Rep 2017;7:6034. [Crossref] [PubMed]
39. Kang J, Schwartz R, Flickinger J, et al. Machine learning approaches for predicting radiation therapy outcomes: a clinician’s perspective. Int J Radiat Oncol Biol Phys 2015;93:1127-35. [Crossref] [PubMed]
40. Valdes G, Solberg TD, Heskel M, et al. Using machine learning to predict radiation pneumonitis in patients with stage I non-small cell lung cancer treated with stereotactic body radiation therapy. Phys Med Biol 2016;61:6105. [Crossref] [PubMed]
41. Jochems A, Deist TM, El Naqa I, et al. Developing and validating a survival prediction model for NSCLC patients through distributed learning across 3 countries. Int J Radiat Oncol Biol Phys 2017;99:344-52. [Crossref] [PubMed]
42. Tramontano AC, Schrag DL, Malin JK, et al. Catalog and comparison of societal preferences (utilities) for lung cancer health states: results from the Cancer Care Outcomes Research and Surveillance (CanCORS) study. Med Decis Making 2015;35:371-87. [Crossref] [PubMed]
43. Sloan JA, Halyard M, El Naqa I, et al. Lessons from large-scale collection of patient-reported outcomes: Implications for Big data aggregation and analytics. Int J Radiat Oncol Biol Phys 2016;95:922-9. [Crossref] [PubMed]
44. Hand DJ. Measuring classifier performance: a coherent alternative to the area under the ROC curve. Mach Learn 2009;77:103-23. [Crossref]
45. Phillips MH, Smith WP, Parvathaneni U, et al. Role of positron emission tomography in the treatment of occult disease in head-and-neck cancer: a modeling approach. Int J Radiat Oncol Biol Phys 2011;79:1089-95. [Crossref] [PubMed]
46. Antonia SJ, Villegas A, Daniel D, et al. Durvalumab after chemoradiotherapy in stage III non-small-cell lung cancer. N Engl J Med 2017;377:1919-29. [Crossref] [PubMed]