Response to Dr. Røe’s comment letter

We agree with Dr. Røe that if we could accurately identify high-risk younger individuals or lighter smokers, that we would consider them for lung cancer screening. This would have to come after adequate demonstration that the benefits outweigh harms and that such screening would be cost-effective, and accomplishing these tasks will not be easy.

Just adding data on younger individuals and lighter smokers to data on higher-risk individuals for modelling does not demonstrate that the model can accurately identify substantial numbers of high-risk younger individuals or lighter-smokers in a meaningful way. Including low-risk younger individuals and lighter smokers in model development data will make the model appear to have better overall prediction with higher AUC.

In the Markaki et al. paper there was no demonstration that the HUNT model was highly predictive in young individuals or light smokers and that important numbers of high-risk candidates for lung cancer screening could be identified. Their paper attempted to demonstrate HUNT model superiority over the NLST criteria, but this is a low bar and was carried out in a fashion that was not straightforward.

An honest comparison of the performance of HUNT versus NLST criteria in the CONOR validation sample can be obtained by assessing the accuracy statistics for the HUNT criteria in the original Markaki et al. paper Table 4 versus NLST criteria in Table 5, in which each criterion are applied to the entire CONOR sample of 46,387.

HUNT criteria (Table 4): sensitivity 81.85%, specificity 78.31%, positive predictive value (PPV) 2.21%. NLST criteria (Table 5): sensitivity 25.6%, specificity 95.5%, PPV 3.3%. (The inconsistent use of decimals within and between tables is in the original Markaki et al. paper.) In this side-by-side comparison, the HUNT criteria sensitivity is superior and the NLST criteria specificity and PPV are superior. Differences in part may be explained by calibration to distinct populations.

The number that was Hunt criteria positive in CONOR validation data was 10,000 and the number of NLST positive was 2,081. Thus, if the HUNT and NLST criteria were applied to a Scandinavian population, the HUNT criteria would select for screening 10,000/2,081 = 4.8 times as many individuals as would be by NLST criteria, and it is expected that a large proportion of them would be at too low a risk to qualify for screening by currently described selection thresholds which range from 1.5% to 2% 6-year or 5% 5-year risks (1-3). The current annual incidence of lung cancer in men and women in individuals less than 55 years of age in China, Japan, Poland, Spain, Sweden, the Netherlands, the United Kingdom and the USA are well below 0.1% (4). Even the Fleischner Guidelines for incidental nodules does not recommending working up an incidental nodule that has less than one percent probability of being lung cancer (5). Screening low-risk individuals will increase harms versus benefits and reduce cost-effectiveness (6).

Dr. Røe suggests that “high-risk persons of younger age could be examined in the future, but probably with less frequent CT screening or with non-invasive techniques”. “Thus, we believe that models that apply on younger populations could be clinically useful.” According to Dr. Røe’s reasoning and model design, all ever-smokers ages 20 to 90 years could be HUNT-model assessed for eligibility for screening. In the CONOR validation data the HUNT criteria had a 97.8% false positive (FP) proportion overall. When the HUNT criteria are applied to a younger population the FP numbers will be very high because the incidence of disease is rare in this population. Applying the sensitivity and specificity for the HUNT model reported in Table 4 to the incidence rates of lung cancer in the 20 to 40-year population, the expected PPV is well below 1%. Fewer than 1 in 100 HUNT-positive individuals will be diagnosed with lung cancer. The harms and costs of screening such a low-risk population are expected to be great. If the HUNT model were able to identify younger age individuals that are at truly higher risk, is there evidence that offering them less frequent CT screening is effective, as is suggested by Dr. Røe?

The Markaki et al.’s paper and Dr. Røe’s comment letter appear to contain a few errors.

The Markaki et al.’s paper Table 4 title states “Of the 45,117 ever smokers, 1,986 were picked by the NLST criteria.” Table 5 states “The number of people in CONOR (validation population) fulfilling NLST criteria was 2,081;” The reason for inconsistent (1,986 vs. 2,081) numbers is unclear.

The true positive, false negative, false positive and true negative proportions reported in Tables 4 and 5 are incorrect. The TP and FN proportions are based on cases and FP and TN proportions are based on non-cases (7). They are not conditional on test result.

According to Dr. Røe’s comment letter, the PLCOm2012 model includes “previous X-ray”. The PLCOm2012 model does not include “previous X-ray”, which was included in an older published model (8) that preceded the PLCOm2012.

Dr. Røe writes: “Actually 64% of our population that developed lung cancer had smoked <30 pack-years at base-line and would not be included by the PLCO or NLST (Table 1).” This is incorrect. The PLCOm2012 is not limited to smokers of ≥30 pack-years; it includes smokers >0 pack-years, just as the HUNT model does.

We make these comments so as to avoid misunderstanding and false expectations.

Acknowledgements

None.

Footnote

Conflicts of Interest: The author has no conflicts of interest to declare.

References

1. Tammemagi MC, Church TR, Hocking WG, et al. Evaluation of the Lung Cancer Risks at Which to Screen Ever- and Never-Smokers: Screening Rules Applied to the PLCO and NLST Cohorts. PLoS Med 2014;11:e1001764. [Crossref] [PubMed]
2. Tammemagi M, Miller B, Yurcan M, et al. Determination of the Eligibility Criteria for Cancer Care Ontario’s Lung Cancer Screening Pilot for People at High Risk (Abstract #9432). American Thoracic Society International Conference; May 18-23, 2018; San Diego, 21 May 2018.
3. Field JK, Duffy SW, Baldwin DR, et al. UK Lung Cancer RCT Pilot Screening Trial: baseline findings from the screening arm provide evidence for the potential implementation of lung cancer screening. Thorax 2016;71:161-70. [Crossref] [PubMed]
4. Malhotra J, Boffetta P, Mucci L. Cancer of the Lung, Larynx, and Pleura. In: Adami HO, Hunter D, Lagiou P, et al. editors. Textbook of Cancer Epidemiology, 3rd Edition. New York, USA: Oxford University Press, 2018.
5. MacMahon H, Naidich DP, Goo JM, et al. Guidelines for Management of Incidental Pulmonary Nodules Detected on CT Images: From the Fleischner Society 2017. Radiology 2017;284:228-43. [Crossref] [PubMed]
6. Kovalchik SA, Tammemagi M, Berg CD, et al. Targeting of low-dose CT screening according to the risk of lung-cancer death. N Engl J Med 2013;369:245-54. [Crossref] [PubMed]
7. Porta MS, Greenland S, Hernán M, et al. A dictionary of epidemiology. Six edition ed. Oxford: Oxford University Press, 2014.
8. Tammemagi CM, Pinsky PF, Caporaso NE, et al. Lung cancer risk prediction: prostate, lung, colorectal and ovarian cancer screening trial models and validation. J Natl Cancer Inst 2011;103:1058-68. [Crossref] [PubMed]