Answer/comment to Prof. Tammemagi’s article “Selecting lung cancer screenees using risk prediction models—where do we go from here”

You have recently published a review article by Prof. Tammemagi (1). We appreciate the opinions stated and constructive criticism of the article to our recently published work (2). However, we kindly request to accept our brief response and clarifications to the criticism.

• The terminology “inflated AUC” carries negative connotations. The HUNT Lung Cancer Model’s AUC is accurately estimated on the population under study, and is higher than comparable models (Table 1). This is indeed a different population than other models, it is a general population age >20, median 48.3 years (Table 1). The prediction problem becomes indeed easier when including younger individuals of lower risk, but also more difficult when including the senior population (age >74 years), in our cohort 18.41% were >74 at baseline. In general, AUCs between models on different populations are not directly comparable. However, the HUNT, due to its full age span, including light smokers, the 16-year follow-up time and the registration of all lung cancers in that period of time, is to our knowledge the only model that is predictive in the whole range of ages and smoke exposures. Moreover, we note that the HUNT model also compares favourably against the NLST criteria, detecting 49% more lung cancers than the NLST for the same number of people screened, showing a clear superiority independent of the AUC value (2).
• Indeed, annual CT screening of younger individuals, may not be advisable. However, in our validation population of 45,341 ever-smokers in Norway, among those who got lung cancer, 21.35% were <55 at baseline, a group effectively excluded by the NLST and PLCO age criteria. Therefore, high-risk persons of younger age could be examined in the future, but probably with less frequent CT screening or with non-invasive techniques (e.g., using blood biomarkers on which several labs including our own currently work on). Thus, we believe that models that apply on younger populations could be clinically useful.
• By including daily cough as a predictor, the AUC improves but not in an artificial way that is implied by the term “inflated”. The question we used was “Do you cough daily during periods of the year?”, not “Do you cough every day now?” where a positive answer would elicit prompt medical examination. History of daily cough was the only among the 10 clinical symptoms included in our analysis that was not filtered out. In our validation population of ever-smokers, 18.3% of those who did not develop lung cancer answered “yes” versus 34.1% in those who developed lung cancer (P<10⁻⁴), showing its strong, independent predictive value and how common it is in smokers. If one should omit this variable, with the same logic, one should omit BMI. Low BMI is a risk factor, but at the same time it can be the first symptom of lung cancer presenting with weight loss. The BMI is still used in several risk prediction models, including the PLCO_m2012 (5). We would argue that the history of daily cough in periods is a very similar type of measure with that of chronic obstructive pulmonary disease (COPD) and history of chest X-ray (both in the PLCO_m2012) because they reflect a history of pulmonary/chest symptoms. However, history of COPD has some problems of bias, and in a risk model, you need correct information for all variables. If one of the variables is wrong, the risk score may be falsely low or high. In the case of COPD, misdiagnosis is very common (under- and overdiagnosis can be five-fold more common than correct diagnosis) (6). That means that “history of COPD” may introduce an uncontrolled bias potentially with serious consequences. In contrast, virtually everybody can answer correct on the daily cough question, and therefore it seems a more robust variable. We believe that including any type of information that increases the predictive accuracy of a model is desirable, but it should be simple and accurate. Surprisingly, a recent study shows that a reduction in lung cancer mortality from CT is greatest in those with normal lung function and those with undiagnosed COPD (7). This seems to be due to the competing risk of death from other smoke-related disorders such as respiratory failure or cardiovascular disease in those with severe COPD. These authors suggest to assess respiratory flow in potential screenees with high risk and exclude those with severe COPD, regardless of age and smoking status.

Table 1 Key differences of the PLCO_m2012 and the HUNT Lung Cancer Model, two externally validated risk prediction models developed in prospective population-based cohorts
Full table

Finally, here are a few advantages of the HUNT model that were under-represented, in our opinion.

• The need to include a younger population and light smokers in lung cancer studies and corresponding models becomes apparent when considering that NLST used selection criteria fail to include an estimated of three quarters of people who went on to develop lung cancer (8). Actually 64% of our population that developed lung cancer had smoked <30 pack-years at base-line and would not be included by the PLCO or NLST (Table 1). Hence we believe the HUNT model is applicable to a more general population of ever-smokers.
• The features that entered the HUNT model were decided by first manually pre-selecting 36 out of 199 clinical variables, based on prior knowledge, and then being reduced to 7 by the data analysis algorithms (age, pack-years, smoking intensity, years since smoking cessation, body mass index, daily cough, and hours of daily indoors exposure to smoke). The filtered-out features are the ones that do not provide additional predictive information given (controlling for, conditioned on) the selected features. It is informative to note that typical features included in models such as the education level were filtered out using this procedure and two novel factors were included (periodical daily cough and hours of daily indoors exposure to smoke).
• We have developed a simple- to-use online calculator that not only calculates the 6-year but also the 16-year risk, unlike other risk calculators. It is freely available for use and ready to be implemented in any prospective screening study (9).

Acknowledgements

None.

Footnote

Conflicts of Interest: The author has no conflicts of interest to declare.

References

1. Tammemagi MC. Selecting lung cancer screenees using risk prediction models-where do we go from here. Transl Lung Cancer Res 2018;7:243-53. [Crossref] [PubMed]
2. Markaki M, Tsamardinos I, Langhammer A, et al. A Validated Clinical Risk Prediction Model for Lung Cancer in Smokers of All Ages and Exposure Types: A HUNT Study. EBioMedicine 2018;31:36-46. [Crossref] [PubMed]
3. Weber M, Yap S, Goldsbury D, et al. Identifying high risk individuals for targeted lung cancer screening: Independent validation of the PLCOm2012 risk prediction tool. Int J Cancer 2017;141:242-53. [Crossref] [PubMed]
4. Tammemagi MC, Katki HA, Hocking WG, et al. Selection criteria for lung-cancer screening. N Engl J Med 2013;368:728-36. [Crossref] [PubMed]
5. Tammemagi MC, Schmidt H, Martel S, et al. Participant selection for lung cancer screening by risk modelling (the Pan-Canadian Early Detection of Lung Cancer [PanCan] study): a single-arm, prospective study. Lancet Oncol 2017;18:1523-31. [Crossref] [PubMed]
6. Gershon AS, Thiruchelvam D, Chapman KR, et al. Health Services Burden of Undiagnosed and Overdiagnosed COPD. Chest 2018;153:1336-46. [Crossref] [PubMed]
7. Young RP, Hopkins RJ. Chronic obstructive pulmonary disease (COPD) and lung cancer screening. Transl Lung Cancer Res 2018;7:347-60. [Crossref] [PubMed]
8. Pinsky PF, Berg CD. Applying the National Lung Screening Trial eligibility criteria to the US population: what percent of the population and of incident lung cancers would be covered? J Med Screen 2012;19:154-6. [Crossref] [PubMed]
9. Available online: http://mensxmachina.org/en/hunt-ntnu-lung-cancer-risk-calculator/.Markaki M, Tsamardinos I, Langhammer A, et al. 2018.