Rates of positive lung cancer screening examinations in academic versus community practice

Introduction

Based on National Lung Screening Trial (NLST) results, the US Preventive Services Task Force (USPSTF) recommends lung cancer screening with low-dose computed tomography (LDCT) for current and former heavy smokers ages 55–80 years (1,2). The NLST protocol defined a positive screening finding as a noncalcified nodule measuring 4mm or greater in the longest diameter (3). Using this threshold, false-positive rates in the NLST were 26.6% at baseline and 21.8% after baseline. Concern over these high false-positive rates observed in the NLST led to development of the Lung CT Screening Reporting and Data System (Lung-RADS) by the American College of Radiology (4).

Lung-RADS is currently in use for nodules detected on LDCT for lung cancer screening and was designed to standardize interpretation and reporting by radiologists, provide consistent management recommendations, and facilitate outcome monitoring (4). A retrospective analysis of the NLST data that applied Lung-RADS criteria found a reduction in false-positive rates to 12.8% at baseline and 5.3% after baseline (5); however, there is limited prospective data on the use of Lung-RADS assessment categories in real world clinical settings. Hence, we sought to compare use of Lung-RADS assessments in a real-world screening population versus the retrospective application to the NLST, and to determine if Lung-RADS use differs in academic versus community settings.

Methods

As part of the North Carolina Lung Screening Registry (NCLSR) we conducted a prospective observational study of lung cancer screening data from four lung cancer screening sites in North Carolina. The NCLSR is funded by the National Cancer Institute to collect patient, radiologist and outcomes data on lung cancer screening to evaluate the adoption of this cancer screening recommendation in population-based settings. We included all NCLSR patients who underwent LDCT for lung cancer screening from 11/2014 to 12/2018.

Using electronic medical record systems data and abstracted radiology text reports, we ascertained patient characteristics and Lung-RADS assessment categories for each screening exam. Patient characteristics included age, race, sex, and smoking status (current versus former). We dichotomized the radiologist reported Lung-RADS assessment into negative [Lung-RADS 1 (Negative) and Lung-RADS 2 (Benign Appearance or Behavior)] and positive [Lung-RADS 3 (Probably Benign) or 4A, 4B, or 4X (Suspicious)]. We dichotomized into these groups to be comparable to the retrospective analysis of the NLST (5). We classified the NCLSR screening sites as academic (n=2) or community (n=2).

We compared the Lung-RADS distribution for baseline and subsequent screening exams in the NCLSR with those in the retrospective analysis of the NLST (5) using chi-square tests. To adjust for potential differences in patient characteristics across screening sites, we employed a logistic regression to compare the likelihood of positive versus negative screening results in academic versus community sites. We stratified the analysis by baseline versus subsequent screen, and included the following covariates: age, race, sex, and smoking status. Because some patients had more than one subsequent screening exam, we controlled for patient-level correlations by including an R-side compound-symmetric random effect. This study was approved by the University of North Carolina Institutional Review Board (No. 17-2352).

Results

We included 4,037 lung cancer screening exams conducted in 2,861 NCLSR patients. The study population undergoing screening was 51.3% ages less than 65 years, although those screened at academic versus community sites were more likely to be ages 65 and older (59.0% vs. 45.2%, respectively, P value <0.01; Table 1). Among those with a known race (n=2,632), 80.6% were white, 16.9% were black, and 2.5% were another race. Overall, 8.0% of patients had missing race and these patients were from community sites. Compared with community sites, patients screened at academic sites were more likely to be black (25.7% vs. 13.5%, P value <0.01). The study population included 46.9% females and 53.4% current smokers, with no differences seen in academic versus community imaging sites.

Table 1 Characteristics of patients who underwent LDCT lung cancer screening in the North Carolina Lung Screening Registry (NCLSR) in academic and community sites
Full table

The distribution of Lung-RADS in the NLST, the NCLSR academic sites, and the NCLSR community sites, stratified by baseline and subsequent lung cancer screening examinations is shown in Table 2. On baseline screening exams, the proportion of positive LDCT exams was statistically significantly higher in NCLSR community sites (17.7%) versus in NCLSR academic sites (11.4%; P value <0.01) and in the NLST (13.6%; P value <0.01). Among subsequent screening exams, the proportion of positive exams was significantly lower in the NLST (5.9%) compared with the NLSR at academic (11.6%; P=0.01) or community (12.7%; P value <0.01) sites. Among NCLSR patients screened in subsequent rounds, the proportion with a positive LDCT result was similar in community and academic settings (P value =0.61). Even after adjusting for patient age, race, sex and smoking status, baseline lung cancer screening exams in NCLSR academic settings remained less likely to have a positive Lung-RADS [P value <0.01; adjusted odds ratio (aOR) =0.66, 95% confidence interval (95% CI): 0.51–0.86] compared to exams in community settings. There was no difference in the likelihood of a positive Lung-RADS assessment for subsequent screening exams in academic versus community settings after adjustment for patient characteristics (P value =0.68; aOR =0.91, 95% CI: 0.58–1.43).

Table 2 Comparison of lung-RADS assessment on baseline and subsequent lung cancer screening examinations in the National Lung Screening Trial and the North Carolina Lung Screening Registry by Site Type and Screening Round
Full table

Discussion

We found that on baseline lung cancer screening exams, positive rates in academic centers were similar to those in the NLST while rates in community sites were higher. On subsequent screening exams, the positive rate at academic sites did not decrease and while the positive rate did decrease for community sites, it did not decrease to levels observed in the NLST. The NLST was conducted primarily in urban, tertiary care hospitals with strict attention to image acquisition, regulation of radiologic interpretation conducted primarily by thoracic radiologists, and detailed follow-up protocols for nodules. These have led to concerns about the generalizability of the NLST results to lung cancer screening in community settings (1,6-8). Our results show the proportion of positive lung cancer screening examinations varied by screening round (baseline vs. subsequent) and imaging center type (academic vs. community), even after adjusting for patient characteristics.

The rates of positive lung cancer screening exams we observed (11.4–17.7% at baseline and 11.6–12.7% on subsequent exams) are similar to those of a 2017 study of lung cancer screening at Cancer Research Network Sites, which reported 10.7% to 21.5% of LDCT exams were positive, although this study did not examine baseline and subsequent screens separately (9). Data from 1,181 baseline lung cancer screening exams conducted as part of an academic screening program reported a positive rate of 12.7%, which is similar to our rate of 11.4% (10). Another study reported the rate of positive lung cancer screening exams among 500 baseline exams in community practice as 24.6%, which is higher than rates in our study (11). Reasons for differences between studies may include the underlying lung cancer risk in the study populations, or the type of radiologist. Our finding of a sustained increase in the proportion of positive screening exams on subsequent scans has implications for screening programs that will conduct annual screening beyond the 3-years analyzed in the NLST.

Our finding of a higher proportion of positive lung cancer screening exams in community versus academic settings at baseline but not on subsequent examinations may be due to radiologists’ training, the availability of prior images for comparison, or clinical experience. In our study, screening exams conducted at academic sites were interpreted by cardiothoracic trained radiologists while those performed at community sites were more likely to be interpreted by general radiologists. The availability of prior images for comparison likely reduced the proportion of patients with a positive LDCT exam (i.e., those who were recalled before the next annual screen) in community sites on subsequent screenings yet we did not see this finding for academic sites. Recall rates in breast cancer screening are significantly lower in mammograms with versus without a comparison mammogram (6.9% vs. 14.9%) (12). Radiologists’ clinical experience could also play a role in the rate of positive screening exams, similar to that observed in mammography interpretation (13).

The rate of positive lung cancer screening exams in the original NLST study (pre-Lung-RADS criteria) was 26.6% at baseline and 21.8% after baseline. Studies evaluating the net benefit and overall cost effectiveness of lung cancer screening have used the pre-Lung-RADS false positive rates but these rates are likely over estimates. In our study the rates of positive screening examinations from over 4,000 screening exams ranged from 12.3% to 16.0%. Subsequent work will evaluate false positive and false negative rates to help inform the benefit to harm ratio of lung cancer screening in real-world setting. Future research is needed to evaluate the impact of imaging site as well as radiologist training and experience on screening outcomes.

Acknowledgments

Funding: This work was supported by the National Institutes of Health under National Cancer Institute (grant R01 CA212014). The research presented in this paper is that of the authors and does not reflect the official policy of the NIH.

Footnote

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/tlcr-19-673). LMH, TB and RHV report grants from NIH/NCI during the conduct of the study. MPR reports grants from NIH/NCI during the conduct of the study; Consultant for a Research Workshop held by Johnson & Johnson; Chair, Medical and Scientific Advisory Board for bioAffinity Technologies; and Member, Medical and Scientific Advisory Board for Biodesix. LB has no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). This study was approved by the University of North Carolina Institutional Review Board (No. 17-2352). Informed consent was taken from all individual participants.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. National Lung Screening Trial Research Team, Aberle DR, Adams AM, et al. Reduced lung-cancer mortality with low-dose computed tomographic screening. N Engl J Med 2011;365:395-409. [Crossref] [PubMed]
2. Moyer VA. U.S. Preventive Services Task Force. Screening for Lung cancer: U.S. Preventive Services Task Force recommendation statement. Ann Intern Med 2014;160:330-8. [PubMed]
3. National Lung Screening Trial Research Team, Aberle DR, Berg CD, et al. The National Lung Screening Trial: overview and study design. Radiology 2011;258:243-53. [Crossref] [PubMed]
4. American College of Radiology. Lung CT Screening Reporting and Data System (Lung-RADS). Available online: https://www.acr.org/Clinical-Resources/Reporting-and-Data-Systems/Lung-Rads, March 1, 2019.
5. Pinsky PF, Gierada DS, Black W, et al. Performance of Lung-RADS in the National Lung Screening Trial: A retrospective assessment. Ann Intern Med 2015;162:485-91. [Crossref] [PubMed]
6. Rai A, Doria-Rose VP, Silvestri GA, et al. Evaluating Lung Cancer Screening Uptake, Outcomes, and Costs in the United States: Challenges with Existing Data and Recommendations for Improvement. J Natl Cancer Inst 2019;111:342-9. [Crossref] [PubMed]
7. American Academy of Family Physicians. Summary of Recommendations for Clinical Preventive Services. Leawood, KS: American Academy of Family Physicians; 2017. Available online: https://www.aafp.org/dam/AAFP/documents/patient_care/clinical. Accessed Dec 1, 2019.
8. Howard DH, Richards TB, Bach PB, et al. Comorbidities, smoking ststus, and life expectancy among individuals eligible for lung cancer screening. Cancer 2015;121:4341-7. [Crossref] [PubMed]
9. Gould MK, Sakoda LC, Ritzwoller DP, et al. Monitoring Lung Cancer Screening Use and Outcomes at Four Cancer Research Network Sites. Ann Am Thorac Soc 2017;14:1827-35. [Crossref] [PubMed]
10. Kaminetzky M, Milch HS, Shmukler A, et al. Effectiveness of Lung-RADS in Reducing False-Positive Results in a Diverse, Underserved, Urban Lung Cancer Screening Cohort. J Am Coll Radiol 2019;16:419-426. [Crossref] [PubMed]
11. Pasquinelli MM, Kovitz KL, Koshy M, et al. Outcomes from a Minority-Based Lung Cancer Screening Program vs the National Lung Screening Trial. JAMA Oncol 2018;4:1291-3. [Crossref] [PubMed]
12. Yankaskas BC, May RC, Matuszewski J, et al. Effect of Observing Change from Comparison Mammograms on Perfomance of Screening Mammography in a Large Community-based Population. Radiology 2011;261:762-70. [Crossref] [PubMed]
13. Miglioretti DL, Gard CC, Carney PA, et al. When radiologists perform best: the learning curve in screening mammography interpretation. Radiology 2009;253:632-40. [Crossref] [PubMed]