Deep learning-based classification of pleural malignancy using medical thoracoscopic images

Introduction

Malignant pleural effusion (MPE) is a common and often challenging complication affecting a substantial proportion of patients with advanced malignancies. According to the 2020 Global Cancer Statistics, the most common cause of MPE is lung cancer, followed by breast cancer and lymphoma. Lung cancer constitutes 11.4% of newly diagnosed cancers and is responsible for 18% of all cancer-related deaths globally, indicating its high clinical burden (1,2). Although substantial progress has been made in lung cancer therapy, particularly through targeted treatments and immunotherapies, rapid and accurate diagnosis of MPE is crucial for improving patient outcomes.

Obtaining a definitive diagnosis of MPE often involves pleural fluid cytology. However, cytological examination alone has a modest sensitivity of about 60%, which may be insufficient for guiding optimal therapy. In contrast, medical thoracoscopy (MT), an endoscopic procedure allowing direct visualization of the parietal pleura, has a diagnostic yield exceeding 90%, primarily due to the ability to perform targeted biopsies (3). Furthermore, with recent advancements in targeted therapies and immunotherapies as cancer treatments, the need for a wide range of molecular and immunological tests has increased, making it even more critical to obtain a sufficient amount of accurate tissue samples. However, slides need to be prepared, and the results need to be confirmed by a pathologist, leading to significant delays in diagnosis. This hinders prompt decision-making regarding additional procedures, such as pleurodesis in the procedure room.

Beyond the need for rapid diagnosis, there is another fundamental challenge: the heterogeneous appearance of malignant pleural lesions. While some cases present with obvious tumor nodules, others show more subtle findings such as pleural thickening, neovascularization, or diffuse infiltration, making it difficult to identify the most suspicious areas for biopsy. In situations where distinct nodules are absent, the diagnostic yield can decrease unless the physician has substantial experience in identifying early or atypical malignant changes.

In recent years, deep learning has emerged as an innovative technology in medical imaging. Numerous studies have applied deep learning models to radiological images and digital pathology slides, achieving high diagnostic accuracies in detecting lung cancer, classifying tumor subtypes, and even predicting treatment responses (4-6). Endoscopic applications of deep learning have also gained traction, especially in gastroenterology, where artificial intelligence (AI)-assisted image analysis has been used for real-time detection of colorectal polyps or early gastric cancer (7,8). Within pulmonary medicine, most AI research has focused on radiology [e.g., nodule detection via computed tomography (CT)] or endobronchial ultrasound (EBUS) images for mediastinal lymph node assessment (9,10). Yet, the utility of AI specifically for medical thoracoscopic images remains unexplored.

In this study, we developed a deep learning model to classify pleural diseases into malignant and benign categories. We present this article in accordance with the TRIPOD reporting checklist (available at https://tlcr.amegroups.com/article/view/10.21037/tlcr-2025-588/rc).

Methods

Study population and data collection

In total, 676 patients from a retrospective cohort who underwent MT at Incheon St. Mary’s Hospital from May 2015 to July 2024 were included in this study. Among them, patients with final pathological diagnoses of non-lung cancer primary metastases other than primary lung cancer or mesothelioma, and those without usable thoracoscopic image data, were excluded. Thoracoscopic images were excluded if they contained prominent surgical instruments, post-biopsy bleeding or alteration, dominant non-pleural structures such as lung, diaphragm, effusion, or significant motion blur. All images used for model training and validation were individually reviewed by experienced clinicians. Only those images that clearly displayed features strongly suggestive of malignant lesions were selected. As a result, 4,932 thoracoscopic images were utilized for training, including 2,093 benign and 2,839 malignant images obtained from 426 patients out of the remaining 540. The classification of benign and malignant conditions was based on the final pathological diagnosis, which was made 5–7 days after the procedure, with tuberculosis and other benign conditions categorized as benign. Clinical data, including age, sex, body mass index (BMI), smoking history, and comorbidities, were collected during admission. The study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. The study was approved by the institutional ethics board of Catholic Medical Center (No. XC24RIDI065) and individual consent for this retrospective analysis was waived.

Statistical analysis

Baseline characteristics of the patients were analyzed using descriptive statistics. Continuous variables were expressed as mean ± standard deviation and compared between groups using the Student’s t-test. Categorical variables were summarized as counts and percentages and compared using the Chi-square test.

Preparation of mediastinoscopy images

Data preprocessing

Prior to model training, preprocessing was performed to address the issue of unnecessary enlargement of endoscopic images caused by metadata, such as dates and patient information. Multiple preprocessing steps were applied to simultaneously achieve data anonymization and image standardization. First, each image was converted from blue, green, red (BGR) to hue, saturation, value (HSV) format, after which a mask was generated by applying a threshold to a specific color range (11). Next, contours were detected using the find Contours function from the OpenCV-Python library, and the bounding Box function was applied to identify a rectangular region tightly encompassing the target object. Then, the image was cropped to exclude areas outside this detected region (Figure 1).

Figure 1 Image preprocessing workflow for thoracoscopic images. HSV, hue, saturation, value.

Finally, the cropped images were uniformly resized to dimensions of [224, 224, 3] to ensure input consistency for the deep learning model.

Data augmentation

Although the volume of data available was sufficient to train a convolutional neural network (CNN), additional data augmentation techniques were applied to further enhance classification performance and mitigate overfitting.

Image data augmentation generally involves artificially increasing the diversity of input images through various geometric transformations. Commonly used augmentation methods include flipping, cropping, rotation, and translation, which have been widely reported to effectively prevent overfitting and improve the generalizability of deep learning models (12).

In this study, these augmentation techniques were implemented in real-time during model training using Keras’s ImageDataGenerator. Specifically, transformations included random rotations within a ±20° range, horizontal and vertical shifts up to 20% of image dimensions, zooming and scaling within a 20% range, and horizontal flipping.

To prevent information leakage, all data augmentation procedures were strictly limited to the training set; no transformations were applied to the validation or test sets. The Final dataset comprised 2,093 malignant and 2,839 benign images. To mitigate potential class imbalance, we applied a class weighting strategy during training. These weights were integrated into the binary cross-entropy loss function to impose greater penalties on the misclassification of underrepresented classes.

Applying these transformations in real-time enhances data diversity without manual preprocessing or data storage overhead, thereby improving model robustness and performance on unseen data.

InceptionV3 training for classification

InceptionV3 was adopted as the base model for transfer learning due to its proven effectiveness in large-scale visual recognition tasks. This model incorporates parallel convolutional filters of varying sizes, enabling simultaneous extraction of fine-grained and coarse features. This architectural design enhances multi-scale feature learning while mitigating the vanishing gradient problem.

A key component of the model is the use of 1×1 convolutional layer in early stages, which significantly reduce the number of parameters without compromising representational capacity (13).

To customize the model, pre-trained weights were loaded. A Global Average Pooling layer was added to compress feature maps, followed by Dropout for regularization. A dense layer with 256 units was used to extract nonlinear features, and a final dense layer with sigmoid activation was implemented for binary classification.

The model was optimized using the Adam optimizer. The loss function used was Binary Cross-Entropy, defined as:

BCELoss(y⌣,y)=−(y×logy⌣+(1−y)×log(1−y⌣))[1]

where y⌣ denotes the continuous output of the sigmoid function ranging between 0 and 1, and y represents the discrete ground truth label.

Performance evaluation

The performance of deep learning algorithms is evaluated using various metrics, with the confusion matrix being one of the most commonly used tools in classification tasks. This matrix is a visual representation of the performance of a classification algorithm trained through supervised learning. It is structured as a 2×2 table comprising four elements. The first element is true positive (TP), which refers to instances where the model correctly predicts the positive class. The second is false negative (FN), representing cases where the model incorrectly predicts the negative class for an actual positive instance. The third is false positive (FP), which occurs when the model incorrectly predicts the positive class for an actual negative instance. Finally, true negative (TN) refers to cases where the model correctly predicts the negative class (14).

The receiver operating characteristic (ROC) curve is another tool for evaluating the performance of a binary classifier. It plots the TP rate (TPR) against the FP rate (FPR), with the X-axis representing the FPR and the Y-axis representing the TPR. The further the curve is from the diagonal (baseline), the better the model’s performance. The area under the ROC curve (AUC) represents the area between the ROC curve and the diagonal line. A larger AUC indicates higher prediction accuracy. In this study, both the confusion matrix and the ROC-AUC curve were used to validate the model’s performance.

In addition to these quantitative metrics, gradient-weighted class activation mapping (Grad-CAM) was utilized as a qualitative tool to visually interpret the model’s predictions. Grad-CAM highlights image regions that are most influential for a given class prediction by leveraging gradient information from the last convolutional layers of the network, providing insight into whether the model’s focus aligns with clinically relevant features (15).

To evaluate the model’s generalization capability and ensure the statistical robustness of its performance metrics, 5-fold cross-validation was performed on the training dataset. The evaluation metrics—accuracy, precision, and AUC—were reported as mean values across the folds, along with their corresponding standard errors (SEs) and 95% confidence intervals (CIs).

Datasets and hyperparameters

The dataset was partitioned into training, validation, and test sets, with respective proportions of 80%, 10%, and 10%. The key hyperparameters employed during model training were as follows: a learning rate of 1×10⁻⁴, with 200 epochs and a batch size of 128. Additionally, early stopping was implemented, where training was halted if the validation loss did not improve for 10 consecutive epochs.

Results

Demographic and clinical characteristics

In total, 426 patients were analyzed in the study. The mean age was 69.5 years, and 278 patients (66.2%) were male. The average BMI was 23.4 kg/m². Regarding smoking history, 186 patients (44.9%) had never smoked, 166 (40.1%) were ex-smokers, and 62 (15.0%) were current smokers. In terms of diagnosis, 175 patients (41.1%) had benign conditions, while 54 patients (12.7%) were diagnosed with granuloma (tuberculosis). Malignancies were found in 197 patients (46.2%), among which non-small-cell lung cancer (NSCLC) with adenocarcinoma (ADC) accounted for the majority (157 cases, 79.7%). NSCLC with squamous cell carcinoma (SqCC) and NSCLC not otherwise specified (NOS) were diagnosed in 16 (8.1%) and 13 patients (6.6%), respectively. Additionally, mesothelioma was diagnosed in six patients (3%) (Table 1).

Diagnostic performance of the InceptionV3 model

Table 2 presents the performance of the proposed model. The model resulted in 240 TPs, 37 FNs, 51 FPs, and 154 TNs (Figure 2), corresponding to an accuracy of 0.818 (95% CI: 0.784–0.853), precision of 0.826 (95% CI: 0.780–0.869), recall of 0.867 (95% CI: 0.826–0.905), and F1 score of 0.846 (95% CI: 0.812–0.876). Figure 3 illustrates the model’s ROC curve, which had an AUC of 0.901 (95% CI: 0.875–0.927). Figure 4 presents Grad-CAM visualizations, which highlight the regions of the input image that the model focused on when making predictions.

Figure 2 Confusion matrix of the deep learning model for pleural malignancy classification.

Figure 3 ROC curve of model performance. AUC, area under the curve; ROC, receiver operating characteristic.

Figure 4 Grad-CAM visualization of thoracoscopic image classifications of the deep learning model. This figure illustrates representative examples of each classification outcome: true negative (A), false negative (B), false positive (C), and true positive (D). (A) Presents a benign case that was correctly classified (true label =0, predicted label =0), whereas (B) shows a malignant case that was incorrectly predicted as benign (true label =1, predicted label =0). (C) Depicts a benign case misclassified as malignant (true label =0, predicted label =1), while (D) demonstrates a malignant case correctly identified by the model (true label =1, predicted label =1). The Grad-CAM heatmaps highlight the image regions that most strongly influenced the model’s predictions, thereby offering insights into the model’s decision-making process and its potential failure modes. Grad-CAM, gradient-weighted class activation mapping.

To assess the model’s generalization ability and the statistical reliability of these metrics, 5-fold cross-validation was performed on the training set. The average performance across the folds was as follows: accuracy, 0.845 (SE 0.0085; 95% CI: ±0.0167); precision, 0.851 (SE 0.0164; 95% CI: ±0.0321); and AUC, 0.921 (SE 0.0041; 95% CI: ±0.0079). These results demonstrate consistent and robust performance across different data partitions.

Discussion

We developed and validated a deep learning model (InceptionV3) using medical thoracoscopic images to distinguish between benign and malignant pleural lesions. It demonstrated high performance, with an AUC of 0.90, and effectively detected malignant cases with a recall of 86%. Our results suggest that integrating AI into routine MT could markedly enhance diagnostic efficiency.

Previous studies have used deep learning models for lung cancer detection and classification, primarily using radiological or pathological slides (4,16). Although these approaches have shown impressive accuracy, they usually depend on imaging modalities such as CT that do not allow real-time decision-making in the procedure room. Also, recent studies have used endoscopic images to detect gastrointestinal cancers and to classify malignant lymph nodes using EBUS (7,8,10,17-19). However, research on deep learning applications specifically for MT images remains scarce.

In the context of pleural diseases, many studies have focused on diagnostic performance using CT (20,21). Although CT can provide comprehensive views of pleural thickening or fluid, it does not offer the direct visualization that thoracoscopy provides. Additionally, a recent Japanese study applied a deep learning model to videos from video-assisted thoracoscopic surgery (VATS) to predict visceral pleural invasion in stage I lung ADC, achieving an AUC of 0.77 (22). However, there has been little research on developing models to classify parietal pleural lesions using medical thoracoscopic images. In this study, we used MT images of the parietal pleura to classify pleural lesions as benign or malignant. The significantly higher AUC (0.94) in our study indicates the feasibility of real-time or near-real-time classification, allowing for immediate clinical decisions.

MT is considered one of the most definitive methods for diagnosing pleural diseases, as it achieves a significantly higher diagnostic yield, often exceeding 90%, than liquid-based cytology (3,23-25). It also allows direct visual assessment of the pleura, enabling targeted biopsy of suspicious areas. This is particularly important when pleural malignancy presents with subtle findings such as diffuse pleural thickening or neovascularization rather than overt nodularity. This study complements the standard diagnostic tools by indicating which areas of the pleura are most likely to be malignant. If feasible hardware and workflow solutions for real-time diagnosis can be established, the proposed model may support real-time decisions about where to take biopsies in situations where immediate pathological confirmation is unavailable. Furthermore, when findings during the procedure strongly suggest malignancy, clinicians can proceed with pleurodesis or obtain additional samples for molecular analyses, thereby making both diagnosis and treatment possible. This approach has the potential to reduce unnecessary biopsies, shorten procedure time, and improve resource efficiency, offering both clinical and economic benefits.

This study had some limitations. Most importantly, the training dataset included only patients with primary lung cancer or mesothelioma. Consequently, the model’s performance in cases of metastatic pleural disease from other primary cancers remains unclear. Another limitation was that this was a single-center study without external validation, which limits the generalizability of our findings. Although the encouraging performance suggests good generalizability, large, prospective, multi-institutional studies are needed to validate our results. Furthermore, while our binary classification (malignant vs. benign) proved highly accurate, subclassifying specific cancer subtypes (e.g., ADC vs. SqCC) or identifying tuberculosis-related granulomas was not considered.

Future research could address these limitations by using larger, multicenter validation datasets to improve generalizability. Further work could develop real-time devices for use during MT. Additionally, efforts could be made to expand the model to classify malignancy subtypes or diagnose other benign pathologies, including tuberculosis. We are actively pursuing follow-up studies on this topic to present results in the near future.

Conclusions

We successfully developed a deep learning-based model for classifying pleural malignancies from MT images, achieving excellent performance. To our knowledge, this was the first study to apply AI to pleural disease classification using thoracoscopic images. Our results could aid the development of real-time applications that improve diagnostic accuracy, guide biopsy site selection, and reduce procedure time.

Acknowledgments

None.

Footnote

Reporting Checklist: The authors have completed the TRIPOD reporting checklist. Available at https://tlcr.amegroups.com/article/view/10.21037/tlcr-2025-588/rc

Data Sharing Statement: Available at https://tlcr.amegroups.com/article/view/10.21037/tlcr-2025-588/dss

Peer Review File: Available at https://tlcr.amegroups.com/article/view/10.21037/tlcr-2025-588/prf

Funding: This research was supported by a Grant of Translational R&D Project through Institute for Bio-Medical convergence, Incheon St. Mary’s Hospital, The Catholic University of Korea; and this study was conducted as part of the National Balanced Development Special Account K-Health National Medical AI Service and Industrial Ecosystem Construction Project funded by the Ministry of Science and ICT and the Korea Information and Communications Promotion Agency (grant No. H0503-24-1001).

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://tlcr.amegroups.com/article/view/10.21037/tlcr-2025-588/coif). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. The study was approved by the institutional ethics board of Catholic Medical Center (No. XC24RIDI065) and individual consent for this retrospective analysis was waived.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Sung H, Ferlay J, Siegel RL, et al. Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries. CA Cancer J Clin 2021;71:209-49. [Crossref] [PubMed]
2. Jany B, Welte T. Pleural Effusion in Adults-Etiology, Diagnosis, and Treatment. Dtsch Arztebl Int 2019;116:377-86. [Crossref] [PubMed]
3. Kassirian S, Hinton SN, Cuninghame S, et al. Diagnostic sensitivity of pleural fluid cytology in malignant pleural effusions: systematic review and meta-analysis. Thorax 2023;78:32-40. [Crossref] [PubMed]
4. Li Z, Zhang J, Tan T, et al. Deep Learning Methods for Lung Cancer Segmentation in Whole-Slide Histopathology Images-The ACDC@LungHP Challenge 2019. IEEE J Biomed Health Inform 2021;25:429-40. [Crossref] [PubMed]
5. Chaunzwa TL, Hosny A, Xu Y, et al. Deep learning classification of lung cancer histology using CT images. Sci Rep 2021;11:5471. [Crossref] [PubMed]
6. Xu Y, Hosny A, Zeleznik R, et al. Deep Learning Predicts Lung Cancer Treatment Response from Serial Medical Imaging. Clin Cancer Res 2019;25:3266-75. [Crossref] [PubMed]
7. Dong Z, Wu L, Mu G, et al. A deep learning-based system for real-time image reporting during esophagogastroduodenoscopy: a multicenter study. Endoscopy 2022;54:771-7. [Crossref] [PubMed]
8. Podlasek J, Heesch M, Podlasek R, et al. Real-time deep learning-based colorectal polyp localization on clinical video footage achievable with a wide array of hardware configurations. Endosc Int Open 2021;9:E741-8. [Crossref] [PubMed]
9. Quanyang W, Yao H, Sicong W, et al. Artificial intelligence in lung cancer screening: Detection, classification, prediction, and prognosis. Cancer Med 2024;13:e7140. [Crossref] [PubMed]
10. Yong SH, Lee SH, Oh SI, et al. Malignant thoracic lymph node classification with deep convolutional neural networks on real-time endobronchial ultrasound (EBUS) images. Transl Lung Cancer Res 2022;11:14-23. [Crossref] [PubMed]
11. Sural S, Qian G, Pramanik S, editors. Segmentation and histogram generation using the HSV color space for image retrieval. Proceedings International Conference on Image Processing; 2002: IEEE. Available online: https://ieeexplore.ieee.org/abstract/document/1040019
12. Shorten C, Khoshgoftaar TM. A survey on image data augmentation for deep learning. J Big Data 2019;6:60. [Crossref] [PubMed]
13. Xu L, Teoh SS, Ibrahim H. A deep learning approach for electric motor fault diagnosis based on modified InceptionV3. Sci Rep 2024;14:12344. [Crossref] [PubMed]
14. Wu MT. Confusion matrix and minimum cross-entropy metrics based motion recognition system in the classroom. Sci Rep 2022;12:3095. [Crossref] [PubMed]
15. Selvaraju RR, Cogswell M, Das A, et al. Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization. Int J Comput Vis 2020;128:336-59.
16. Zhong Y, Cai C, Chen T, et al. PET/CT based cross-modal deep learning signature to predict occult nodal metastasis in lung cancer. Nat Commun 2023;14:7513. [Crossref] [PubMed]
17. Du Y, Yang R, Chen Z, et al. A deep learning network-assisted bladder tumour recognition under cystoscopy based on Caffe deep learning framework and EasyDL platform. Int J Med Robot 2021;17:1-8. [Crossref] [PubMed]
18. Liu D, Peng X, Liu X, et al. A real-time system using deep learning to detect and track ureteral orifices during urinary endoscopy. Comput Biol Med 2021;128:104104. [Crossref] [PubMed]
19. Kitajima K, Matsuo H, Kono A, et al. Deep learning with deep convolutional neural network using FDG-PET/CT for malignant pleural mesothelioma diagnosis. Oncotarget 2021;12:1187-96. [Crossref] [PubMed]
20. Kröner PT, Engels MM, Glicksberg BS, et al. Artificial intelligence in gastroenterology: A state-of-the-art review. World J Gastroenterol 2021;27:6794-824. [Crossref] [PubMed]
21. Ozcelik N, Ozcelik AE, Guner Zirih NM, et al. Deep learning for diagnosis of malign pleural effusion on computed tomography images. Clinics (Sao Paulo) 2023;78:100210. [Crossref] [PubMed]
22. Shimada Y, Ojima T, Takaoka Y, et al. Prediction of visceral pleural invasion of clinical stage I lung adenocarcinoma using thoracoscopic images and deep learning. Surg Today 2024;54:540-50. [Crossref] [PubMed]
23. Chang CH, Ost DE. Malignant pleural disease: a pragmatic guide to diagnosis. Curr Opin Pulm Med 2022;28:282-7. [Crossref] [PubMed]
24. Gayen S. Malignant Pleural Effusion: Presentation, Diagnosis, and Management. Am J Med 2022;135:1188-92. [Crossref] [PubMed]
25. Piazzolla M, De Pace CC, Porcel JM, et al. Local Anesthetic Thoracoscopy: A Focus on Indications, Techniques and Complications. Arch Bronconeumol 2024;60:423-30. [Crossref] [PubMed]