In this study, Ardila et al. aimed to build an end-to-end approach that performs both lesion localization and lung cancer risk categorization based on the input CT data, in order to simulate a complete workflow of radiologists for lung cancer CT screening.
Keywords: Graph Convolutional Networks, Multi-Label Chest X-ray, image classification, Graph Representation
0. Motivation, Objective and Related Works:
Motivation:
With an estimated 160,000 deaths in 2018, lung cancer is the most common cause of cancer death in the United States.
Lung cancer screening using low-dose computed tomography has been shown to reduce mortality by 20–43% and is now included in US screening guidelines 1–6.
Existing challenges include inter-grader variability, and high false-positive and false-negative rates.
Objectives:
Propose a deep learning algorithm:
+ Uses a patient’s current and prior computed tomography volumes to predict the risk of lung cancer.
+ Achieves a state-of-the-art performance (94.4% area under the curve) on 6,716 National Lung Cancer Screening Trial cases, and performs similarly on an independent clinical validation set of 1,139 cases.
Perform two-part retrospective reader study:
1) When prior computed tomography imaging was not available, our model outperformed all six radiologists with absolute reductions of 11% in false positives and 5% in false negatives.
2) Where prior computed tomography imaging was available, the model performance was on-par with the same radiologists.
1) Pipeline:
Methods:
Mask R-CNN (Segmentation): Compute the center of its bounding box and to determine an alignment with the prior volume.
Modified RetinaNet (3D and No FPN) (Detection): Propose the most nodule-like regions. Initialized by first training on LIDC and then trained on radiologist-annotated lesion bounding boxes collected on the NLST dataset.
3D inflated InceptionV1 (Full-volume model): Predict cancer probability and then used the last layer before the fnal probability, take 1,024 numbers as the output for this model, and use them as features later on.
3D Inception (Cancer risk prediction model): In all cases, 3D Inception is used to extract features. Throughout the model components, our approach is to classify and extract features from 3D volumes, heavily based on this 3D Inception model. After the top two candidate ROIs were passed to the second-stage malignancy classification model. It was trained as a single convolutional neural network with shared parameters across all detected ROIs. Each ROI was passed through this network to predict its individual malignancy score. The final prediction was generated by combining the two probability scores
The workflow included a full assessment of CT volume, region-of-interest detection, comparison to prior imaging, and correlation with biopsy-derived results (Fig 1).
The “end-to-end” design is particularly important when considering AI application in lung cancer because the human workflow consists of multiple steps, and these steps can interact with each other, which adds further complexity to the process.
Replicating the entire workflow using the AI model, although challenging, has the potential to provide more robust solutions to improve efficiency and performance of lung cancer CT screening, rather than providing separate tools for individual steps.
Fig. 1. Overall Modeling Framework
For each patient, the model uses a primary low-dose CT (LDCT) volume and, if available, a prior LDCT volume as input. The model then analyzes suspicious and volumetric ROIs as well as the whole-LDCT volume and outputs an overall malignancy prediction for the case, a risk bucket score (LUMAS), and localization for predicted cancerous nodules.
Using the state-of-the-art AI approaches, the authors built a three-dimensional (3D) CNN model for the whole-CT volume analyses using screening CT.1 Then, a model for CNN region-of-interest (ROI) detection was trained to detect 3D cancer candidate regions in the CT volume. Finally, a CNN cancer risk-prediction model was developed, which operated on the outputs from the other two models (Fig 2).
The risk prediction model can also incorporate regions of interest from the prior scans by assessing the regions corresponding to the cancer candidate regions on the current scan.
Fig. 2. Illustration of the Architecture of the End-to-End Cancer Risk Prediction Model
The model is trained to encompass the entire CT volume and automatically produce a score predicting the cancer diagnosis. In all cases, the input volume is first resampled into two different fixed voxel sizes as shown. Two ROI detections are used per input volume, from which features are extracted to arrive at per-ROI prediction scores via a fully connected neural network. The prior ROI is padded to all zeros when a prior is not available.
Furthermore, the two-part retrospective reader study was performed to compare the model performance with that of six U.S. board-certified radiologists. In the first part, each of the six radiologists and the model evaluated a subset of the test dataset consisting of 507 participants (83 of whom with cancer) from NLST, without access to previous CT scans. The patient demographics and clinical history were given to the radiologists but not to the model. The performance of all six radiologists trended at or below the model’s AUC of 95.9%. In the second part, the radiologists and the model evaluated current and previous year CT scans and interpreted 308 CT volumes from the cohort used in the first part of the comparison. Of note, these 308 scans were not from the initial baseline screening CT; rather, each scan had the previous year CT that was presented as a comparison. The model’s AUC was 92.6%, which was similar to the radiologists’ performance. Interestingly, both the model and the radiologists performed worse in this second part of the comparison compared with the first. Presumably this was because obviously malignant cases were diagnosed at the first screening and were not included in the cohort for the second part of the study, leaving more challenging evaluations for the second part. One of the goals of AI implementation is to reduce the workload for radiologists regarding straightforward evaluations, allowing them to spend increased time on complex cases. The results of the comparison indicate the need for further work to achieve this goal.
Results:
Our model achieves a state-of-the-art performance (94.4% area under the curve) on 6,716 National Lung Cancer Screening Trial cases, and performs similarly on an independent clinical validation set of 1,139 cases.
The model performed similarly with an AUC of 95.5% on an independent clinical validation set of 1,139 cases, demonstrating a step toward automated image evaluation for lung cancer risk estimation using AI.
Conclusion:
In conclusion, applying AI approaches to medicine has opened a number of new avenues for optimizing lung cancer detection, diagnosis, and prognostication. Along with the expanding applications of the AI models in various aspects of lung cancer, the attempts to understand the black box nature of the AI models are becoming particularly important so that physicians can also learn from the AI models that have been effectively trained—meaning that they have been created with sufficient volume and breadth of data—and can proceed to develop further sophisticated models to address truly challenging issues in lung cancer practice. ✦
References:
https://www.iaslc.org/iaslc-news/ilcn/artificial-intelligence-lung-cancer-opening-new-avenues-detection-diagnosis-and
