Evaluation of artificial intelligence in the detection and characterization of pulmonary nodules

Objective To evaluate the detection and qualitative diagnostic efficacy of artificial intelligence (AI) in pulmonary nodules.

Method A retrospective study method was used to select 355 patients (205 females and 150 males, aged (55.1±12.2) years old) from the lung nodule case database of Hebei Petro China Central Hospital from 2020 to 2021 through simple random sampling. Lung CT images were imported into the AI system. The diagnostic results of AI were compared with those of three junior professional physicians. Two intermediate professional physicians reviewed the CT images in accordance with the double-blind principle, and the consistent opinions of two intermediate professional physicians were used as reference standards for the diagnosis of true nodules. The sensitivities of AI and junior professional physicians in the detection of pulmonary nodules were also compared. A total of 105 patients underwent preoperative-CT guided puncture histopathological examination or postoperative histopathological examination after lung tissue resection. The histopathological examination results were used as the "gold standard" to compare the sensitivity, specificity, positive predictive value, negative predictive value, and accuracy of AI and the deputy chief physician in the qualitative diagnosis of pulmonary nodules. Intergroup comparison of counting data was conducted using chi-square or Fisher's exact probability test.

Results A total of 1 072 true nodules were detected in the CT images of 355 patients. Among these nodules, 1 063 were detected by AI, with a sensitivity of 99.16% (1 063/1 072), and 9 were missed. A total of 1 009 true nodules were detected by the junior professional physicians, with a sensitivity of 94.12% (1 009/1 072), and 63 were missed. In terms of pulmonary nodule detection, AI exhibited a significantly higher sensitivity than the junior professional physicians, and the difference was statistically significant (χ²=41.907, P<0.05). Meanwhile, 105 patients were confirmed to have 88 malignant nodules and 17 benign nodules via histopathological examination. A total of 86 cases were true positive, 15 were false positive, 2 were true negative, and 2 were false negative in the qualitative diagnosis of pulmonary nodules using AI. The deputy chief physician identified 83 true positive cases, 1 false positive case, 16 true negative cases, and 5 false negative cases in the qualitative diagnosis of pulmonary nodules. The specificity and positive predictive value of the qualitative diagnosis of pulmonary nodules by the deputy chief physician were significantly higher than those of AI (94.12% (16/17) vs. 11.76% (2/17) and Fisher's exact probability test, P<0.05; 98.81% (83/84) vs. 85.15% (86/101)), χ²=9.172, P<0.05).The deputy chief physician attained a lower sensitivity than AI in terms of the in qualitative diagnosis of pulmonary nodules, but no statistically significant difference was observed (94.32% (83/88) vs. 97.73% (86/88), χ²=0.595, P>0.05). A higher negative predictive value was detected in the qualitative diagnosis of pulmonary nodules by the deputy chief physician compared with that of the AI. However, the difference was not statistically significant (76.19% (16/21) vs. 50.00% (2/4), Fisher's exact probability test, P>0.05). Overall, the deputy chief physician attained a higher accuracy in the qualitative diagnosis of pulmonary nodules compared with the AI (94.29% (99/105) vs. 83.81% (88/105)), χ²=8.796, P<0.05).

Conclusions AI shows a high sensitivity in the detection and qualitative diagnosis of pulmonary nodules. However, its specificity, positive predictive value, negative predictive value, and accuracy are low. In clinical work, physicians can use the good sensitivity of AI to improve work efficiency, but it cannot replace manual analysis results as the standard for the qualitative diagnosis of pulmonary nodules.