Real-time detection of colon polyps during colonoscopy using YOLOv7
264 viewsDOI:
https://doi.org/10.54939/1859-1043.j.mst.CSCE7.2023.122-134Keywords:
Colorectal cancer; Deep learning; object detection; Polyp detection.Abstract
Deep learning has made brilliant achievements in detecting colonic polyps in colonoscopy videos in recent years. However, the detection of colonic polyps in colonoscopy videos is problematic because of the complex environment of the colon and the various shapes of polyps. Therefore, researchers need to spend a lot of time searching for real-time detection systems with good performance and that are suitable for the current equipment and working environment. This paper aimed to investigate the polyp detection potential of the state-of-the-art deep learning model You Only Look Once version 7. We implemented, trained, and tested the polyp detection model using open public datasets: Kvasir-Seg, CVC-ClinicDB, CVC_ColonDB, and ETIS-LaribPolypDB. Validation of the test set utilizing Recall, Precision, F1 Score, and Average Precision (AP) showed that the model achieved the highest performance on CVC-ClinicDB with 83.3% Recall, 80.6% Precision, 81.9% F1 Score, 75% AP@0.5, 51.8% AP and the mean processing time per frame was 20ms. The automatic polyp detection model exhibited good performance, as evidenced by the high detection sensitivity and rapid processing. This model can help endoscopists improve polyp detection performance during the colonoscopy procedure.
References
[1]. Baxter, Nancy N., et al. "Association between colonoscopy and colorectal cancer mortality in a US cohort according to site of cancer and colonoscopist specialty." Journal of Clinical Oncology 30.21: 2664, (2012). DOI: https://doi.org/10.1200/JCO.2011.40.4772
[2]. Doubeni, Chyke A., et al. "Screening colonoscopy and risk for incident late-stage colorectal cancer diagnosis in average-risk adults: a nested case–control study." Annals of internal medicine 158.5_Part_1: 312-320, (2013).
[3]. Leufkens, A. M., et al. "Factors influencing the miss rate of polyps in a back-to-back colonoscopy study." Endoscopy 44.05: 470-475, (2012). DOI: https://doi.org/10.1055/s-0031-1291666
[4]. D. Jha, P. H. Smedsrud, M. A. Riegler, P. Halvorsen, T. de Lange, D. Johansen, and H. D. Johansen, "Kvasir-SEG: A segmented polyp dataset,'' in Proc. Int. Conf. Multimedia Modeling. Springer, pp. 451-462, (2020). DOI: https://doi.org/10.1007/978-3-030-37734-2_37
[5]. Misawa, Masashi, et al. "Artificial intelligence-assisted polyp detection for colonoscopy: initial experience." Gastroenterology 154.8: 2027-2029, (2018). DOI: https://doi.org/10.1053/j.gastro.2018.04.003
[6]. Wang, Pu, et al. "Development and validation of a deep-learning algorithm for the detection of polyps during colonoscopy." Nature biomedical engineering 2.10: 741-748, (2018). DOI: https://doi.org/10.1038/s41551-018-0301-3
[7]. Urban, Gregor, et al. "Deep learning localizes and identifies polyps in real time with 96% accuracy in screening colonoscopy." Gastroenterology 155.4: 1069-1078, (2018). DOI: https://doi.org/10.1053/j.gastro.2018.06.037
[8]. Redmon, Joseph, et al. "You only look once: Unified, real-time object detection." Proceedings of the IEEE conference on computer vision and pattern recognition, (2016). DOI: https://doi.org/10.1109/CVPR.2016.91
[9]. Tajbakhsh, Nima, Suryakanth R. Gurudu, and Jianming Liang. "Automated polyp detection in colonoscopy videos using shape and context information." IEEE transactions on medical imaging 35.2: 630-644, (2015). DOI: https://doi.org/10.1109/TMI.2015.2487997
[10]. Tajbakhsh, Nima, Suryakanth R. Gurudu, and Jianming Liang. "A classification-enhanced vote accumulation scheme for detecting colonic polyps." Abdominal Imaging. Computation and Clinical Applications: 5th International Workshop. Proceedings 5. Springer Berlin Heidelberg, (2013). DOI: https://doi.org/10.1007/978-3-642-41083-3_7
[11]. Wang, Yi, et al. "Part-based multiderivative edge cross-sectional profiles for polyp detection in colonoscopy." IEEE Journal of Biomedical and Health Informatics 18.4: 1379-1389, (2013). DOI: https://doi.org/10.1109/JBHI.2013.2285230
[12]. Park, Sun Young, and Dusty Sargent. "Colonoscopic polyp detection using convolutional neural networks." Medical Imaging: Computer-Aided Diagnosis. Vol. 9785. SPIE, (2016). DOI: https://doi.org/10.1117/12.2217148
[13]. Tajbakhsh, Nima, Suryakanth R. Gurudu, and Jianming Liang. "Automatic polyp detection in colonoscopy videos using an ensemble of convolutional neural networks." IEEE 12th International Symposium on Biomedical Imaging (ISBI). IEEE, (2015). DOI: https://doi.org/10.1109/ISBI.2015.7163821
[14]. R. Zhang,Y. Zheng, C. C.Y. Poon, D. Shen, and J.Y.W. Lau, “Polyp detection during colonoscopy using a regression-based convolutional neural network with a tracker”, Pattern Recognit., vol. 83, pp. 209-219, (2018). DOI: https://doi.org/10.1016/j.patcog.2018.05.026
[15]. Liu, Ming, Jue Jiang, and Zenan Wang. "Colonic polyp detection in endoscopic videos with single shot detection based deep convolutional neural network." IEEE Access 7: 75058-75066, (2019). DOI: https://doi.org/10.1109/ACCESS.2019.2921027
[16]. Wang, Chien-Yao, Alexey Bochkovskiy, and Hong-Yuan Mark Liao. "YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. (2023). DOI: https://doi.org/10.1109/CVPR52729.2023.00721
[17]. Bernal, Jorge, Javier Sánchez, and Fernando Vilarino. "Towards automatic polyp detection with a polyp appearance model." Pattern Recognition 45.9: 3166-3182, (2012). DOI: https://doi.org/10.1016/j.patcog.2012.03.002
[18]. Bernal, Jorge, et al. "WM-DOVA maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physicians." Computerized medical imaging and graphics 43: 99-111, (2015). DOI: https://doi.org/10.1016/j.compmedimag.2015.02.007
[19]. Silva, Juan, et al. "Toward embedded detection of polyps in wce images for early diagnosis of colorectal cancer." International journal of computer assisted radiology and surgery 9: 283-293, (2014). DOI: https://doi.org/10.1007/s11548-013-0926-3
[20]. Ma, Yiting, et al. "LDPolypVideo benchmark: a large-scale colonoscopy video dataset of diverse polyps." Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, Proceedings, Part V 24. Springer International Publishing, (2021). DOI: https://doi.org/10.1007/978-3-030-87240-3_37
[21]. COCO Detection Challenge (Bounding Box). Available online: https://competitions.codalab.org/competitions/20794.
[22]. Brandao, Patrick, et al. "Towards a computed-aided diagnosis system in colonoscopy: automatic polyp segmentation using convolution neural networks." Journal of Medical Robotics Research 3.02: 1840002, (2018). DOI: https://doi.org/10.1142/S2424905X18400020
[23]. Zheng, Yali, et al. "Localisation of colorectal polyps by convolutional neural network features learnt from white light and narrow band endoscopic images of multiple databases." 40th annual international conference of the IEEE engineering in medicine and biology society (EMBC). IEEE, (2018). DOI: https://doi.org/10.1109/EMBC.2018.8513337
[24]. Tian, Yu, et al. "One-stage five-class polyp detection and classification." IEEE 16th international symposium on biomedical imaging (ISBI 2019). IEEE, (2019). DOI: https://doi.org/10.1109/ISBI.2019.8759521
[25]. Jia, Xiao, et al. "Automatic polyp recognition in colonoscopy images using deep learning and two-stage pyramidal feature prediction." IEEE Transactions on Automation Science and Engineering 17.3: 1570-1584, (2020).
[26]. Qadir, Hemin Ali, et al. "Toward real-time polyp detection using fully CNNs for 2D Gaussian shapes prediction." Medical Image Analysis 68:101897, (2021). DOI: https://doi.org/10.1016/j.media.2020.101897
