Ensemble architectures and fusion techniques for convolutional neural networks applied to medical image analysis

Abstract

Computer vision algorithms such as convolutional neural networks are used to automate processes in medicine and support diagnosis. These algorithms minimize human error during medical image analysis and reduces inter-operator variability. In this study, to support the diagnosis, three strategies involving fusion of convolutional neural networks were proposed. First, ensemble architectures were used in the gastrointestinal image classification task. Second, through the fusion of convolutional models, a new model was proposed to detect landmarks in images of lateral cephalograms, hand X-rays and lung X-rays. The third analysis tested whether image preprocessing would help convolutional models in the task of landmark detection and region segmentation. The proposed strategies were evaluated based on common metrics in the literature such as mean radial error and F1-score. In addition, aligning with the concepts of green computing, resource consumption and pollutant emissions were also evaluated. For the classification task, the proposed ensemble achieved an F1-score of 0.910, matching the literature, however, using lower cost equipment. For landmark detection, through model fusion, considering the success detection rate (SDR) between the predicted landmarks and the original landmarks, we achieved SDR of 95.72% for the lateral cephalogram and 99.56% for the hand x-rays, both considering a distance up to 4mm. For lung x-rays, we obtained an SDR 84.21% considering 6 pixels of distance. Our proposal also reduced execution time, energy consumption and carbon emissions by around 65%. The preprocessing strategy showed no with significant improvements over the results.