Segmentation of Retinal Blood Vessels using an Attention-Gated U-Net (AG-U-Net)

Introduction

The retinal vascular system contains valuable information regarding the eye’s condition [1], and segmenting retinal vessels is a crucial task for analyzing eye fundus images and diagnosing fundus diseases [2, 3]. This process is essential for various advanced applications, including evaluating artery/vein ratios [4], analyzing blood flow [5], assessing image quality [6], registering and synthesizing retinal images [7, 8], and aiding in the early detection of systemic vascular diseases [1]. Consequently, the automatic segmentation of retinal blood vessels from fundus images has emerged as a prominent research focus in the medical imaging domain [1].

Early methods for retinal vessel segmentation were unsupervised and utilized traditional image processing techniques such as mathematical morphology [9, 10] or modified edge detection operations [11]. These approaches aimed to enhance vessel intensities in retinal images through pre-processing and subsequent thresholding for segmentation. Despite ongoing research into advanced filtering methods for retinal vessel segmentation in recent years [12, 13], these techniques have struggled to achieve competitive performance on standard benchmarks. This limitation is likely attributed to their challenges in effectively handling images with pathological structures and adapting to various appearances and resolutions.

On the other hand, artificial intelligence (AI)-based methods, including machine learning (ML) and deep learning (DL), have demonstrated more promising outcomes and superior performance compared to traditional approaches [3]. Liskowski P, et al. [14] introduced a supervised segmentation technique utilizing a deep neural network trained on a large dataset preprocessed with global contrast normalization, zero-phase whitening, and augmented with geometric transformations and gamma corrections. Suryani E, et al. [15] employed a self-organizing graph artificial neural network for blood vessel segmentation, involving preprocessing, segmentation, and performance analysis stages. Wang S, et al. [16] proposed a supervised method based on feature and ensemble learning. Zhou L, et al. [17] suggested a discriminative feature learning approach using a convolutional neural network (CNN) for the dense conditional random field model. Fu H, et al. [18] treated retinal vascular segmentation as a boundary detection task, incorporating multi-scale context information, a side output layer for hierarchical structure learning, and conditional random fields for long-term pixel dependence modeling. Zhou Y, et al. [19] presented an end-to-end synthetic neural network with components like a symmetric equilibrium generative adversarial network, multi-scale features refine blocks, and an attention mechanism to enhance vessel segmentation capabilities. Xiuqin P, et al. [20] proposed a retinal vessel segmentation method based on an improved deep learning U-Net model to address performance degradation issues in residual networks with extreme depth.

To address the challenges of low segmentation accuracy and incomplete segmentation of small vessels, this study introduces an enhanced segmentation model called the attention-gated U-Net. This model incorporates an attention gate to improve segmentation performance. Our methodology integrates adaptive histogram equalization with contrast limitation (CLAHE), median filtering data normalization, and multi-scale morphological transformation to enhance vascular-feature information. Additionally, artifact correction is achieved through adaptive gamma correction. The preprocessed results are then segmented using the attention-gated U-Net (AG-U-Net) model to accurately segment fine vessels.

Dataset

The Retinal blood vessel segmentation dataset [26] is an openly available dataset that includes high-resolution retinal fundus images and ground truth labels for retinal blood vessels. These fundus images were captured using state-of- the-art imaging equipment. Each image is accompanied by detailed pixel-level ground truth annotations that precisely identify the blood vessel locations. The dataset provides corresponding pixel-wise annotations in a binary mask format for each image, where blood vessel pixels are denoted as 1 and background pixels as 0.

For faster training, the images were resized to 224 x 224 pixels and stored in JPEG format during pre-processing. The 80 images were divided into three sets: training, validation, and test, with proportions of 70%, 10%, and 20%, respectively. Data augmentation was excluded to minimise computational costs and avoid introducing extra noise to the initial dataset. Our primary objective was to assess the enhanced AG-U-Net’s performance and contrast it with the baseline U-Net segmentation model. Re-labelling was also sidestepped to prevent bias, considering the lack of external annotators and blinding [27].

Data Pre-Processing

The hue and saturation levels in individual retinal colour images exhibit significant variation. Each basic image must be converted into an intensity image, which is then normalised to achieve zero mean and unit variance. The normalised intensities are subsequently adjusted to fall within the range of 0 to 255. In human vision, a gamma correction algorithm is applied to process the images. The Contrast Limited Adaptive Histogram Equalization (CLAHE) technique is a commonly employed pre-processing method for retinal vessel segmentation. It enhances the quality of available information, thereby improving the performance of segmentation models. The CLAHE 27 technique is utilized to enhance the contrast of retinal images by controlling noise amplification in neighbouring regions and addressing low-intensity contrast issues. By strengthening the contrast of retinal images, the CLAHE algorithm preserves overall brightness and colour balance, aiding in the differentiation of retinal vessels from the background. This enhancement can boost the accuracy of segmentation models, particularly in scenarios where the contrast between vessels and the background is minimal.

Model Training, Testing and Results Extraction

During the development of the AG-U-Net and baseline segmentation models, 80 retinal images and masks were utilized for training. The models were configured with a kernel size of 3x3, a learning rate of 10-5, a batch size of 8, and a dropout rate of 50%. The activation function employed was ReLU, and the loss function used was Dice, as studies have indicated that these choices can enhance accuracy and address pixel imbalance, respectively [28, 29].

Training and validation were conducted using 64 pairs of retinal images and masks, following the initial dataset split. The objective of the model is to minimize the loss function during training, while the validation set aids in fine-tuning the parameters. At the end of each training epoch, the model is saved if its validation score improves, and the training data is shuffled to prevent bias from the presentation order. Subsequently, the segmentation model is evaluated using a test set consisting of 16 pairs of retinal images and masks. Each iteration aims to produce a model that surpasses its predecessor in performance on the test set.

Evaluation Metrics

In vessel segmentation, various evaluation metrics can be utilised to gauge the performance of the segmentation model, including accuracy, F1 score, Jaccard score, recall score, and precision. These metrics rely on true-positives (TPs), true-negatives (TNs), false-positives (FPs), and false- negatives (FNs) derived from comparing the predicted binary vessel map to the ground truth binary vessel map. This comparison helps evaluate the segmentation results for addressing the challenge of binary segmentation of retinal vessels. TPs represent the pixels correctly identified as vessels in both the ground truth and predicted maps, while TNs denote the pixels correctly identified as non-vessels in both maps. FPs indicate pixels identified as vessels in the predicted map but not in the ground truth map, and FNs refer to pixels classified as vessels in the ground truth map but not identified as vessels in the predicted map.

Accuracy refers to the percentage of correctly identified pixels in the predicted binary map, calculated as [(TP + TN)/ (TP + TN + FP + FN)]. Precision represents the proportion of predicted vessels correctly identified by the model, calculated as [TP/ (TP + FP)]. Sensitivity, also known as recall, indicates the fraction of true vessels correctly identified by the model, calculated as [TP/ (TP + FN)]. The F1-score is the harmonic mean of sensitivity and precision, providing a balanced evaluation that considers both false positives and false negatives in its calculation. It is computed as [(2 × {precision × recall})/ (precision + recall)]. The Jaccard score, or IoU (intersection over union), assesses the agreement between the ground truth and predicted binary maps, calculated as [TP/ (TP + FP + FN)]. These evaluation metrics are essential for comparing the performance of different vessel segmentation models, such as AG-U-Net and baseline U-Net, and for optimizing the segmentation algorithm’s parameters to enhance overall performance.

Discussion

The AG-U-Net model for retinal vascular segmentation has shown promising outcomes in accurately recognizing and segmenting vessels in retinal images. This enhanced approach focuses on specific image regions and extracts relevant features for segmentation through the utilization of global and local attention mechanisms. The AG-U-Net model has demonstrated an accuracy of 0.9488, precision of 0.8186, recall of 0.7419, a dice score of 0.7765, and a Jaccard score of 0.6356 in the Retinal blood vessel segmentation dataset. These results indicate that the AG-U-Net model outperforms the traditional U-Net model in terms of segmentation accuracy.

Additionally, a potential future improvement could involve incorporating a more extensive and diverse set of training data. The current training of the network is based on a limited sample of images, which may not fully capture the variability present in retinal scans. By expanding the training dataset to include a broader range of images, the network’s ability to generalise to new data could be enhanced, leading to improved performance on previously unexplored content. Furthermore, enhancing the AG-U- Net model could involve incorporating different modalities beyond conventional grayscale images. For example, integrating channels like fluorescein angiography or optical coherence tomography could provide the network with additional context and information for more accurate vein segmentation. Utilising the TLA network for retinal vascular segmentation has the potential to significantly enhance the precision and effectiveness of retinal vessel analysis in the future. Moreover, a sudden decrease in Intersection over Union (IoU) indicates a lack of a comprehensive ground truth training dataset. Moving forward, efforts can be directed towards leveraging the existing limited training data to enhance the model’s generalisation capabilities. Lastly, improving the transparency of annotators’ qualifications and the consensus process for addressing inter-observer variability in annotation can help mitigate uncertainties and doubts surrounding the accuracy of ground truth labels.

Conclusion

The results demonstrate that the enhanced AG-U- Net segmentation model outperforms the standard U-Net baseline model in accurately segmenting retinal blood vessels, achieving a high accuracy score of 0.9488 on the dataset. This improvement is notably superior to the baseline U-Net model’s performance. The enhanced method excels in segmentation-specific metrics like dice score and IoU, as well as overall performance, showcasing its effectiveness in accurately delineating retinal blood vessels. The approach exhibits adaptability and performs well even in challenging scenarios. Moving forward, there is potential to develop a more intricate model that can identify vessels with greater precision and enhance the connectivity of vascular structures. Additionally, enhancing the interpretability of deep learning models is essential for understanding the model’s detected regions of interest. Visualization techniques can aid in comprehending the model’s inner workings, leading to informed decision-making and more efficient problem- solving.

Conflict of Interest: All authors have disclosed no conflict of interests. Funding/Support: This  study  received  no  specific grant from any funding agency in the public, commercial, or not- for-profit sectors.

Data Availability Statement: All data is publicly available and can be retrieved from open-source platforms like Google Dataset Search and Kaggle. The link to the dataset used was also cited in the reference.