DEFN: Dual-Encoder Fourier Group Harmonics Network for Three-Dimensional Indistinct-Boundary Object Segmentation
DEFN: Dual-Encoder Fourier Group Harmonics Network for Three-Dimensional Indistinct-Boundary Object Segmentation.
Xiaohua Jiang, Yihao Guo, Jian Huang, Yuting Wu, Meiyi Luo, Zhaoyang Xu, Qianni Zhang, Xingru Huang, Hong He, Shaowei Jiang, Jing Ye, Mang Xiao
Sir Run Run Shaw Hospital, Hangzhou Dianzi University
Fig. 1: The system structure of our proposed 3D segmentation architecture.
Fig. 2: The network structure of our proposed DEFN.
We proposed DEFN, a 3D OCT segmentation network for indistinct-boundary object segmentation with unobvious characteristics that are easily interfered with, such as macular holes and macular edema.
This repository contains the official Pytorch implementation for DEFN and DWC Loss, as well as the pre-trained model for DEFN.
Fig. 3: The detailed structure of the FuGH module.
The Fourier Group Harmonics (FuGH) module enhances noise reduction in medical image sequences by employing FFT for feature extraction in the frequency domain, enabling targeted noise filtration and efficient processing of periodic patterns.
Fig. 4: The detailed structure of the S3DSA module.
The Simplified 3D Spatial Attention (S3DSA) module improves the segmentation of macular holes and edema in fundus OCT sequences by an optimized spatial attention mechanism. It refines focus on crucial regions, enhancing segmentation quality and computational efficiency.
Fig. 5: The detailed structure of the HSE module.
The Harmonic Squeeze-and-Excitation Module (HSE) combines FuGH and Squeeze-and-Excitation (SE) blocks to enhance the segmentation performance of macular holes and macular edema, by extending the model's view field and recalibrating feature weights.
We have provided the GitHub links to the PyTorch implementation code for all networks compared to the experiments herein.
3D UX-Net, nnFormer, 3D U-Net, SegResNet, SwinUNETR, TransBTS, UNETR, DeepResUNet, ResUNet, HighRes3DNet, MultiResUNet, SegCaps, V-Net
Fig. 6: The Training Results using Isolated Strategy.
Segmentation results employing the isolated macular hole injection method, comparing the proposed DEFN, DEFN+DWC Loss, and prior segmentation models. The evaluation spans four classes: All (Average across all classes), MH (Macular Hole), ME (Macular Edema), and RA (Retina). The best values for each metric are highlighted in red, while the second-best values are highlighted in blue, and the values of our model are bolded.
Fig. 7: The Training Results using the Comprehensive Strategy.
Segmentation results employing the comprehensive macular hole injection method, comparing the proposed DEFN, DEFN+DWC Loss, and prior segmentation models. The evaluation spans four classes: All (Average across all classes), MH (Macular Hole), ME (Macular Edema), and RA (Retina). The best values for each metric are highlighted in red, while the second-best values are highlighted in blue, and the values of our model are bolded.
Fig. 8: The Fine-tuning Results using the Isolated Strategy.
Segmentation results of fine-tuning after isolated macular hole injection training, comparing the proposed DEFN, DEFN+DWC Loss, and prior segmentation models. The evaluation spans four classes: All (Average across all classes), MH (Macular Hole), ME (Macular Edema), and RA (Retina). The best values for each metric are highlighted in red, while the second-best values are highlighted in blue, and the values of our model are bolded.
Fig. 9: The Fine-tuning Results using the Comprehensive Strategy.
Segmentation results of fine-tuning after comprehensive macular hole injection training, comparing the proposed DEFN, DEFN+DWC Loss, and prior segmentation models. The evaluation spans four classes: All (Average across all classes), MH (Macular Hole), ME (Macular Edema), and RA (Retina). The best values for each metric are highlighted in red, while the second-best values are highlighted in blue, and the values of our model are bolded.
Fig. 10: The various 3D reconstruction results.
Five cases are presented with their original images and reconstruction outcomes. The first row exhibits the original images for each case. Rows two to five show the reconstructions based on four different rendering styles, while the sixth row provides a top view of the reconstruction results. Within the reconstructions, yellow regions indicate macular holes and blue regions signify macular edema.
This project is licensed under the MIT license.