D-Former: A U-shaped Dilated Transformer for 3D Medical Image Segmentation

Introduction

With the recent development of deep learning methods and computer vision algorithms, medical image segmentation has been revolutionized and achieved remarkable progresses (e.g., automatic liver and tumor lesion segmentation , brain tumor segmentation , and multiple sclerosis (MS) lesion segmentation ). Fully convolutional network (FCN) was first proved effective for general image segmentation tasks, which became a predominant technique for medical image segmentation .

To this end, a family of U-shaped networks [20, 77] extended the sequential FCN frameworks into encoder-decoder type architectures, alleviating the spatial information loss using skip-connections.

Competitive solutions were given by DeepLab models , which applied atrous convolutions instead of pool layers to expand the receptive field and introduced fully connected conditional random field (CRF) to maintain fine details.

Research Question

To investigate the necessity of the Global Scope Module (GSM), we replace it by the Local Scope Module (LSM), with the other architectural components unchanged.

Methodology

Medical image segmentation, as one of the critical computer-aided medical image analysis problems, aims to capture precisely the shapes and volumes of target organs and tissues by pixel-wise classification, obtaining clinically useful information for diagnosis, treatment, and intervention. (3) To further incorporate relative and absolute position information among patches, we apply a dynamic position encoding method to learn it from the input directly.

Study Design

Quantitative results on the Synapse dataset are reported in Table 1 , which show that our method outperforms the previous work by a clear margin.

It is notable that the concurrent Transformer-based methods nnFormer and MISSFormer achieve some performance gains compared to the CNN-based methods, while our method still brings further improvement in the average DSC by 1.43% compared to nnFormer and by 6.87% compared to MISSFormer.

Results & Findings

However, it was observed that vital details can be missing with the decrease of feature map sizes when FCN models go deeper. Although these CNN-based methods have achieved great performances in medical image segmentation tasks, they still suffered from limited receptive fields and cannot capture long-range dependency, leading to sub-optimal accuracy and failing to meet the needs of various medical image segmentation scenarios.

• However, it was observed that vital details can be missing with the decrease of feature map sizes when FCN models go deeper.
• Although these CNN-based methods have achieved great performances in medical image segmentation tasks, they still suffered from limited receptive fields and cannot capture long-range dependency, leading.
• On semantic medical image segmentation, Transformer-combined architectures can be divided into two categories: the main one adopts self-attention like operations to complement CNNs [1, ; the.
• (2) Compared with common 2D natural scene images, processing 3D medical images inevitably incurs larger model sizes and computation costs, especially when computing global feature interactions.
• Although some adaptations were proposed to reduce the operation scopes of self-attention (e.g., Pyramid Vision Transformer used progressive scaling pyramids to reduce the computation of large.

Practical Applications

For 3D medical images (e.g., 3D MRI scans), they may divide the input images into 2D slices and process the individual slices with the 2D models, which could lose useful 3D contextual information. Given the fact that position information could be dynamically learned based on the input x itself, a drawback in the previous work that requires a fixed number of patches can be avoided.

However, since the cross entropy loss is apt to perform well for uniform class distribution while Dice loss is more suitable for target objects of large sizes , each of them alone may not be effective for medical image segmentation tasks that involve imbalanced classes and target objects of small sizes.

Moreover, we introduced the dynamic position encoding block, making it possible to flexibly learn vital position information within input sequences.

CNN-based Segmentation Networks

Various CNN-based networks have been developed since U-Net, including U-Net++ and U-Net3+. These networks utilize skip connections and mechanisms to enlarge the receptive field, such as dilated convolutions, which have shown improved performance in segmentation tasks.

Transformers for Segmentation Tasks

Transformers in medical image segmentation can be categorized into two types: those that supplement CNNs and those that serve as the main architecture. Models like TransUNet and UNETR leverage both CNNs and Transformers to enhance segmentation performance.

Source Paper Figures and Captions

Source-paper figure

Illustrates the hierarchical structure and components of the D-Former model, highlighting its encoder-decoder design and the use of D-Former blocks.