Deep Residual Learning For Compressed Sensing Mri
Deep Residual Learning For Compressed Sensing Mri
Deep Residual Learning For Compressed Sensing Mri
Bio Imaging and Signal Processing Lab., Dep. of Bio and Brain Engineering, KAIST
16
4.2. Network training
The original k-spaces were retrospectively downsampled by 4
times with 13 ACS (autocalibration signal, 5 percents of total
PE) lines in the k-space center. As shown in Fig. 1, the resid-
ual is constructed as the difference between the reconstruc-
tion images from fully sampled data and the down-sampled
data. During the training, residual images were used as labels
(Y ) whereas the aliased images from down-sample data were
used as input (X). Since MR images are complex valued and
standard CNNs are real-valued, we trained the two residual
networks: one for the magnitude and the other for the phase
of the images. Both networks have the same residual learning
structure (however, due to the page limit, we only show the
magnitude results).
Fig. 3. (a) Single scale residual learning with a modified The network was implemented using MatConvNet tool-
deconvolution network framework[12] with symmetric con- box(ver.20, http://www.vlfeat.org/matconvnet/) in MATLAB
tracting path. (b) Multi scale residual learning with U-net 2015a environment (Mathworks, Natick). We used a GTX
architecture. 1080 graphic processor and i7-4770 CPU (3.40GHz). The
weights of convolutional layers were initialized by Gaussian
wl , bl represent weights and bias of l-th convolution layer, random distribution with Xavier method to achieve proper
respectively. Then, the l-th layer of the network performs fol- scale. This helped to prevent the signal from exploding or
lowing operation, repeatively, f l (xl ) := σ(BN (wl ∗xl +bl )), vanishing in the early stage of learning. The stochastic gradi-
where σ(·) is the ReLu function, and BN is a batch normal- ent descent (SGD) method with momentum was used to train
ization. In contracting path, we concatenate xl along channel the weights of the network and minimized the loss function.
dimension [11]. It took about 9 hours for training the network.
Fig. 3 illustrates the several network configuration we To verify the performance of the network, for multi-
have investigated for residual learning. Fig. 3(a) is single channel dataset, we compared the reconstruction results with
scale residual learning with a modified deconvolution net- those of GRAPPA. We also compared the ALOHA [1] recon-
work framework[12]. Fig. 3(b) is a multi-scale residual struction as the state-of-the-art CS algorithm for both single
learning with additional pooling and unpooling(conv trans- and 4-channel reconstructions.
pose) layers on top of Fig. 3(a). To make the number of 5. RESULTS
network features similar to Fig. 3(a), the number of channels
In single channel experiment with x4 acceleration (Fig. 4(b)),
are doubled after the pooling layer. This architecture is of-
there exists significant amount of aliasing artifacts from the
ten called the U-net [11]. In the following, we investigate
zero-filled reconstruction. Moreover, due to coherent aliasing
the performance of each network configuration for residual
artifacts from uniform downsampling, most of the existing
learning.
CS algorithm failed and only ALOHA was somewhat suc-
cessful with slightly remaining aliasing artifacts. However,
4. MATERIALS AND METHODS
the residual learning results clearly showed very accurate re-
4.1. MR dataset construction visually and quantitatively by removing the co-
herent aliasing artifacts. In four channel parallel imaging ex-
We used brain MR image dataset, which consists of total 81 periments in Fig. 4(c), GRAPPA shows the strong aliasing ar-
axial brain images from 9 subjects. The data were acquired in tifacts due to the insufficient number of coils and ACS lines.
cartesian coordinate with a 3T MR scanner that has four Rx ALOHA reconstruction could remove most of the aliasing ar-
coils (Siemens, Verio). The following parameters were used tifacts, but the results were not perfect due to the coherent
for SE and GRE scans: TR 3000-4000ms, TE 4-20ms, slice sampling. However, the proposed method provided near per-
thickness 5mm, 256× 256 acquisition matrix, 4 coils, FOV fect reconstruction.
240× 240, FA 90 degrees. For the dataset, we split the train- In Fig. 5(a)(b), the convergent plots for a test data set from
ing and test data set by randomly choosing about 80% of total single and four channel reconstruction are illustrated. Among
images for training and about 20% for testing. For data aug- the various residual learning architectures, the multi-scale
mentation, we generated 32 times more training samples by residual learning (Fig. 3(b)) provided the best reconstruction
rotating, shearing and flipping the images. For single channel results. Moreover, residual learning was significantly better
experiments, we chose 1 channel data from the four channel than direct image learning with the same U-net architecture
data. as shown in Fig. 5.
17
only multi channel data but also single channel data. Even
with severe coherent aliasing artifacts, the proposed residual
learning successfully learns the aliasing artifacts, whereas the
existing parallel and CS reconstruction fails. Moreover, com-
pared to existing algorithms which need heavy computational
cost, the proposed network produces the results instantly.
7. REFERENCES
[1] Kyong Hwan Jin, Dongwook Lee, and Jong Chul Ye, “A gen-
eral framework for compressed sensing and parallel MRI using
annihilating filter based low-rank hankel matrix,” IEEE Trans.
on Computational Imaging, (in press), 2016.
[2] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton, “Im-
agenet classification with deep convolutional neural networks,”
in Advances in Neural Information Processing Systems, 2012,
pp. 1097–1105.
[3] Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, and
Lei Zhang, “Beyond a Gaussian denoiser: Residual learn-
ing of deep CNN for image denoising,” arXiv preprint
arXiv:1608.03981, 2016.
[4] Peter L Bartlett and Shahar Mendelson, “Rademacher and
Fig. 4. (a) Original image: (top) single channel, (bottom) 4- Gaussian complexities: Risk bounds and structural results,”
channel. (b) Single channel reconstruction results at x4 accel- Journal of Machine Learning Research, vol. 3, no. Nov, pp.
eration. (c) 4 channel reconstruction at x4 acceleration. The 463–482, 2002.
resulting normalized mean square error(NMSE) is displayed [5] Matus Telgarsky, “Benefits of depth in neural networks,” arXiv
at the bottom of each figure. preprint arXiv:1602.04485, 2016.
[6] Monica Bianchini and Franco Scarselli, “On the complexity
of neural network classifiers: A comparison between shallow
and deep architectures,” IEEE Trans. on Neural Networks and
Learning Systems, vol. 25, no. 8, pp. 1553–1565, 2014.
[7] Shanshan Wang, Zhenghang Su, Leslie Ying, Xi Peng, Shun
Zhu, Feng Liang, Dagan Feng, and Dong Liang, “Accelerating
magnetic resonance imaging via deep learning,” in 2016 IEEE
13th International Symposium on Biomedical Imaging (ISBI).
IEEE, 2016, pp. 514–517.
[8] K Hammernik, F Knoll, D Sodickson, and T Pock, “Learning
a variational model for compressed sensing MRI reconstruc-
Fig. 5. NMSE convergence graph for test data. (a) single tion,” in Proceedings of the International Society of Magnetic
channel reconstruction, (b) four channel reconstruction. Resonance in Medicine (ISMRM), 2016.
[9] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun,
The reconstruction time of GRAPPA was about 30 sec- “Deep residual learning for image recognition,” arXiv preprint
onds for multi channel image under the aforementioned hard- arXiv:1512.03385, 2015.
ware setting. The reconstruction time for ALOHA was about
[10] Herbert Edelsbrunner and John Harer, “Persistent homology-a
10 min for four channel and about 2 min for single channel survey,” Contemporary Mathematics, vol. 453, pp. 257–282,
data. The proposed network only took less than 41 ms for 2008.
multi channel image and about 30 ms for single channel im-
[11] Olaf Ronneberger, Philipp Fischer, and Thomas Brox, “U-net:
age.
Convolutional networks for biomedical image segmentation,”
in International Conference on Medical Image Computing and
6. DISCUSSIONS AND CONCLUSIONS Computer-Assisted Intervention. Springer, 2015, pp. 234–241.
In this article, we have presented an accelerated MRI re- [12] Hyeonwoo Noh, Seunghoon Hong, and Bohyung Han, “Learn-
construction method from uniform downsampled MR brain ing deconvolution network for semantic segmentation,” in Pro-
data using residual learning. By learning aliasing artifacts ceedings of the IEEE International Conference on Computer
that have simpler topology, the resulting risk of the proposed Vision, 2015, pp. 1520–1528.
residual learning can be reduced so that we can obtain more
accurate MR images. The proposed method works on not
18