Automated segmentation of ophthalmological OCT images

https://doi.org/10.2352/ISSN.2470-1173.2017.6.MOBMU-307 © 2017, Society for Imaging Science and Technology Automated segmentation of ophthalmological OCT images Friedrich Müller 1 , and Reiner Creutzburg 2 1 Nürnberger Str. 24a, D-10789 Berlin, Germany Technische Hochschule Brandenburg, Department of Informatics and Media, IT- and Media Forensics Lab, P.O.Box 2132, D-14737 Brandenburg, Germany 2 Email: [email protected], [email protected] Abstract OCT (Optical coherence tomography) has become a popular method for macular degeneration diagnosis. The advantages over other methods are: OCT is noninvasive, it has a high penetration and it has a high resolution. However, the always present speckle noise and the low contrast differences make it hard to segment the layers for the measurements correctly. The aim of this paper is to show the importance of optimizing the retinal segmentation process. Actual automatic segmentation algorithms are capable of detecting up to eleven layers in real time, but often fail at images with (strong) macular degeneration, which are complicating the separation of the layers from each other. This paper sums up some actual aspects of developments in retinal segmentation and shows the limits of actual algorithms. As a comprehensive test process for this paper, we tested all common image processing algorithms and implemented found promising, modern OCT segmentation methods. The result is a wide scale analysis which can be used as a roadmap for optimizing the process of retinal segmentation. Promising algorithms were found with the Canny edge detector, graph cuts and dynamic programming. Combining these algorithms results, the graph-, gradient-, intensity information, and decreasing the search region step by step has shown to be a fast and reliable solution. All tests were using 2D image data, 3D data could be used as well but plays no role in this paper. The testing process includes pre-filtering for image denoising, which can be done fast and is creating better preconditions for the segmentation process. Introduction To analyse OCT images one has to deal with multiple problems. Speckle noise is disturbing edge detectors as well as the intensity based techniques. Blood vessels creating shadows through all the layers making it harder to follow a border, and motion artifacts complicate the 3D analysis even more than the 2D analysis. To handle these problems a lot of strategies have been developed. In [39] the gradient information of two “scales” were combined with dynamic programming to have always reliable border information of different layers. [58] presents a global approach with active contours for refining the outer boundaries and a kmeans edge detection for the inner ones. Another segmentation method based on dynamic programming was presented in [59], a robust detection of three layers on noisy images was reached. 102 We developed a layer detector which is capable to detect eight boundaries. We tried to find as many of them as possible robustly. A following test process was done to configure the algorithms to leave out the ones which were mostly false detected. Our approach is based on a combination of Canny edge detection and dynamic programming. Denoising Most of the presented segmentation methods are starting with a denoising pre-filter. Median/Mean filters, directional filters and nonlinear diffusion filters are the most common and the fastest. In our tests we have shown that the detection quality is enhanceable using a better denoising filter. In [60] modern denoising filter for OCT images were developed and tested. We can confirm their results but have chosen the BM3D [29] filter for our framework. Needless to say, execution time is an important point and a prefiltering takes extra time, so we used well optimizable algorithms and tested GPU supported processing successfully. The images 1 to 3 show an unfiltered original OCT image and several examples for filtered results. Obviously, if the noise is reduced the edge detection can work more successfully. Segmentation Algorithms like graph cuts or k-means segmentation have been tested successfully [1], but need a more intense initialization process and are computational more costly then the following algorithms. It was shown that robust edge detection can be established by the Canny edge detector. The parameters have been adapted in multiple detection steps in different image regions. Starting with the more prominent edges to the lesser ones. The following images 8 and 9 show the segmentation results with automatic multiple layer boundary detection in an unfiltered and filtered image, respectively. Even without denoising some of the boundaries were well detected. The preprocessed (denoised) image shows a more precise detection, which is clearly visible. This helps especially if the images are even more corrupted by artifacts than the example. Algorithm For example, a solution was developed using dynamic programming. Thus, eight borders are identified in OCT images of healthy eyes and without the nerve fiber head present in the image. Test images of the described eye diseases were not available IS&T International Symposium on Electronic Imaging 2017 Mobile Devices and Multimedia: Enabling Technologies, Algorithms, and Applications 2017 Figure 1. Original OCT image. from the test device. In order to achieve the goal, the improved noise reduction was used by the BM3D algorithm and the already proven dynamic programming method [37]. Fig. 5 shows the algorithm flow. The original image is first denoised by the BM3D filter. For this, the best values determined in the above tests (see annex) were determined. An optional Gauss filter with a small kernel size of three pixels has been found to be advantageous in various receptacles for suppressing the vein shadow. This produces a Sobel and a Canny edge image, which are then combined with the OR operation. In the image thus combined, one of the lateral columns (left / right) is searched for the highest brightness values (strongest gradients), which at the same time have a certain distance from each other. These are the starting points for the search in the DP graph. By their order, number and intensity, it is possible to estimate the limit. The strongest three are the most representative boundaries, the top of which is the vitreous / NFL, the second the IS / OS and the third the RPE / Chroroidea boundary. Then the graph is created after dynamic programming. In this, the first boundary (vit./NFL boundary) is found first by the uppermost starting point. The area above (glass body) is removed from the graph. Secondly, the bottom limit is detected by its start point and the area under it is removed from the graph. At the third and middle of the stronger border, the image is divided into two regions, each of which is searched separately. A new endpoint finding, limited to the search areas, is possible here but not necessary. Since only two equidistant boundaries are searched for in the lower region, the thickness of the latter is trimmed and the starting points are set on the thirds. Of these, the lower, i.e. the receptor toothing / fraction K boundary is found first, the search range is reduced after the detection and the connection site / OS boundary is searched. In the upper area of NFL to IS, the remaining starting points found by the search are already well-recognized. Again, a new search is not required, but would positively influence the reliability. The NFL / GCL boundary is recognized first. For this, the parametrization of the Canny algorithm is crucial, otherwise the limit may change from the fovea to the IPL / INL boundary. This is detected and recognized as the next prominent boundary, after which the upper region is cut off. The next higher limit is the OPL / ONL. In order to avoid recognition errors, the search space was limited to 20 pixels above the IS / OS. The result of the algorithm can be seen in Fig. 9. The prominent borders show no mistakes. In the RPE area, vein shadows produce slight bulges. Only the mean limit (OPL / ONL) runs partly into the INL / OPL boundary. However, the problem can be solved by re-detecting the superimposed Canny edges with adapted parameters. This would simultaneously lead to the recognition of two new boundaries, since the altered search space and the edges of the Canny algorithm, which are more visible by the parameter changes, would then allow this. In general, the parametrization was kept simple compared to [39]. A great advantage of the developed method lies in the simplicity and its further development potential. Since the number of the eight borders detected here, as described, can be further increased by parameter adaptation and re-detection of certain regions. A parameter setting of all values is sufficient here to detect all eight borders. The straightening (horizontally flattening) of the RPE was not required in order nevertheless to achieve similar and partly better results than in the elaborate designed algorithm in [37]. Summary and Discussion of Results Within the scope of this thesis, a number of methods for noise reduction, segmentation and boundary finding in OCT image data were investigated. The findings of the tests were combined in a segmentation program. B-scans of healthy patients can be automatically segmented with the help of this. It recognizes eight layer boundaries. The basic approach of improving the de-noise pre-filter proved successful. As a result, not only the complexity but also the error rate of the algorithm has been reduced. Compared to comparable work fewer processing steps are also required. The computational speed of the developed method has been neglected and requires optimization to achieve speeds of one IS&T International Symposium on Electronic Imaging 2017 Mobile Devices and Multimedia: Enabling Technologies, Algorithms, and Applications 2017 103 Figure 2. Denoising results. 104 IS&T International Symposium on Electronic Imaging 2017 Mobile Devices and Multimedia: Enabling Technologies, Algorithms, and Applications 2017 Figure 3. Denoising results (continued). IS&T International Symposium on Electronic Imaging 2017 Mobile Devices and Multimedia: Enabling Technologies, Algorithms, and Applications 2017 105 Figure 4. Combining segmentation results 106 IS&T International Symposium on Electronic Imaging 2017 Mobile Devices and Multimedia: Enabling Technologies, Algorithms, and Applications 2017 Figure 5. Algorithm flow Figure 6. Result of algorithm of Fig. 5 Figure 7. Block-matching filtered OCT image. IS&T International Symposium on Electronic Imaging 2017 Mobile Devices and Multimedia: Enabling Technologies, Algorithms, and Applications 2017 107 Figure 8. Segmented unfiltered OCT image. Figure 9. Segmented layers in filtered OCT image. 108 IS&T International Symposium on Electronic Imaging 2017 Mobile Devices and Multimedia: Enabling Technologies, Algorithms, and Applications 2017 minute for the calculation of a C-scan. During the research of the OCT segmentation procedures in the course of this work it was found that each author uses different methods for the evaluation. In the best case a selection shot of a certain manufacturer is segmented and compared with the manual results of eye doctors for deviation in pixels. A standardized public database with recordings from all manufacturers would make sense for the objective quantitative comparison of the procedures. Templates for ideal results could also be filed here. To evaluate the quality of the segmentation algorithms, such an approach already exists at the University of Berkeley [50]. During the compilation of this thesis, the currently popular development of the processing of OCT recordings has, if necessary, produced procedures which have not been included in the research. Many new publications of recent years and months show the existing development potential in this area. Improvement possibilities The OCT segmentation approach developed here also has development potential. This starts with the exclusion and segmentation algorithms, which are excluded in this work, which are worth a further investigation, as well as new developments in this field. The developed algorithm is real-time without the step of BM3D de-noise. However, since speed was not the focus of this work, the computing time was considered secondary. Here an optimized continuation makes sense. For this, current GPU implementations of the BM3D filter could serve to shorten the processing time. Simple, fast de-noise filters with a simultaneous search range limitation also offer confident examination approaches. The algorithm should be further improved by adjusting the parameters of the edge detector in each new search area as in [39]. The potential of several artefact-corrected edge images in combination can be seen in Fig. 10. This is best automated achievable result, synthesized from the results of out tested algorithms, here 14 borders are detected. Acknowledgments The authors wish to thank Dipl.-Ing. J. Finger, Dipl.Ing. M. Stasche and Dipl.-Ing. (FH) B. Dammann of Roland Consult GmbH in Brandenburg (Germany) for their generous support and encourage. References [1] Friedrich Müller: “Bildverarbeitungsmethoden zur Schichtenunterteilung artefaktbehafteter Netzhautschnittbilder” Master Thesis, Fachhochschule Brandenburg, Fachbereich Informatik und Medien, 2015 [2] Duker, Jay S., Nadia K. Waheed, and Darin R. Goldman. Handbook of Retinal OCT. Saunders 2014 [3] ”Roland Consult - RETIangio.” Roland Consult - RETIangio. Accessed April 19, 2015. http://www.roland-consult.com/cms/products/ retiangio.html. [4] ”Retinal OCT Imaging - Ophthalmic Photographers’ Society.” Retinal OCT Imaging - Ophthalmic Photographers’ Society. Accessed April 19, 2015. http://www.opsweb. org/?page=RetinalOCT. [5] Jähne, Bernd. Digitale Bildverarbeitung. Berlin: Springer, 2012. [6] Svoboda, Tomas, Vaclav Hlavac, and Jan Kybic. Image Processing, Analysis, and Machine Vision: A MATLAB Companion. Toronto: Thomson Learning, 2008. [7] ”Augenärztliche Privatpraxis Scholl & Augustin.” Augenärztliche Privatpraxis Scholl & Augustin. 19 Apr. 2015. http://www.augenaerzte-baden-baden.de/cms/ iwebs/default.aspx?mmid=6470&smid=24202 [8] ”Farbwahrnehmung: Signalverarbeitung in AugeNetzhaut, Metamerie, Rezeptoren: Stäbchen Zapfen, Aufbau Menschliches Auge.” 19 Apr. 2015. http: //www.filmscanner.info/Farbwahrnehmung.html. [9] ”Farbwahrnehmung.” Wikipedia. Wikimedia Foundation. 19 Apr. 2015. http://de.wikipedia.org/wiki/ Farbwahrnehmung. [10] ”Retina - DocCheck Flexikon.” DocCheck Flexikon. 19 Apr. 2015. http://flexikon.doccheck.com/de/Retina. [11] ”Projektion (Nervensystem).” Wikipedia. Wikimedia Foundation. 19 Apr. 2015. http://de.wikipedia.org/wiki/ Projektion_(Nervensystem). [12] ”Eye.” Eye. 19 Apr. 2015. http://histology.med. umich.edu/eye. [13] ”F4-opth-5-101:Clarifying the Mechanism of Idiopathic Macular Hole Development in Fellow Eyes Using Spectraldomain Optical Coherence Tomography- Open-i.” F4opth-5-101:Clarifying the Mechanism of Idiopathic Macular Hole Development in Fellow Eyes Using Spectraldomain Optical Coherence Tomography- Open-i. 19 Apr. 2015. http://openi.nlm.nih.gov/detailedresult. php?img=3037037_opth-5-101f4&req=4. [14] Tso, Mark O.m. ”Pathology of Cystoid Macular Edema.” Ophthalmology 89.8 (1982): 902-15. [15] ”Diabetes und Auge.” Diabetes und Auge. 19 Apr. 2015. http://www.neuhann.de/diabetes-und-auge/. [16] ”Gliose - DocCheck Flexikon.” DocCheck Flexikon. 19 Apr. 2015. http://flexikon.doccheck.com/de/Gliose. [17] ”Epiretinale Gliose Und Makulaforamen.” Makulaforamen. 19 Apr. 2015. http://www.neuhann.de/ makulaforamen/. [18] Bouma, Brett E. Handbook of Optical Coherence Tomography. New York: Informa Healthcare, 2009. [19] ”Seminarvortrag: Optische Kohärenztomographie” Seminarvortrag: OCT. 19 Apr. 2015. http://www. physik.uni-regensburg.de/forschung/schwarz/ KlassischeOptik/9-Kohaerenztomographie.pdf. [20] J. Braunbeck: Über eine Erscheinung beim Durchgang monochromatischen, kohärenten Lichtes durch trübe Medien. In: Naturwissenschaften. 49, Nr. 17, 1962, S. 389 [21] M. Forouzanfar, H. Abrishami-Moghaddam, and M. Gity, “A new multiscale Bayesian algorithm for speckle reduction in medical ultrasound images,” Signal, Image and Video Processing, Springer, vol. 4, pp. 359-75, Sep. 2010 [22] J.S.Lee, “Digital image enhancement and noise filtering by use of local statistics”, IEEE Transactions, Mar 1980. [23] Charandeep Singh Bedi and Dr. Himani Goyal. Article:Qualitative and Quantitative Evaluation of Image Denoising Techniques.International Journal of Computer Applications 8(14):31–34, October 2010. Published By Foun- IS&T International Symposium on Electronic Imaging 2017 Mobile Devices and Multimedia: Enabling Technologies, Algorithms, and Applications 2017 109 Figure 10. Approximately ideal, automatically generated result from these images. Vein shadows produce also bulges here. dation of Computer Science. [24] Weickert, Joachim. Anisotropic Diffusion in Image Processing. Stuttgart: B.G. Teubner, 1998. [25] F. Estrada, D. Fleet, and A. Jepson, “Stochastic Image Denoising”, British Machine Vision Conference , London 2009. [26] C. Tomasi and R. Manduchi. 1998. Bilateral Filtering for Gray and Color Images. In: Proceedings of the Sixth International Conference on Computer Vision (ICCV ’98). IEEE Computer Society, Washington DC [27] Buades, Antoni (20–25 June 2005). ”A non-local algorithm for image denoising”. Computer Vision and Pattern Recognition, 2005 2: 60–65. doi:10.1109/CVPR.2005.38 [28] ”Non-local Means.” Wikipedia. Wikimedia Foundation. 19 Apr. 2015. http://en.wikipedia.org/wiki/ Non-local_means. [29] Dabov, K.; Foi, A.; Katkovnik, V.; Egiazarian, K.: Image denoising with block-matching and 3D filtering. In: Proceedings of SPIE Electronic Imaging ’06, 2006 [30] Nobuyuki Otsu (1979). ”A threshold selection method from gray-level histograms”. IEEE Trans. Sys., Man., Cyber. 9(1): 62–66. [31] Canny, John. ”A Computational Approach to Edge Detection.” IEEE Transactions on Pattern Analysis and Machine Intelligence PAMI-8.6 (1986): 679-98. [32] Richard Bellman: Dynamic Programming. Princeton University Press, 1957. [33] D.M. Greig, B.T. Porteous and A.H. Seheult (1989), Exact maximum a posteriori estimation for binary images, Journal of the Royal Statistical Society Series B, 51, 271–279. [34] Boykov YY, Jolly MP: Interactive graph cuts for optimal boundary and region segmentation of objects in n-d images. Int. C. Comput. Vision, 2001: 01:105–112 [35] Pei-Gee Ho, ISBN 978-953-307-228-9, 550 pages, Publisher: InTech, Chapters published April 19, 2011 under CC BY-NC-SA 3.0 license [36] Huang, Yijun, Ronald P. Danis, Jeong W. Pak, Shiyu Luo, James White, Xian Zhang, Ashwini Narkar, and Amitha Domalpally. ”Development of a Semi-Automatic Segmentation 110 [37] [38] [39] [40] [41] [42] [43] [44] [45] [46] Method for Retinal OCT Images Tested in Patients with Diabetic Macular Edema.” Ed. Orhan Aktas. PLoS ONE 8.12 (2013): E82922. Chiu, Stephanie J., Xiao T. Li, Peter Nicholas, Cynthia A. Toth, Joseph A. Izatt, and Sina Farsiu. ”Automatic Segmentation of Seven Retinal Layers in SDOCT Images Congruent with Expert Manual Segmentation.” Optics Express 18.18 (2010): 19413. Shalbaf, F. ; Auckland Bio-Eng. Inst., Univ. of Auckland, Auckland, New Zealand ; Turuwhenua, J. ; Vaghefi, E. ; Dokos, S.. “An image processing pipeline for segmenting the retinal layers from optical coherence tomography images” Image and Vision Computing New Zealand (IVCNZ), 2013 Yang, Qi, Charles A. Reisman, Zhenguo Wang, Yasufumi Fukuma, Masanori Hangai, Nagahisa Yoshimura, Atsuo Tomidokoro, Makoto Araie, Ali S. Raza, Donald C. Hood, and Kinpui Chan. ”Automated Layer Segmentation of Macular OCT Images Using Dual-scale Gradient Information.” Optics Express 18.20 (2010): 21293. ”Image Denoising Benchmark: Results, Scripts, and Data.”Image Denoising Benchmark: Results, Scripts, and Data. 20 Apr. 2015. http://www.cs.utoronto.ca/ ~strider/Denoise/Benchmark/index.html. Getreuer, Pascal. ”Rudin-Osher-Fatemi Total Variation Denoising Using Split Bregman.” Image Processing On Line (2012). Norbert Wiener: Extrapolation, Interpolation, and Smoothing of Stationary Time Series. Wiley, New York NY 1949. Fan, Hui, Yongliang Wang, and Jinjiang Li. ”Image Denoising Algorithm Based on Dyadic Contourlet Transform.” Journal of Software 6.6 (2011). ”Canny Edge Detector.” Canny Edge Detector. 20 Apr. 2015. http://rsbweb.nih.gov/ij/plugins/canny/ index.html. Deriche, Rachid. ”Using Canny’s Criteria to Derive a Recursively Implemented Optimal Edge Detector.”International Journal of Computer Vision 1.2 (1987): 167-87. ”’A Trou’ Wavelet Filter and Adaptive Median Filter.” IS&T International Symposium on Electronic Imaging 2017 Mobile Devices and Multimedia: Enabling Technologies, Algorithms, and Applications 2017 [47] [48] [49] [50] [51] [52] [53] [54] [55] [56] [57] [58] [59] [60] ’A Trou’ Wavelet Filter and Adaptive Median Filter. 20 Apr. 2015. http://rsb.info.nih.gov/ij/plugins/ a-trous-wavelet-filter.html Francisco Estrada, David Fleet, Allan Jepson. “Stochastic Image Denoising” (2009) Witkin, A. P. ”Scale-space filtering”, Proc. 8th Int. Joint Conf. Art. Intell., Karlsruhe, Germany, 1019–1022, 1983. Arbelaez, P., M. Maire, C. Fowlkes, and J. Malik. ”Contour Detection and Hierarchical Image Segmentation.” IEEE Transactions on Pattern Analysis and Machine Intelligence 33.5 (2011): 898-916. ”The Berkeley Segmentation Dataset and Benchmark.” The Berkeley Segmentation Dataset and Benchmark. 20 Apr. 2015. http://www.eecs.berkeley.edu/Research/ Projects/CS/vision/bsds/. Garvin MK, Abramoff MD, Kardon R, Russell SR, Xiaodong W, Sonka M. Intraretinal layer segmentation of macular optical coherence tomography images using optimal3–D graph search. IEEE Trans Med Imaging. 2008;27:1495–505 Niblack, W (1986), An introduction to Digital Image Processing, Prentice-Hall Grady, L., and C.v. Alvino. ”The Piecewise Smooth Mumford–Shah Functional on an Arbitrary Graph.” IEEE Transactions on Image Processing 18.11 (2009): 2547-561. ”GitHub.” GitHub. 20 Apr. 2015. https://github.com/ pangyuteng/caserel/commits/master. Hasan, Mohammad Mahedi, ”Adaptive Edge-guided Blockmatching and 3D filtering (BM3D) Image Denoising Algorithm” (2014). University of Western Ontario - Electronic Thesis and Dissertation Repository. Paper 2003. ”Thesis (Selection of Subject).”Thesis (Selection of Subject). 20 Apr. 2015. https://is.cuni.cz/studium/ eng/dipl_st/index.php?doo=detail&did=156093. Srinivasan, Pratul P., Stephanie J. Heflin, Joseph A. Izatt, Vadim Y. Arshavsky, and Sina Farsiu. ”Automatic Segmentation of up to Ten Layer Boundaries in SD-OCT Images of the Mouse Retina with and without Missing Layers Due to Pathology.” Biomedical Optics Express 5.2 (2014): 348. Rossant, Florence, Ghorbel, Itebeddine, Bloch, Isabelle, Pâques, Michel and Tick, Sarah: ”Automated Segmentation of Retinal Layers in OCT Imaging and Derived Ophthalmic Measures.” Paper presented at the meeting of the ISBI, (2009) Chiu SJ, Izatt JA, O’Connell RV, et al. “Validated automatic segmentation of AMD pathology including drusen and geographic atrophy in SD-OCT images.” Invest. Ophthalmol. Vis. Sci., 53:53–61 (2012) Leyuan Fang, Shutao Li, Qing Nie, Joseph A. Izatt, Cynthia A. Toth, and Sina Farsiu: ”Sparsity based denoising of spectral domain optical coherence tomography images.” Biomed. Opt. Express 3, 927-942 (2012) IS&T International Symposium on Electronic Imaging 2017 Mobile Devices and Multimedia: Enabling Technologies, Algorithms, and Applications 2017 111