Final
Final
Final
Presented by
Ma Lai Yee Myint
Ph.D CEIT – 5 (13th Batch)
1
Introduction
• Urinary Calculi are small, hard deposits of mineral and acid
salts on the inner surfaces of the kidneys (drops along the
urinary track).
http://www.pilotfriend.com/aeromed/medical/images2/25.jpg
2
Background
• Why doing research in urinary calculi Disease?
– Prevalence of urinary calculi disease and recurrence rate
Lifetime prevalence of incident kidney stones in the United
10.0%
8.8%
9.0%
8.0%
7.0%
6.0% 5.2%
5.0%
4.0% 3.2%
3.0%
2.0%
1.0%
States
0.0%
Fig: Prevalence of urinary calculi disease from 1976 to 2010 Fig: Recurrent rates of urinary calculi forming after treatment [1]
[1]
3
Background
• GBD(GLOBAL BURDEN OF DISEASES) 2010
– Years of life lost (YLLs) quantify premature mortality in Myanmar [2][3]
Chronic Kidney Disease in period of 20 years (from 1990 to
2010)
1.80%
1.60%
1.40%
1.20%
1.00%
0.80%
0.60%
0.40%
0.20%
0.00%
1990 2010
% of total disease
4
Related Researches Advantage Limitation
An artificial intelligence-based clinical Predict treatment procedure (stent Using patient history variable,
Previous Works
decision support system for large kidney
stone treatment [4].
place after stone removal, blood urinary stone parameters and
transfusion during the surgery or not) laboratory data
S.T.O.N.E. Nephrolithometry: Novel Surgical Predict only PCNL treatment success Using stone parameters: stone size
Classification System for Kidney Calculi [5] or the risk of perioperative (S), tract length (T), obstruction (O),
complications after PCNL number of involved calices (N), and
essence or stone density (E)
Measuring stone volume - Three-dimensional Classify: Small, Medium and Large No single formula for estimating
software reconstruction or an ellipsoid stone stone volume
algebra formula? [6]
Predicting shockwave lithotripsy outcome for Predict the probability of stone free Using patient history variable,
urolithiasis using clinical and stone computed outcome after ESWL treatment urinary stone parameters
tomography texture analysis Variables [7]
Kidney Stone Diagnosis and Management [8] Predict treatment type Using linear measurement of stone
1. Based on HU value (* inappropriate in irregular stone,
2. Based on size eg: staghorn calculi)
5
Related Researches Advantage Limitation
Image Quality Improvement in Kidney Stone Detect the stones in CT, identify the size Using semi-automated program for
Previous Works
Detection on Computed Tomography Images [9] and location of stone based on pixel stone parameter prediction
count
The Usefulness of The Hounsfield Unit And Stone To evaluate the use of stone density Using software application, firstly
Heterogeneity Variation in Predicting The variation coefficient (SDVC) as an output mean and SD (standard
Shockwave Lithotripsy Outcome [10] indicator of stone heterogeneity for deviation) in manual-selected stone
predicting extracorporeal shock wave region, and then calculated SDVC as
lithotripsy (ESWL) outcome in urinary SD/mean *100.
calculi
3D Medical Image Visualization and Volume To visualize 3D stone object and to For 3D visualization, it was used an
Estimation of Pathology Zones [11] estimate stone volume algorithm using standard OpenGL
graphic library and extract the stone
region using histogram thresholds (semi-
automatic segmentation)
Noncontract Computed Tomography Parameters for Predict mean stone density, stone Using available imaging 3D software,
Predicting Shock Wave Lithotripsy Outcome in heterogeneity index (SHI) and variation stone parameters are measured to
Upper Urinary Tract Stone Cases [12] coefficient of stone density (VCSD) for provide ESWL outcomes prediction
predicting ESWL outcomes
Stone heterogeneity index as the standard To group High SHI and Low SHI for ESWL Define the stone heterogeneity index
deviation of Hounsfield units: A novel predictor for success and for stone fragility (SHI) as the standard deviation (SD) on
shockwave lithotripsy outcomes in ureter calculi CT using available imaging 3D software
[13]
6
Bone
Stone
Stent
Slice Selection • Difficulty to select the largest slice among all included slices in a stone
Parameter • Difficulty to measure the stone parameters for diagnose the stone problem and
Measuring to make medical treatment decisions without the help of expert person 8
Objectives
System Design
Input Imag e
3D segmentation
Soft_organ removing
No
Is there stone?
Yes
Feature Extraction
CT parameters
and 2D and 3D
visualization for
Diagnosis and
Treatment type
decis ion
End
10
Contribution
11
11
Contribution
Stone size
12
Stone density
(only HU value)
Stone size
Stone size
Stone density
(only HU value) 13
Stone density
(only HU value)
Stone
heterogeneity
index
J. Y. Lee, J. H. Kim, D. H. Kang, D. Y. Chung, D. H. Lee, and H. Do, “Stone heterogeneity index as the
Surgical Management of Urinary Lithasis, by MohammadBaghdadi,
standard deviation of Hounsfield units : A novel predictor for shock- wave lithotripsy outcomes in
Urology resident KKNGH, https://slideplayer.com/slide/1570041/
13
ureter calculi,” Sci. Rep., vol. 6, no. 23988, pp. 1–7, 2016, doi: 10.1038/srep23988.
Contribution
Previous work New contribution
• Stone size, density (only HU value) & other hardware • Seven effective CT parameters (Area, 3D
devices (urine, blood, patient’s history, etc…) volume, Diameter, Density, Average HU,
• Stone parameter using CT SHI, VCSD) {not only for diagnosis but
also for treatment planning]
Largest cross-sectional slice of the largest stone where • Automate stone region selection (without
more than one stone was treated. Stone volume is the sum any noise, without using any drawing
of individual stone volumes in cases with more than one tools)
stone • Auto 2D stone parameters measurements
(auto largest slice selection)
Stone area on this selected slice (Manually or • Auto 3D stone parameters measurements
• Completely depend on radiologist (may be variability-reduce
Automatically)
to get Time consuming & variability
accurate CT information) (Not only one slice)
• 3D visualization (to show the shape and external structure of • Automated computer aided system
the stone together with other high intensity objects such • 3D visualization and 2D internal structure
as bones) visualization (to display shape, number,
the stone composition), clear output
without any disturbances
14
Proposed Methods
3D Parameter Measuring and 3D
Visualization
3D Parameter
Measuring
and 3D
Visualization
3D
Segmentation False Positive Feature
(Noise Reduction Extraction
Removal)
2D Parameter
Measuring
and Internal
Structure
Visualization
16
Thresholding and Otsu’s Method
• Thresholding:
– Simplest methods for image segmentation
– Assign _ every pixel above a defined intensity (threshold value) as one type of region (foreground),
whereas every pixel below that intensity as another type of region (background).
– Two types of thresholding :
• Global Thresholding: only one threshold value for entire image
• Local Thresholding: different value for different regions
• Otsu’s Method
– Used to find the threshold value for segmentation
– Based on selecting the lowest point between
two classes (peaks).
17
Morphological Operations
• Morphological operation:
– A broad set of shape-based image processing operations.
– The value of each pixel in the output image is based on a comparison of the corresponding
pixel in the input image with its neighbors. By choosing the size and shape (Circular disks,
crosses, diamonds, lines and squares etc.) of the neighborhood, a morphological operation
can be constructed that it is sensitive to specific shapes in the input image.
• Four basic morphological operations
– Morphological Dilation : to fill holes and close gaps in lines
– Morphological Erosion : for shrinking an object in the image and removing isolated pixels
– Morphological Opening : application of erosion followed by dilation (used to remove small
objects)
– Morphological Closing : application of dilation followed by erosion (used to close small holes
in an objects) 18
Morphological Operations
• Morphological operation:
– A broad set of shape-based image processing operations.
– The value of each pixel in the output image is based on a comparison of the corresponding
pixel in the input image with its neighbors. By choosing the size and shape (Circular disks,
crosses, diamonds, lines and squares etc.) of the neighborhood, a morphological operation
can be constructed that it is sensitive to specific shapes in the input image.
• Four basic morphological operations
– Morphological Dilation : to fill holes and close gaps in lines
– Morphological Erosion : for shrinking an object in the image and removing isolated pixels
– Morphological Opening : application of erosion followed by dilation (used to remove small
objects)
– Morphological Closing : application of dilation followed by erosion (used to close small holes
in an objects) 19
3D Segmentation (Noise Removal)
20
Intensity-based thresholding method
3D Parameter
Measuring
and 3D
Visualization
3D
Segmentation False Positive Feature
(Noise Reduction Extraction
Removal)
2D Parameter
Measuring
and Internal
Structure
Visualization
23
False-positive reducing
24
Volume and average HU values -based thresholding method
Step 1, Calculate the required threshold values ( and ) using following equation, and
regarded with 0.03 and 35 as consideration of the relationship between volume and
HU of kidney stone.
– is the average HU for each object, n is the number of voxels in an object. µ is the HU value of each voxel in an object.
Step 2, Segment the image into two groups of pixels and without using, and
•
Step 3, Develop the new image with , after elimination group.
25
Proposed Methods
3D Parameter
Measuring
and 3D
Visualization
3D
Segmentation False Positive Feature
(Noise Reduction Extraction
Removal)
2D Parameter
Measuring
and Internal
Structure
Visualization
26
Geometric Features
Localization 100 500 120
1 1 0
Area
M 1 1 0
Volume
I۪ M=R 900 800 0
∑ µ𝑖
𝑖 − 1
𝐻𝑈 𝑎𝑣𝑔 =
𝑛
Diameter
HU Density
Stone
𝐻𝑈 Heterogeneity Index (as SD)
𝑎𝑣𝑔
𝐻𝑈𝐷 =
𝐷
3D Parameter
Measuring
and 3D
Visualization
3D
Segmentation False Positive Feature
(Noise Reduction Extraction
Removal)
2D Parameter
Measuring
and Internal
Structure
Visualization
29
Largest Slice Selection
feature extraction
900 800 200
900 800 200
II 700
700
600
600
180
180
11 11 00
M 11 11 00
00 500
500 00
I۪ M=R 900
900 800
800 00
700
700 600
600 00
31
2D Internal Structure Visualization
•Step 1. Set HU= (x1, x2, x3, …, xi), xi is represented the HU values of each pixel in the specified
region.
•Step 2. For i=1to i,
if ( xi > 3000) out =8;
else if ( xi > 2500 && HU <= 3000 ) out =7;
else if ( xi > 2000 && HU <= 2500 ) out =6;
else if ( xi > 1500 && HU <= 2000 ) out =5;
else if ( xi > 1000 && HU <= 1500 ) out =4;
else if ( xi > 500 && HU <=1000 ) out =3;
else if ( xi > 200 && HU <= 500 ) out =2;
else out=1; * One of new contribution of the research
End
End
•In here, 1,2,3,..,8 is described as the eight different colors to represent each pixel intensity range.
•Step 3. Display with each value of pixels with 1,2,3,…,8. 32
Proposed Methods
3D Parameter
Measuring
and 3D
Visualization
3D
Segmentation False Positive Feature
(Noise Reduction Extraction
Removal)
2D Parameter
Measuring
and Internal
Structure
Visualization
33
3D Parameter Measurements 100
100
500
100
900
500
120
500
800
120
120
200
100 500900 120800 200
I
100 500900 120800
700
200
600 180
I
100 500 120
I 900 800
700
200
600 180
feature extraction I
0 1 0
700 600 180
I
0 1 0
700 600 180
0 1 0
1 1 0
M0 1 1 0 1
1
0
1 0
M0 1
1
0
1 1 0 1 0
1 1 1 0 1 0
900 800 200
M 1 1 0
100 500 120
M
100 500 120
900 800 200 0 500 0
1 1 0
M
900 800 200
I 0 500 0
I۪M=R
M=R
700 600
0 180
1 0
100 500 120
1 1 0
I 700 600 180
0 500 0
I۪
100 500
900 120
800 200
0
1
1
1
0
0
900 800 0
M
I۪ M=R
I 900 800
700 200
600
0 180
1
1
0
1
1
1
0
0
0 500900 0 800 0
I 700 600 180
0 500900 0 800 0
I۪M=R
M=R
1 1 0
M 0 1 0
1 01 0
500 0 700 600 0
M 0 500 0
I۪ M=R
I۪
1 1 0
0 1 0
1 0 900 800 0 900 800700 0 600 0
I۪ M=R
0 500 0
I۪ M=R
M 1 1
0
0
900
500
800
700
0
0
600 0
900 800
700
0
600 0
700 600 0
700 600 0
I۪ M=R 0 500
900 0
800 0
700 600 0
I۪ M=R 900 800
700 0
600 0
700 600 0
34
Proposed Methods
• Iso-surface generation techniques using Marching Cubes
Algorithm for 3D reconstruction
Isosurface generation is a technique that visualizes the 3D volume
datasets using intermediate polygonal based representation
Marching cubes, Dividing Cubes and Dual Contouring
• Marching Cubes Algorithm
Employ a divide and conquer approach
Operates on two basic steps: detection of active cells and provoking
the triangles inside each active cell
35
Marching Cubes Algorithm
• Input Data
Input (voxelization representing a scalar field v=f(x,y,z))
isovalue (threshold value) to classify samples (inside/outside the surface)
• Function
Generate : Geometry (vertices ) and Topology (triangles connecting these
vertices)
1 for vertices
• Output Surface
Isosurface joining all points with the
choosen isovalue.
36
Input (CT) Image
Measuring CT values
3D Auto Measurement
Measuring CT values
Hypodense and Isodense
(Average HU, Area, (Average HU, Area, Volume,
2D Auto Measurement
Region Removing
Diameter, Density)
Feature Extraction
Diameter, Density)
Unneeded Hyperdense
Region Removing Stone Heterogeneity Index Stone Heterogeneity Index
Segmented Object
(ROI)
Result of after Unneeded high intensity Result of after false positive removing
region removing
39
Results Analysis 1
(Pre-Processing)
• After loading 500-600 slices of each patient, it is constructed the original image which consists of
over 500000 labels.
• In the first process, the algorithm can remove all regions having hypodense and isodense structure
from the abdomen CT image and reduce the unwanted objects up to 4300 labels on average from
over 500000 labels.
• In the second process, hyperdense region removing, the algorithm removes the bony skeleton, bed
mat, and some bones but the average of the remaining labels is about 112.
• The final output is clear in visualization and remains only an average of 21 labels in an image.
• So, it can remove most of unwanted objects.
where, TP and FN are used for kidney stones which are correctly detected and not detected in system,
TP FN Sensitivity
66 3 95.7%
Could carry out the unneeded regions removing with a significant stone detecting.
41
Outcomes
(Pre-Processing)
• The proposed method, unwanted region removing, gives good support in kidney stone
detection, and correctly remove the unneeded region and provide the output image in a
clear form.
• Because of using thresholding methods based on the nature of kidney stones on CT
imaging, the proposed scheme is simple and easy to understand for kidney stone
classification.
• The result is good and satisfactory.
• According to the analysis, the proposed model could detect the kidney stones with 95.7%
in sensitivity.
• Therefore, the performance of the proposed scheme is effective and efficient with good
sensitivity. 42
Experimental Results
(Manual Parameter Measurements)
Area
Diameter
Average HU
SD (stone heterogeneity index, SHI)
Variation Coefficient of Stone Density (VCSD)
43
Density (from average HU, Average HU/diameter)
Experimental Results
(2D and 3D Parameter Measurements)
45
Experimental Results
(2D and 3D Parameter Measurements)
46
Results Analysis
(2D and 3D Parameter Measurements)
Table III. Summarization the Results of Each Analysis of the Proposed Study 47
Outcomes
(2D and 3D Parameter Measurements)
• Can choose the largest slice among all input slices for 2D Measurement.
• Can give the CT information (parameters) accurately and automatically for
both 2D and 3D measurement.
• Can measure in any circular shape and irregular shape because of pixel-based
measurement (manual with ellipse or circle tools, shape-based measurement).
• Can give CT information for all stones in a patient simultaneously.
• More robust, more accurate, less time-consuming
• Can reduce the variability which can give better information for diagnosis
• Can provide the clear output with 3D visualization that could rotate in any
direction
• Can display the internal structure of the stone with pixel intensity revealing 48
Expected Benefits of research
49
Limitations
• More robust 3D parameter (slightly difference compare with 2D
measuring)
• Impossible to get 3D ground truth data in Myanmar (only 2D ground truth
data)
• Produce only the CT parameter measuring because of ground truth
dataset limitation.
• Require 2.2 GHz for CPU and 2 GB of GPU to speed up the processing time
and produce 3D images quickly.
50
Further Extension
• It is envisaged that more CT kidney stone datasets with more ground truth data
categories will be collected in the future to assist treatment type classification
in addition to CT information.
• If more ground truth data and CT data sets can be collected, more accurate
treatment decisions can be made for urologists.
• By using CT images, not only stones in the urinary system, but also stones in
other organs in the human body, such as gallstones, can be carried out based
on this research system, and 3D automated system models can be continued.
51
Conclusion
• Seven types of parameters as CT information that can speed up the diagnosis process were
presented.
• The stone region was automatically selected without any noise.
• The largest slice was also automatically selected using automatic 2D stone parameter
measurements.
• The proposed study was able to show 3D visualization and 2D internal structure visualization
using automated computer system to display the stone composition.
• By using the proposed system, CT can be used to accurately and automatically measure any
shape, reduce variability, and provide clear output with 3D visualization that can rotate in any
direction.
• Believe that the research work can provide to assist in diagnosing the stone problem and
making medical treatment decisions without the help of an expert person.
52
References
[1] M. S. C. Morgan and M. S. Pearle, “Medical management of renal stones,” Bmj, vol. 2005, p. i52, 2016.
[2] YLLs, “WHO-Health metrics_GBD profile: Myanmar,” vol. 2010, no. Gbd, pp. 1–4, 1990.
[3] “SUPPORT LIFE EXPECTANCY RESEARCH!,” WORLD HEALTH ORGANIZATION 2017, 2017. [Online]. Available:
http://www.worldlifeexpectancy.com/myanmar-kidney-disease.
[4] Shabaniyan, T., Parsaei, H., Aminsharifi, A. et al. An artificial intelligence-based clinical decision support system for
large kidney stone treatment. Australas Phys Eng Sci Med 42, 771–779 (2019).
https://doi.org/10.1007/s13246-019-00780-3
[5] Zhamshid Okhunov, M.D., Arvin K. George, M.D., Arthur D. Smith, M.D., and Zeph Okeke, M.D., Arthur Smith Institute
for Urology, Hofstra North Shore-Long Island Jewish School of Medicine, New Hyde Park, New York
http://dx.doi.org/10.1016/j.urology.2012.10.086
[6] W. Finch, R. Johnston, N. Shaida, A. Winterbottom, and O. Wiseman, “Measuring stone volume - Three-dimensional
software reconstruction or an ellipsoid algebra formula?,” BJU Int., vol. 113, no. 4, pp. 610–614, 2014.
[7] HelenW. Cui, Mafalda D. Silva, AndrewW. Mills, BernardV. North and BenjaminW.Turney, “Predicting shockwave
lithotripsy outcome for urolithiasis using clinical and stone computed tomography texture analysis variables,”
Natureresearch, 11 October 2019.
53
References (cont’d)
[8] Dushyant V Sahani, MD, “Kidney Stone Diagnosis and Management.,” Department of Radiology, Massachusetts General
Hospital., vol. 27, no. 2, pp. 162–7, 2013.
[9] Ebrahimi, S., & Mariano, V.Y. (2015). Image Quality Improvement in Kidney Stone Detection on Computed Tomography
Images. Journal of Image and Graphics, 3.
[10] Oktay C, Çoraplı M, Tutuş A. The usefulness of the Hounsfield unit and stone heterogeneity variation in predicting the
shockwave lithotripsy outcome. Diagn Interv Radiol. 2022 May;28(3):187-192. doi: 10.5152/dir.2022.20945. PMID:
35748199; PMCID: PMC9634939.
[11] Krechetova, K., Glaz, A., Platkajis, A. (2008). 3D Medical Image Visualization and Volume Estimation of Pathology Zones.
In: Katashev, A., Dekhtyar, Y., Spigulis, J. (eds) 14th Nordic-Baltic Conference on Biomedical Engineering and Medical
Physics. IFMBE Proceedings, vol 20. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-69367-3_142
[12] Yamashita, S., Kohjimoto, Y., Iwahashi, Y., Iguchi, T., Nishizawa, S., Kikkawa, K., & Hara, I. (2018). Noncontrast Computed
Tomography Parameters for Predicting Shock Wave Lithotripsy Outcome in Upper Urinary Tract Stone Cases. BioMed
Research International, 2018. https://doi.org/10.1155/2018/9253952
[13] J. Y. Lee, J. H. Kim, D. H. Kang, D. Y. Chung, D. H. Lee, and H. Do, “Stone heterogeneity index as the standard deviation of
Hounsfield units : A novel predictor for shock- wave lithotripsy outcomes in ureter calculi,” Sci. Rep., vol. 6, no. 23988, pp.
1–7, 2016, doi: 10.1038/srep23988. 54
Achievements
1. Lai Yee Myint, Su Su Maung, and Khine Thin Zar, "Removal of Unwanted Object in 3D CT Kidney Stone
Images and 3D Visualization," 2020 24th International Computer Science and Engineering Conference
(ICSEC), 2020, pp. 1-5, doi: 10.1109/ICSEC51790.2020.9375155.
2. Lai Yee Myint and Khine Thin Zar, “Automated Size Measurements of Kidney Stone on 3D CT Image,”
Mandalay Technological University Journal of Science, Engineering and Technology, Volume 8, December
2021, pp. 348-352.
3. Lai Yee Myint and Khine Thin Zar, “Modify HU Density Estimation for Urinary Calculi on 3D CT Image,”
Engineering and Technology, Yangon Technological University, December 2022.
4. Lai Yee Myint and Khine Thin Zar, “Automated Measurement of Stone Heterogeneity Index and
Variation Coefficient of Stone Density on CT Image,” Journal of Engineering Technology and Applied
Physics, Vol 5 No 1 (2023), pp. 54-62, https://doi.org/10.33093/jetap.2023.5.1
55
THANK YOU VERY MUCH FOR
YOUR ATTENTION !!
56