Automatic image analysis of plant root structures

zyxw zy zyxwvu Automatic Image Analysis of Plant Root Structures * Qian Huang, Ani1 I<. Jain, George C. Stockman Pattern Recognition a n d Image Processing Laboratory Department of Computer Science and Alvin J . M.Smuc1;er Department of Crop arid Soil Sciences h4 i chi ga 11 Stat e University East Lansing, hII 4SS24-1037 zyxwvutsr zyxwvutsrqp zyxwvutsrqp zyxwvutsr zyxwvutsrqp zyxwvutsrq zyxwv Abstract is presented in t,liis paper. The goals of such a system a.re to accurately identify all the roots in the image, to classify different types of spatial interactions among r o o k , aiid to generate a semantically consistent, description of the root network at different levels of a.bstraction. T h e nzaiiirhzzotron technaque has proelided ayrzczilt u r a l sczeatists t h e opportuiiaty of obserziiiig 7 h i : o sphere activztaes w z t k o u t destroying loot s t r u c t u r e s N o n e t h e l e s s , t h e laborzous analysts of t h e d a t a still prohabats zts wade applacataoiis A d v a n c e d i m a g e t i n dersiaiidaiig techiizques are needed t o derive s a i i s f c i c l o r y descraptions of p l a n t root iieiiuorks a n a n e f i c r e n t orrd robust way. This p a p e r p r e s e n t s a plnni roof image analysis s y s t e m deszgned a s a blackboard n i c h i l e c i u r e wath a h i e r a r c h y of d a t a a b s t r a c t i o n s Iniporiunt propertzes of p l a n t roots are used t h r o u g h o u t t h e processiiig a n d niultzple s o u r c e s of tnforinaiaon are combiaerl t o resolve uii cert aaiit zes an z m o g e an i e r p ret aiz o i i Ez perzinental results from s o m e s t a g e s of t h e r e a ~ a r c l i arc g i v e n whach s u p p o r t t h e overall processing s c h e m e 1 Morphological Properties of Roots 2 The physiological properties of roots play a very important, role in this recognition task. A plant root of any specie appears t o be mostly straight ( l i n e a r i t y , to have symmetric edges on both sides ( s y m m e t r y , and to be long and thin ( r a t i o of l e n g t h t o w i d t h > a known value). Unfortunately, in practice, the observaiic,e of these distinct properties of roots depends on tlie qua.lity of digitized images. Important information may get lost, in the process of sampling or quantization. Our task is, ideally, to ca.pture these properties through 1) identifying existing root properties, 2) recovering whatever is lost in the process of imaging by collecting supportive evidence from multiple sources of inforina.tion, and 3) deriving conclusions by hypothesis generation and testing. Ba.sed on the morphological properties of roots and the minirhizotron technique, the following general assumptions a.re made: 1 Introduction Plant root resea.rch provides agricultural and ecological scientists with infornmtioii which delineat,es inany of the dynamic interactions between palnt and soil. A root network can be described by a set. of features computed from individual rook such as leii@h, width, volume, branch frequency, and certain relat,ioiiships specifying how roots interact wit,li each ot#lier. The ultimate goal of root resea.rch is t.0 ma.simize the plant and soil productivity through improved soil modifications and genetically engineering root geometry and plasticity. The iniiiirliizotron technique, a non-destructive sampling method, allows scient,ist#sto o5serve rhizosphere activities through a transparent tube in the soil. Images of plant root,s a.re recorded by inserting a. small video ca.mera into tlie tube [ 5 ] . The advantage of collecting underground inforilia.tion electronically does not ease the cont.inuing prol,lems of the laborious analysis a.nd interpretat,ion of these images. Advanced automatic image processing techniques are, therefore, needed t,o efficient,ly derive satisfactory descriptions of root networl~. An ai1t.omastic image analysis system for plant, root, net,worlts e Each root segment can be approximated as a uniform cylinder. e The gray level of root pixels is higher (brighter) than that of background pixels. Let I,. x,y> represent the intensity of root pixels and I ~ [ . Ey), represent tlie intensity of background (consisting of soil, cracks, insects, etc.) pixels. Both I,.(x, y) and I b ( 2 , y) are random variables and G(ub, with Gaussian distributions G(u,., 0,") gt), respectively. zyxwvutsrqp 3 blackboard architecture, consisting of three major 569 0-8186-2915-0/92 $3.00 0 1992 IEEE Overall System Design The root image analysis system is designed as a 'This work is supported in part by an NSF grant C'DA8806599 a i d by a grant from USAID/lIT..\/RISU. componentjs: the blackboa.rd data structure, a. set of knowledge sources, and the corresponding cont,rol mechanism. This design is illustra.ted in Figure 1. The blacliboard data structure is a hierarchy of representations at different levels of abst,ractions. While each level of the hierarchy provides a partia.1 solution space, the whole hierarchy forms a complete solut,ioii space for our problem. A more detailed descript,ion of each level is given below: post t,he hypothesis along wit,li t,he parameters on the cont.rol blackboard. During t.he process of evolving a hierarchical representa.tion, many measurements of different levels are computed simultaneously. When the root network representation is established, all the features describing tmheroot network can be accessed by the user to mswer various types of questions related to the plant root network. This paper presents the results of extracting both regional and boundary information from input images. Other stages of the system will be discussed in future publica.tions. zyxwvu zyxwvutsrqpo Pixel level: input image F(x, y), initial boundary map B(x, y), and initial region segmented iina.ge R(x, Y). 4 2D level: connected components CC, polyline representation P L , curve representation C, neighborhood gra.ph of connected component.s and curves N G , symmetric pa.irs of boundary segments S P , 2D ribbons, aad closed boundaries of roots. Extracting Boundary Segments Two important features of roots are 1) edge symmet r y , and 2) elongated shape. To detect the symmetry, it is often preferable to have either reliable regional or boundary information. Boundary, region, and other features computed from data are combined to extract sat,isfactory representations of roots. The following subtasks are related to the current goal: 1) estimating bimodal Gaussian parameters, 2) forming polyline representat,ion P L , 3) deriving curve representation C, 4) esta.blishing region and boundary neighborhood graph N G , 5) linking broken root edges, and 6) evalua.ting the performance. 24D level: volumetric representat,ion of roots. Network level: attributed gmph of the root. network. zyxwvu zyxwvutsrqpon zyxwvutsrqponm zyxwvutsrq zyxwv zyx zyx zyxwvutsrq zyxwv The lowest level of the data hierarchy is a. digitsali n age of roots at pixel level and the highest level is a, root network represented as an attributed graph a.t a semantic level. An attributed graph is defined as G =(V,E ) , where V is a set of vertices, represent,ing individual objects of iiit,erest, and E is a. set of directed arcs e k =(q,vj), denoting a relationship between objects vi and u j . All the entities in G (bot,li vi’s and ej’s) can have their own associated sets of semantic attributes. For example, a root vertex can have attributes of length and a pointer to its intersecting root, while an arc can have the attribut,e xouer indicating the existence of a crossover relationship. The set of knowledge sources is a nat4ura,lpart,ition of the task of root network recognition. Each kno\vledge source is separate and independent, cont,ributing to a particular partial solution whenever t,he opportunities arise. A knowledge source usua.lly solves one simple problem using specific doma.in knowledge and the information available a t all levels of t,he d a h liierarchy in order to overcome the ainbiguit,ies that arise due to the poor quality of the input image. The results generated by any knowledge source will be post.ed on the blackboard so that they can be ut,ilized by ot,hers. The activations of knowledge sources will be controlled through a set of monitors which respond to the changes on the blackboard. The focus of a,ttention can be determined dynamically in terms of the current, st.atus of the data on the blackboard. A single focus of attention may activate a set of knowledge sources simultaneously in a parallel computing environmentl a n d may form a priority list in a sequential computing environment. A monitor will act on certain changes on the blackboard and trigger corresponding hypothesis generators based on the types of inforiimtion ga.tlierecl. Hypothesis generators will collect evidence from multiple sources to obtain associated pa.ra.ineters and t,lien E s t i m a t i n g bimodal Gaussian parameters A bimodal Gaussian density function is fitted to the intensity histogram of an input image [l] to estimate u.b, CT?,u r , and U , ’ , representing the mean and variance of the background and root intensity distributions, respectively. If the bimodality test is passed, a threshold T is determined using the Bayes decision theory to minimize the classification error. The threshold T is a.pplied to F ( z ,y) to obtain the initial region-based segmentation R ( z ,y). A labeling algorithm then computes the set CC = {CCi} of connected components from R(2 ,y) . Fonniiig p o l y l i n e representation An initia.1 edge map E ( z , y) is derived from Canny operator, defining the initial boundary segmentation of an input i1na.g-e. The Freeman chain code is obtained by applying an edge tracing algorithm 21. This algorithm uses desired properties of roots s u d as linearity as heuristics whenever multiple choices of tracing direction (ambiguities) are present. Root boundaries are often broken due to many reasons. Recovering the connectivity is done in two stages. Small gaps (one pixel a.part) are filled in the process of tracing a.nd they a,re filled only in a direction interval 8 k 68, where B is the desired direction computed from the effective history of tracing (previous 10 traced pixel points). The value of 6B is kept small in order to preserve the straightness of the traced segment. The lin1;ing of larger gaps will be discussed later. For every such derived chain, a set of critical points is detected using eight different “arm length” to improve the robustness [2]. These critical points define a. polyline representation P L = { L I F l C f ) }by dividing connected segments into a set of straight line Li’s, 570 zyxwvutsr zyxwvuts zyxwvu zyx zyxwvu each of which can have an associa.ted set of features F (such as orientation and length) and a. confidence measure C f . Here, Cf is a normalized weighted suin of desired feat,ures of root boundary (for example, edge strength and average intensity contrast). D e r i v i n g c u r v e representation A relaxat.ion on the detected curvatures gives another level of abstraction, curve representatioil C = {C,(:‘”}. The degree of relaxation can be tuned according to the specie of the roots that the syst8ein is currently processing. Two neighboring straight lines Lj and Li+l can be merged if the critical point between them is within the relaxation range. Tlie features a.nd confidence measure for the merged curve a.re calculated as the weighted sum of the corresponding measures from the lines to be merged. Tlie weight,s are determined dynamically as the rat,io of the leiigt81i of a straight line to the length of the merged curve. Identifying spurious edge segments is essent,ial to avoid unnecessary ambiguities. Heuristics a.re applied to accomplish the task. Examples are tmhenoise a n d soil-crack heuristics. Establishing neighborhood graph Region and boundary segmeihtions provide ‘LD shape information from different perspectives. Combining them improves the robustness of shape recognition ([3],[4]). We eshblish a neighborhood gra,ph N G = { VNC, E N G } describing the adjaa.cency relationship between connected components CCi’s and curves Ci’s. Here, V , G = CCU C and -E=,vc; = {(CCi,Cj)”kI Ci XJS~ CCj}. The positiona.1 relat,ionship X revealed in N G will be used as one source of information in later processing. Evaluatiiig the p e r f o r m a n c e A set of representative root images are chosen for test purpose. To evaluate the performance of the syst e m quantitatively, the ground truth a t pixel level for the test images is established by labeling pixels as root or non-root, classes through several panels of plant and soil scientists. Then the ground truth at other levels of representation can be abstracted. To test the system, the processing results at each stage will be evaluated against the corresponding ground truth. 5 Experiiiieiital Results Figures 2 and 3 show the region-based segmentation results for two of the five test images. An average classificat,ion accuracy of 95% is obtained using the region-based segmentation ground truth on all five test images. Although this accuracy depends on the number of points in the resultant regions, they indicate rela.t.ive performances. Misclassifications occurred in situations of noisy background or poor contrast. They will b e reclassified again based on new evidence. Figures 4 and 5 show the boundary processing results at different stages for the same two root images. From part (b) of these figures, we can see the effectiveness of the two heuristics discussed before. The connectivity of boundaries is improved after both 1) small ga.ps are filled during edge tracing and 2) larger gaps are linked through the more sophisticated linking operation (see p r t (c)). zyxwvutsr zyxwvutsrq zyxwvut 6 Coiiclusioiis The experimental results from the current stage of the research a.re encouraging. The design of the system allows not only bottom up but also top down processing which leads to a more robust system because t,lie mistalies made at one level can be corrected at another level of the processing by using newly derived evidence or knowledge. The fusion of multiple sources of iiiforina.tion allows more reliable solutions t o ambiguities occurred in image interpretation. The blackboard architecture provides both an opportunistic problem solving scheme and the flexibility of expanding the system to a, wider range of plant species. Our fra.mework tries to give a general treatment to a set. of common problems in image network understanding such as bmnching analysis, crossover analysis, etc. Therefore, the methodology developed here should also be applicable to other domains such as aiidysis of arterial systems in medical imagery. L i n k i n g b r o k e n root edges The linking operation can be described as a stat.e space search with initial open list OPEN = C and goal state OPEN = 0. A priority list defined by t,he confidence measures of Ci’s is inaintained for OPEN throughout the search process. At any moment,, tlie item with the highest priority is processed i n a. threephase paradigm: evidence collection, hypothesis generation, and hypothesis testing. Different types of evidence are collected from all the iiiforniat,ioii derived so far, including the input data, to prune tlie expansion tree. The hypotheses about linking ca.ndida.tes are generated based on the evidence collected and a set. of constraints defined to maintain the seina.ntic consistency of linked segments. All the hypotheses t,liat,1ea.d to an increase in confidence value (may be none) will compete and the one that yields the largest increase is the winner. Any item that has a descendant in t,he above process will be returned to OPEN a.fter l i n l h g with updated set of features and confidence measure. Otherwise, it will be removed from OPEN. The linking is performed by genera.ting a. cubic spline for the gap to guarantee the smoothness of the linked curve. Such added segments a.re saved in a special data structure and can be removed if any liigher level abstraction shows the incorrectmilessof the Iiiiking. Refer eii ce s Chow, C. I<. and Kaneko, T. (1972), “Automatic Boundary Detection of the Left Ventricle from Cineangiogiams,” Computers and Biomedical Research, no. 5 , pp. 388-410. Miwa, Hiiobuni and Kanade, Takeo (1988), “Contour Feature Extraction From Images: Lines and Ellipses,” CBIU- RI- TR-88-19, Robotics Institute, CarnegieMellon University. Nazif, Ahmed and Levine, Martin D. (1984), “Low Level Image Segmentation: An Expert System,” 57 1 zyxwvutsrqponm zyxwvutsrqp zyxwvutsrq zyxwvutsrqp zyxwvutsrqp I E E E Trans. o n P A M I , vol. PAMI-6, no. 5, pp. 555577 [4] Qian, Jianzhong, Ehrich, Roger W., and Caniplwll, James B. (1990), “DNESYS - An Expert System For Automatic Extraction of Drainage Networks From Digital Elevation Data,” IEEEE Trans. on Geoscience and Remote Sensing, vol. 28, no. 1, pp. 39-45. [ 5 ] Taylor, H. M. and Upchurch, D. R. (1987), “Miiiiliri- zotron Tube Advantages and Disadva.nt.ages,” Alethods and Application f o r Meastiring Root Dyn(iiiiic.s, H. M. Taylor ed., ASA Special Publication, no. 50. Madison WI. Bimodal Gaussian parameter estkation Canny “8‘ detector zyxwvutsrqpo zyxwvutsrqponmlkj Connecred cowonen‘s labeling Poiyline representarion formation Curve representation formation Neighborhood graph conslrucllon Symmetry detection Figure 2: RPsults of fitting bimodal Gaussian distribution (a) Input image, (b) Intensity histogram of (a) with the vertical line indicating threshold T , (c) Segmented image of (a) using T . Figure 3 : Results of fitting bimodal Gaussian distributtion (a) Inpiit image, (b) Intensity histogram of ( a ) with the vertical line indicating threshold T , (c) Segmented image of (a) using T . 2D ribbonfinding 2 . 5 0 tube extension formation Branching analysis Network construction Figure 4: Boundary information at various stages a Initial boundary map from C a m y edge detector, [b] Applying noise and soil-crack heuristics to (a), (c) Result of two-stage connectivity recovering. Data hierarchy Monirors I Control mechanism Figure 1: Root image analysis system diagram Figure 5: Boundary information a t various stages a Iiiitial boundary map from Canny edge detector, [b] Applying noise and soil-crack heuristics to (a), (c) Result of t,wo-stage connectivity recovering. 512