Feature Object Detection on the urban road surface by The Application of Three Line Scanner Imagery

Sompoch Pantavungkour ** Ryosuke Shibasaki

*Center of Spatial Information Science University of Tokyo, 4-6-1, Cw503, Komaba, Meguru-Ku, Tokyo, 153-8505, Japan ** Email: shiba @skl.iis.u-tokyo.ac.jp **Center of Spatial Information Science University of Tokyo, 4-6-1, Cw503, Komaba, Meguru-Ku, Tokyo, 153-8505, Japan

KEY WORDS: Three Line Scanner (TLS), Region Growing and Shape Description ABSTRACT The Three Line Scanner or TLS Technology has emerged for the few years. The merit characteristic of Three Line Scanner distributes the digital image with very high spatial resolution, few centimeters. By meanwhile, the traffic problem in mega city remains more and more severe such as Traffic congestion, Accident etc. Moreover, due to human and time-consuming process by manual vehicle counting, the automatic method for road monitoring becomes challenging research. The research focuses on development on the application of Three Line Scanner Image application for automatic road surface detection and detection of objects on the urban road. The research presents the novel and simple algorithm to detect the feature on the urban area based on the application the region growing and the shape descriptor as well as the road line reconstruction algorithm by Radon Transformation. The robust method in this experiment has been developed from the simple shape extraction such as the roads including their surface and boundary etc. The promising results is shown and discussed in this study. 1. Introduction The traffic problems in Mega-Cities around our planet such as Tokyo, New York etc remain more and more severe. This destructive problem cause not hue finial losing but much human life damage annually. Simultaneously, Remote Sensing Technology for Traffic Solution becomes practical and widespread due the Remote Sensing advantage with wide-area covering, temporal information and high spatial resolution. In U.S.A, for example, Remote Sensing Project for Traffic issue has been operated with the cooperation among several government agencies, academic institutes and also private sections under the National Transportation Policy in [4]. Recently, Three Line Scanner (TLS) is the novel technology of Digital Airborne Sensor emerging. TLS (Three Line Scanner) consists of three parallel one-dimensional CCDs mounted on the imaging plane (See Fig.1). It obtains seamless high-resolution images (5-10cm on the ground) with three viewing directions (fore, nadir, aft) simultaneously mainly to generate 3D spatial data accurately with RTK -GPS and INS and also TLS images are only taken under excellent weather at least in Urban Area.

Figure No.1 Three Line Scanner

In this paper, the image processing of TLS for road monitoring have been developed and presented. The paper is organized as follows. In Section no.2, related researches are reviewed. Section no.3 and no.4 present our framework and algorithm. The result by our approach such as Road Surface definition has been shown and discussed in Section no.5. Finally, Section no.6 outlines the conclusions and the possible further direction of this works. 2. Related Research Compared with human eye, it is no easy to distinguish the road surface and objects on road from Vertically Aerial Image by computer. A few score years, automatic Object detection from Aerial Image become challenging due to tedious time-consuming manual detection. Many researches focus on Road and on road detection from Aerial Image. The method of feature detection divided into two ways: Knowledge and Structure Bases. In [6] Zhao T.and Nevatia R. applied the Neural Network algorithm to detect the vehicle from low-resolution aerial image. Feature detection from High-resolution image has been resolved with the Structure Model by. Spiegel, M in [5]. In spite of their good result performance, all of those works, however, were applied by Frame aerial image, or high-resolution image from space borne, not image from the line sensor from Airborne. For our research, Three Line Scanner Image with very high resolution from Airborne has been used to develop algorithm for road monitoring originally. This paper proposed and discussed the algorithm of feature detection by using Three Line Scanner image in next chapters. 3. Framework The objective of this experiment is to develop the automatic algorithm of feature detection such as vehicle on street from Three Line Scanner Image. The framework has been categorized into three steps: Preparation, Single Image Processing and Stereo Image processing (see Figure no.2). At the time, only preparation and some part of Single Image Processing are discussed in this paper.

Framework of Vehicle Detection

Road Extraction - Automation

Shadow Correction Stereo Image Processing

Supporting information

Vehicle Detection

Classification (1)

Classification (2)

Final Result

Figure no.2 the framework of this study The purpose of Single Image Processing of TLS is to identify the road surface as Region of Interesting and vehicle detection. Based on the Assumption, the vehicles exist only on the road or the street in this experiment The algorithm of Road Detection and Vehicle has been discussed in Chapter 4. 4. Road Detection Surface Definition and Vehicle

To define the road surface from TLS nadir image, the algorithm has been developed. (See Figure no.3)

Region Growing Segmentation for TLS image Thresholding of Region Selection Intensity Analysis - Very Bright Feature

Thresholding of Region Selection by Shape Description Length, Width, Ratio(L/W), Orientation

Region growing is a fundamentally different approach to segmentation than brightness thesholding. It requires the user to select a point within a feature and then expands the region, adding pixels that are similar in brightness to the growing region. This method often shows a tendency to "leak out" of the feature wherever one single boundary pixel is added in error, so setting the tolerance on pixel brightness differences is very important.(John C. Russ in [2]) and all segmented regions in TLS image are labeled and the neighborhood relation among them is created. Shape Description Due to the rectangular shape of vehicle and the lane line of the road and the road surface, some Property of Shape Description aided to extract the vehicle candidate and the line on the road are the width, the length, the ratio of the length to width and the sodality of each segmented region by integrating very high brightness of the line on the road. The definition of some properties of Shape Description used in this study is descried as below: The area: The road surface is almost biggest and homogenous area with some specific bright value in the image. The area of segmented region is measure by counting the actual number of pixels in the region. The width: Based on the fixed and constant width of lane line, the width of line is less then 30 cm. and the width of the expected region as the vehicle candidate is between 100 cm and 500 cm and, thus, those parameters are converted to the image unit in a pixel per 5 cm from the resolution of TLS image in this study. The width of feature in the segmented TLS image is calculated from the bounding box, the smallest rectangle that can contain the region. The length: the length of the line on the road is defined as the minimum length of the lane line and the length of segmented region has been calculated from the bounding box as same as the region width. The ratio of length to width: this ratio is to measure the elongation of the object. The line feature on the road is of high elongation and thus this ratio of the object as vehicle candidate is less than 4 in Fuse. The sodality: The sodality of region is to measure the density of region due to the high density of the

Line Detection By Radon Transformation

Line Reconstruction Buffering the Road Surface from Line Image Profile Analzsis

Selection of the feature on Road By Line Equation Parameter

Finding Center of Gravity for all Regions

Road Surface and Object on Road

4.1 Preparation Stage The Raw image has been prepared to create the fundamental information and detect the Road Surface by some robust methods as below: Region Growing

road 1ine feature Area/ConvexArea.

by

computing

as

Surface Objects, Non Road Surface object in Road and Out -Road Features.

The centroid: Coordinates of the center of mass of the region for testing the position of object generally The brightness of the object: the brightness of all segmented regions is the crucial information to discriminate the type of the segmented region in the TLS image: Road Surface and Non-road object by analyzing automatically or semiautomatically to find the huge area of the region with its special brightness. Line Detection, Reconstruction. To determine the direction of road surface generate the boundary of the road surface and the orientation of vehicle as well as the direction of the template moving in the further processing, the Radon Transformation or Hough Transformation has been applied to find the line and obtain the parameter of the line equation. The method of line detection and the reconstruction by Hough transformation is depicted as below: Consider the parametric representation of a line: In parameter space (m,c), any straight line in image space is represented by a single point. Any line that passes through a point (p,q) in image space corresponds to the line c=mp+q in parameter space. y= mx+ c (1) To detect straight lines in an image: Quantize the parameter space (m,c) and create an accumulator array (each dimension in the array corresponds to one of the parameters). For every 1 pixel (xi,yi) in the binary image calculate c=mxi+yi for every value of the parameter m and increment the value of the entry (m,c) in the accumulator array by one.(by Michale A. Wirth in [3] ) By the Hough Transformation methods, the direction of road has been detected from the slope parameter and this parameter is applied to create the buffer area as the boundary of road surface by using the shifting parameter. Based on the surface of the street, all regions in the image is categorized into two types: On-street objects and Out-street objects by using checking the position of region centiod with the line property at boundary of Area Surface. Thus the types of object are able to discriminate three types: R oad

Set Operation To generate the masked image and get the on street road region only and gathering the information to select the Vehicle Candidate, Union, Difference and Intersection have been applied. Morphological Operation Due to many noise in On-street region image, the dilation and erosion have been applied. The noise free On-street regions have been detected at this stage. 4.2 Nadir Single Image Processing Step In the preparation step, the road surface and the On-Street feature are detected. The further processing, Single Image Processing, is to identify the vehicle candidate discussed as below: Vehicle Detection To detect the vehicle, noise-free On-street features are relabeled. Some are selected to be the candidate of vehicle by the special criteria. The criteria has been developed from the geometric constrain of vehicle, shape description by following 1. 2. 3. The vehicle is on only the road surface. The direction of vehicles is parallel with the road detected from charter 3. 2 The size of vehicle is during 4.8 m and 80 2 m

This criterion is able to detect almost regions on the street. (See Figure no.5) The regions detected

from algorithm will be identified with the templates that are defined as vehicle in the further research.

Creating the On-Street Feature Image by merging the Out-Street and Road Surface

Tokyo. This image is distributed by STARLABOCo.Ltd. Therefore, the spatial resolution is 5 centimeter. Due to the huge volume of TLS image, the entire image is divided into

Masking Image with On-street Regions

Reducing the Noise by Morphology: Dilation and Erosion

Relabeling regions in the Masking Image

Selecting the Vehicle Candidate based on some criteria 1. Location: Only the road 2. Direction: Parallel with road 3. Size: 4.8 m 2 to 80 m 2 4. Width: More than 1 m

Figure no.6 Nadir TLS image above the study area tested with new algorithm in this study

Vehicle Candidate
Figure no.5 Procedure of Vehicle Candidate Searching 5. Experiment The algorithm has been verified by Three Line Scanner Image taken on February, 26 2001 at 13:00 at 500-meter altitude in Ichagaya Ward, i) Selected Vehicle Candidates Figure no.6 (conts.) several portions. The algorithm has been developed on Matlab. The Figure no.6 shows the

road surface boundary detected and the segmented regions on and out of the road surface. Moreover, the vehicle candidates is able to detect by the algorithm described in Figure no.6 6. Conclusion This paper presented the promising result of automatic object detection on the road by using TLS image, high resolution Airborne Image. Specially, the roads and their orientation have been detected because they are vital evidence as the Region of Interesting of Vehicle Location Recommendation and further research Now the vehicle detection from Nadir TLS image is developed by using the road surface and their direction as the Region of Interesting to detect the vehicle. The vehicle identification will be solved by Deformable Template Matching soon. Acknowledgement The authors would like to thank you for STARLABO Co.Ltd and their staffs such as Mr. Morita, Dr. Chen and Mr. Miyake as research coordinator for TLS images supporting kindly including the solving of TLS technical problem. Moreover, we would like to thank specially Mr. Nakagawa for his programming suggestion to make this study provide the promising results. Reference: 1. Fuse T.,Yoda H., Shimizu E.,Tsutsumi M., Auto-Recognition of Vehicle Maneuvers Based on Spatio-Temporal Clustering with Images from High Altitude Platforms, 2001,3rd International Image Sensing Seminar on New Development in Digital Photogrammetry, Gifu, Japan, 24-27 September 2001 http://www.chikatsulab.g.dendai.ac.jp/wgv4/presentation/07_04Fuse. PDF 2. John C. Russ, Inc, 2001, Tutorial: The Image Processing and Measurement Cookbook, Reindeer Graphics, http://www.reindeergraphics.com/tutorial/index.ht ml

Guelph , http://hebb.cis.uoguelph.ca/~mwirth/Teaching/CIS 6320/CIS6320.html 4.National Consortium, 2002 National Consortium on Remote Sensing in Transtation, Research Project, www.ncgia.ucsb.edu/ncrst 5. Spiegel, M, 2000,Automatic detection of vehicles from digital aerial imagery by neural networks, Diploma thesis, the Technical University Munich, Munich German, http://www.photo.verm.tumuenchen.de/site_lpf/site_new/site_eng/team_mit arbeiter.htm 6. Zhao T., Nevatia R., 2001, Car Detection in Low Resolution Aerial Image, ICCV 2001, http://iris.usc.edu/Outlines/papers/2001/tao_car_ic cv-01.pdf

3. Michael A. Wirth, 2001 CIS*6320 Lecture note on Image Processing Algorithm and Applications, Winter 2001, Biocomputing Group, Department of Computing and Information Science, University of