Gist: A Mobile Robotics Application of Context-Based Vision in Outdoor Environment

Gist: A Mobile Robotics
Application of Context-Based
Vision in Outdoor Environment
Christian Siagian
Laurent Itti
Univ. Southern California, CA, USA
Outline
 Mobile robot localization
 Biological approach to vision
 Gist model
 Testing and results
 Discussion and conclusion
Mobile Robot Localization
 Where are we?
 Localization
=
identifying
landmarks
Mobile Robot Localization
 Indoors: strong assumptions of flat walls, narrow hallways, and solid angles
• Ranging sensors (laser and sonar) for mapping
 Outdoors: less conforming set of surfaces
• Ranging sensors are less effective, vision is better
Robot Vision Localization
 Object-based Vision Localization
 Objects as landmarks
 Region-based Vision Localization
 regions as landmarks
 Scene-based Vision Localization
 Scenes as a whole as Landmarks
 Color histograms [Ulrich
and Nourbakhsh 2000]
 Fourier Transform
[Oliva & Torralba 2001]
 Wavelet pyramids
[Torralba 2003]
 Histogram of Dominant
features [Renniger &
Malik 2004]
Gist
 Definition and background
 Essence, holistic characteristics of an image
 Context information obtained within a eye saccade
(app. 150 ms.)
 Evidence of place recognizing cells at
Parahippocampal Place Area (PPA)
 Biologically plausible models of Gist are yet to be
proposed
 Nature of tasks done with gist
 Scene categorization/context recognition
 Region priming/layout recognition
 Resolution/scale selection
Human Vision
Architecture
 Visual Cortex:
 Low level filters,
center-surround, and
normalization
 Saliency Model:
 Attend to pertinent
regions
 Gist Model:
 Compute image
general characteristics
 High Level Vision:
 Object recognition
 Layout recognition
 Scene understanding
Gist Model
 Utilize the same Visual Cortex raw features in
the saliency model [Itti 2001]
 Gist is theoretically non-redundant with Saliency
 Gist vs. Saliency
 Instead of looking at most conspicuous locations
in image, looks at scene as a whole
 Detection of regularities, not irregularities
 Cooperation (Accumulation) vs. competition
(WTA) among locations
 More spatial emphasis in saliency
 Local vs. global/regional interaction
Gist Model
Implementation
 V1 Raw image feature-Maps
 Orientation Channel
• Gabor filters at 4 angles
(0,45,90,135) on 4 scales
= 16 sub-channels
 Color:
• red-green and blue-yellow
center surround each with
6 scale combinations
= 12 sub-channels
 Intensity
• dark-bright center-surround
with 6 scale combinations
= 6 sub-channels
= Total of 34 sub-channels
Gist Model Implementation
 Gist Feature Extraction
 Average values of predetermined grid
Gist Model
Implementation
 Dimension Reduction
 Original:
34 sub-channels x
16 features
= 544 features
 PCA/ICA reduction:
80 features
• Kept >95% of variance
Gist Model
Implementation
 Dimension Reduction
 Original:
34 sub-channels x
16 features
= 544 features
 PCA/ICA reduction:
80 features
• Kept >95% of variance
 Place Classification
 Three-layer neural
networks
System
Example
Run
Testing & Results
 Site selection:
 Different challenges appearance-wise
 Variability in area covered/ path
lengths
 Various lighting conditions
 Single-view filming
 Clean break between segments
 Scalability: combine all sites
Map of Experiment Sites
Site 1: Building Complex
Site 1 Experiment
Input Image Gist Feature-vectors
System Output PCA/ICA reduced features

Site 1 Results
Output Label
Assigned
Label
Site 2:Vegetation-filled Park
Site 2 Result
Output Label
Assigned
Label
Site 2 Experiment

Site 3: Open Field Park
Site 3 Experiment

Site 3 Result
Output Label
Assigned
Label
Combined Sites Result
Discussion & Conclusion
 Result of current model:
 Success rate between 82.48% and 87.93%
 Combined rate of 85.96%
 4.73% error in inter-site classification
 Integrating saliency for robot navigation
 Localization within segment
• Identifying discriminating cues in the environment
• Issues in object-based systems still applies
 Bad view detection
• Foreground objects sometimes occlude whole view
 Obstacle avoidance, exploration, etc.
Discussion
 Integration of gist and saliency in general
 Single representation of both models
 Influence of saliency to gist and vice versa
• Involvement of saliency in improving gist estimation
• Gist helpful in identifying/filtering salient location
 Testing the limits of Gist: psychophysics
experiments
• Change blindness test for large scale layout changes
• Varying exposure time
• Isolation of bottom up - top down influences

Gist: A Mobile Robotics Application of Context-Based Vision in Outdoor Environment

Uploaded by

Copyright:

Available Formats

Gist: A Mobile Robotics Application of Context-Based Vision in Outdoor Environment

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Gist: A Mobile Robotics Application of Context-Based Vision in Outdoor Environment

Uploaded by

Copyright:

Available Formats

Gist: A Mobile Robotics

System Output PCA/ICA reduced features

System Output PCA/ICA reduced features

System Output PCA/ICA reduced features

You might also like