The paper presents a review of 200 references in content-based image retrieval. The paper starts ... more The paper presents a review of 200 references in content-based image retrieval. The paper starts with discussing the working conditions of content-based retrieval: patterns of use, types of pictures, the role of semantics, and the sensory gap. Subsequent sections discuss computational steps for image retrieval systems. Step one of the review is image processing for retrieval sorted by color, texture, and local geometry. Features for retrieval are discussed next, sorted by: accumulative and global features, salient points, object and shape features, signs, and structural combinations thereof. Similarity of pictures and objects in pictures is reviewed for each of the feature types, in close connection to the types and means of feedback the user of the systems is capable of giving by interaction. We briefly discuss aspects of system engineering: databases, system architecture, and evaluation. In the concluding section, we present our view on: the driving force of the field, the heritag...
Page 1. Similarity Indexing with the SS-tree * David A. White Ramesh Jain dwhite0cs.ucsd. edu jai... more Page 1. Similarity Indexing with the SS-tree * David A. White Ramesh Jain dwhite0cs.ucsd. edu jain0ece.ucsd.edu Visual Computing Laboratory University of California, San Diego 9500 Gilman Drive, Mail Code 0407 La Jolla, CA 92093-0407 ...
Page 1. Similarity Indexing: Algorithms and Performance David A. White Ramesh Jain [email protected]... more Page 1. Similarity Indexing: Algorithms and Performance David A. White Ramesh Jain [email protected][email protected] Visual Computing Laboratory University of California, San Diego 9500 Gilman Drive, Mail Code 0407 La Jolla, CA 92093-0407 Abstract ...
The need for content-based access to image and video information from media archives has captured... more The need for content-based access to image and video information from media archives has captured the attention of researchers in recent years. Research e orts have led to the development of methods that provide access to image and video data. These methods have their roots in pattern recognition. The methods are used to determine the similarity in the visual information content extracted from low level features. These features are then clustered for generation of database indices. This paper presents a comprehensive survey on the use of these pattern recognition methods which enable image and video retrieval by content. ?
ACM Transactions on Multimedia Computing, Communications, and Applications, 2006
Extending beyond the boundaries of science, art, and culture, content-based multimedia informatio... more Extending beyond the boundaries of science, art, and culture, content-based multimedia information retrieval provides new paradigms and methods for searching through the myriad variety of media all over the world. This survey reviews 100+ recent articles on content-based multimedia information retrieval and discusses their role in current research directions which include browsing and search paradigms, user studies, affective computing, learning, semantic queries, new features and media types, high performance indexing, and evaluation techniques. Based on the current state of the art, we discuss the major challenges for the future.
Effective and efficient tools for segmenting and content-based indexing of digital video are esse... more Effective and efficient tools for segmenting and content-based indexing of digital video are essential to allow easy access to video-based information. Most existing segmentation techniques do not use explicit models of video. The approach proposed here is inspired and influenced by well established video production processes. Computational models of these processes are developed. The video models are used to classify the transition effects used in video and to design automatic edit effect detection algorithms. Video segmentation has been formulated as a production model based classification problem. The video models are also used to define segmentation error measures. Experimental results from applying the proposed technique to commercial cable television programming are presented.
The goal of the VIMSYS project is to create an information management system that can retrieve im... more The goal of the VIMSYS project is to create an information management system that can retrieve images or parts of images in addition to textual data. The system allows users to incrementally formulate a query starting from semantic pictorial objects. 11 allows incompletely specified and similarity-based queries. This paper presents a new layered data model and the design of the query processing unit for this system. The data model is a combination of object-oriented and functional models, and permits multiple representations of image entities. The knowledge module guides the user to progrcssively refine similarity-based queries and to query by defining new semantic objects and composing its attributes. Design considerations for the access structure of the system are also discussed, alongwith some of the problems of creating efficient mechanisms of access.
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000
AbstractÐThe paper presents a review of 200 references in content-based image retrieval. The pape... more AbstractÐThe paper presents a review of 200 references in content-based image retrieval. The paper starts with discussing the working conditions of content-based retrieval: patterns of use, types of pictures, the role of semantics, and the sensory gap. Subsequent sections discuss computational steps for image retrieval systems.
Amarnath Gupta ([email protected]) is a senior software scientist in Virage, Inc., a developer ... more Amarnath Gupta ([email protected]) is a senior software scientist in Virage, Inc., a developer and vendor of VIR systems in San Mateo, Calif.
ACM Transactions on Multimedia Computing, Communications, and Applications, 2006
Extending beyond the boundaries of science, art, and culture, content-based multimedia informatio... more Extending beyond the boundaries of science, art, and culture, content-based multimedia information retrieval provides new paradigms and methods for searching through the myriad variety of media all over the world. This survey reviews 100+ recent articles on content-based multimedia information retrieval and discusses their role in current research directions which include browsing and search paradigms, user studies, affective computing, learning, semantic queries, new features and media types, high performance indexing, and evaluation techniques. Based on the current state of the art, we discuss the major challenges for the future.
Effective and efficient tools for segmenting and content-based indexing of digital video are esse... more Effective and efficient tools for segmenting and content-based indexing of digital video are essential to allow easy access to video-based information. Most existing segmentation techniques do not use explicit models of video. The approach proposed here is inspired and influenced by well established video production processes. Computational models of these processes are developed. The video models are used to classify the transition effects used in video and to design automatic edit effect detection algorithms. Video segmentation has been formulated as a production model based classification problem. The video models are also used to define segmentation error measures. Experimental results from applying the proposed technique to commercial cable television programming are presented.
The goal of the VIMSYS project is to create an information management system that can retrieve im... more The goal of the VIMSYS project is to create an information management system that can retrieve images or parts of images in addition to textual data. The system allows users to incrementally formulate a query starting from semantic pictorial objects. 11 allows incompletely specified and similarity-based queries. This paper presents a new layered data model and the design of the query processing unit for this system. The data model is a combination of object-oriented and functional models, and permits multiple representations of image entities. The knowledge module guides the user to progrcssively refine similarity-based queries and to query by defining new semantic objects and composing its attributes. Design considerations for the access structure of the system are also discussed, alongwith some of the problems of creating efficient mechanisms of access.
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000
AbstractÐThe paper presents a review of 200 references in content-based image retrieval. The pape... more AbstractÐThe paper presents a review of 200 references in content-based image retrieval. The paper starts with discussing the working conditions of content-based retrieval: patterns of use, types of pictures, the role of semantics, and the sensory gap. Subsequent sections discuss computational steps for image retrieval systems.
Amarnath Gupta ([email protected]) is a senior software scientist in Virage, Inc., a developer ... more Amarnath Gupta ([email protected]) is a senior software scientist in Virage, Inc., a developer and vendor of VIR systems in San Mateo, Calif.
The data volume in multimedia is dominated by its digital video component. The effective manageme... more The data volume in multimedia is dominated by its digital video component. The effective management of this com-ponent is essential to the development of successful multi-media systems. Di@l Video Segmentation (the problem of decomposing video into its component units ) is one ...
IEEE Transactions on Pattern Analysis and Machine Intelligence, 1999
With complex multimedia data, we see the emergence of database systems in which thefundamental op... more With complex multimedia data, we see the emergence of database systems in which thefundamental operation is similarity assessment. Before database issues can be addressed, it isnecessary to give a definition of similarity as an operation.In this paper we develop a similarity measure, based on fuzzy logic, that exhibit severalfeatures that match experimental findings in humans. The model is dubbed Fuzzy
IEEE Transactions on Knowledge and Data Engineering, 2001
In this paper we discuss briefly some aspects of image semantics and the role that it plays for t... more In this paper we discuss briefly some aspects of image semantics and the role that it plays for the design of Image Databases. We argue that images don't have an intrinsic meaning, but that they are endowed with a meaning by placing them in the context of other images and by the user interaction. From this observation, we conclude that in an image database users should be allowed to manipulate not only the individual images, but also the relation between them. We present an interface model based on the manipulation of configurations of images. * Simone Santini and Ramesh Jain are with the Visual Computing Laboratory, University of California, San Diego, {ssantini,jain}@ece.ucsd.edu. Amarnath Gupta is with the San Diego Supercomputer Center, [email protected] • It is grounded in action. The database can estabilish the meaning of an image based on the actions of the user when the image is presented. In a database situation, the only action allowed to the user is asking a query. Therefore, the meaning of an image will be revealed to the database by interpreting the sequence of queries posed by the user 1 .
Until recently, the management of large image databases has relied exclusively on manually entere... more Until recently, the management of large image databases has relied exclusively on manually entered alphanumeric annotations. Systems are beginning to emerge in both the research and commercial sectors based on 'content-based' image retrieval, a technique which explicitly ...
Video is composed of audio-visual information. Pro-viding content based access to video data is e... more Video is composed of audio-visual information. Pro-viding content based access to video data is essential for the sucessful integration of video into computers. Organizing video for content based access requires the use of video metadata. This paper explores the nature video ...
The paper presents a review of 200 references in content-based image retrieval. The paper starts ... more The paper presents a review of 200 references in content-based image retrieval. The paper starts with discussing the working conditions of content-based retrieval: patterns of use, types of pictures, the role of semantics, and the sensory gap. Subsequent sections discuss computational steps for image retrieval systems. Step one of the review is image processing for retrieval sorted by color, texture, and local geometry. Features for retrieval are discussed next, sorted by: accumulative and global features, salient points, object and shape features, signs, and structural combinations thereof. Similarity of pictures and objects in pictures is reviewed for each of the feature types, in close connection to the types and means of feedback the user of the systems is capable of giving by interaction. We briefly discuss aspects of system engineering: databases, system architecture, and evaluation. In the concluding section, we present our view on: the driving force of the field, the heritag...
Page 1. Similarity Indexing with the SS-tree * David A. White Ramesh Jain dwhite0cs.ucsd. edu jai... more Page 1. Similarity Indexing with the SS-tree * David A. White Ramesh Jain dwhite0cs.ucsd. edu jain0ece.ucsd.edu Visual Computing Laboratory University of California, San Diego 9500 Gilman Drive, Mail Code 0407 La Jolla, CA 92093-0407 ...
Page 1. Similarity Indexing: Algorithms and Performance David A. White Ramesh Jain [email protected]... more Page 1. Similarity Indexing: Algorithms and Performance David A. White Ramesh Jain [email protected][email protected] Visual Computing Laboratory University of California, San Diego 9500 Gilman Drive, Mail Code 0407 La Jolla, CA 92093-0407 Abstract ...
The need for content-based access to image and video information from media archives has captured... more The need for content-based access to image and video information from media archives has captured the attention of researchers in recent years. Research e orts have led to the development of methods that provide access to image and video data. These methods have their roots in pattern recognition. The methods are used to determine the similarity in the visual information content extracted from low level features. These features are then clustered for generation of database indices. This paper presents a comprehensive survey on the use of these pattern recognition methods which enable image and video retrieval by content. ?
ACM Transactions on Multimedia Computing, Communications, and Applications, 2006
Extending beyond the boundaries of science, art, and culture, content-based multimedia informatio... more Extending beyond the boundaries of science, art, and culture, content-based multimedia information retrieval provides new paradigms and methods for searching through the myriad variety of media all over the world. This survey reviews 100+ recent articles on content-based multimedia information retrieval and discusses their role in current research directions which include browsing and search paradigms, user studies, affective computing, learning, semantic queries, new features and media types, high performance indexing, and evaluation techniques. Based on the current state of the art, we discuss the major challenges for the future.
Effective and efficient tools for segmenting and content-based indexing of digital video are esse... more Effective and efficient tools for segmenting and content-based indexing of digital video are essential to allow easy access to video-based information. Most existing segmentation techniques do not use explicit models of video. The approach proposed here is inspired and influenced by well established video production processes. Computational models of these processes are developed. The video models are used to classify the transition effects used in video and to design automatic edit effect detection algorithms. Video segmentation has been formulated as a production model based classification problem. The video models are also used to define segmentation error measures. Experimental results from applying the proposed technique to commercial cable television programming are presented.
The goal of the VIMSYS project is to create an information management system that can retrieve im... more The goal of the VIMSYS project is to create an information management system that can retrieve images or parts of images in addition to textual data. The system allows users to incrementally formulate a query starting from semantic pictorial objects. 11 allows incompletely specified and similarity-based queries. This paper presents a new layered data model and the design of the query processing unit for this system. The data model is a combination of object-oriented and functional models, and permits multiple representations of image entities. The knowledge module guides the user to progrcssively refine similarity-based queries and to query by defining new semantic objects and composing its attributes. Design considerations for the access structure of the system are also discussed, alongwith some of the problems of creating efficient mechanisms of access.
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000
AbstractÐThe paper presents a review of 200 references in content-based image retrieval. The pape... more AbstractÐThe paper presents a review of 200 references in content-based image retrieval. The paper starts with discussing the working conditions of content-based retrieval: patterns of use, types of pictures, the role of semantics, and the sensory gap. Subsequent sections discuss computational steps for image retrieval systems.
Amarnath Gupta ([email protected]) is a senior software scientist in Virage, Inc., a developer ... more Amarnath Gupta ([email protected]) is a senior software scientist in Virage, Inc., a developer and vendor of VIR systems in San Mateo, Calif.
ACM Transactions on Multimedia Computing, Communications, and Applications, 2006
Extending beyond the boundaries of science, art, and culture, content-based multimedia informatio... more Extending beyond the boundaries of science, art, and culture, content-based multimedia information retrieval provides new paradigms and methods for searching through the myriad variety of media all over the world. This survey reviews 100+ recent articles on content-based multimedia information retrieval and discusses their role in current research directions which include browsing and search paradigms, user studies, affective computing, learning, semantic queries, new features and media types, high performance indexing, and evaluation techniques. Based on the current state of the art, we discuss the major challenges for the future.
Effective and efficient tools for segmenting and content-based indexing of digital video are esse... more Effective and efficient tools for segmenting and content-based indexing of digital video are essential to allow easy access to video-based information. Most existing segmentation techniques do not use explicit models of video. The approach proposed here is inspired and influenced by well established video production processes. Computational models of these processes are developed. The video models are used to classify the transition effects used in video and to design automatic edit effect detection algorithms. Video segmentation has been formulated as a production model based classification problem. The video models are also used to define segmentation error measures. Experimental results from applying the proposed technique to commercial cable television programming are presented.
The goal of the VIMSYS project is to create an information management system that can retrieve im... more The goal of the VIMSYS project is to create an information management system that can retrieve images or parts of images in addition to textual data. The system allows users to incrementally formulate a query starting from semantic pictorial objects. 11 allows incompletely specified and similarity-based queries. This paper presents a new layered data model and the design of the query processing unit for this system. The data model is a combination of object-oriented and functional models, and permits multiple representations of image entities. The knowledge module guides the user to progrcssively refine similarity-based queries and to query by defining new semantic objects and composing its attributes. Design considerations for the access structure of the system are also discussed, alongwith some of the problems of creating efficient mechanisms of access.
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000
AbstractÐThe paper presents a review of 200 references in content-based image retrieval. The pape... more AbstractÐThe paper presents a review of 200 references in content-based image retrieval. The paper starts with discussing the working conditions of content-based retrieval: patterns of use, types of pictures, the role of semantics, and the sensory gap. Subsequent sections discuss computational steps for image retrieval systems.
Amarnath Gupta ([email protected]) is a senior software scientist in Virage, Inc., a developer ... more Amarnath Gupta ([email protected]) is a senior software scientist in Virage, Inc., a developer and vendor of VIR systems in San Mateo, Calif.
The data volume in multimedia is dominated by its digital video component. The effective manageme... more The data volume in multimedia is dominated by its digital video component. The effective management of this com-ponent is essential to the development of successful multi-media systems. Di@l Video Segmentation (the problem of decomposing video into its component units ) is one ...
IEEE Transactions on Pattern Analysis and Machine Intelligence, 1999
With complex multimedia data, we see the emergence of database systems in which thefundamental op... more With complex multimedia data, we see the emergence of database systems in which thefundamental operation is similarity assessment. Before database issues can be addressed, it isnecessary to give a definition of similarity as an operation.In this paper we develop a similarity measure, based on fuzzy logic, that exhibit severalfeatures that match experimental findings in humans. The model is dubbed Fuzzy
IEEE Transactions on Knowledge and Data Engineering, 2001
In this paper we discuss briefly some aspects of image semantics and the role that it plays for t... more In this paper we discuss briefly some aspects of image semantics and the role that it plays for the design of Image Databases. We argue that images don't have an intrinsic meaning, but that they are endowed with a meaning by placing them in the context of other images and by the user interaction. From this observation, we conclude that in an image database users should be allowed to manipulate not only the individual images, but also the relation between them. We present an interface model based on the manipulation of configurations of images. * Simone Santini and Ramesh Jain are with the Visual Computing Laboratory, University of California, San Diego, {ssantini,jain}@ece.ucsd.edu. Amarnath Gupta is with the San Diego Supercomputer Center, [email protected] • It is grounded in action. The database can estabilish the meaning of an image based on the actions of the user when the image is presented. In a database situation, the only action allowed to the user is asking a query. Therefore, the meaning of an image will be revealed to the database by interpreting the sequence of queries posed by the user 1 .
Until recently, the management of large image databases has relied exclusively on manually entere... more Until recently, the management of large image databases has relied exclusively on manually entered alphanumeric annotations. Systems are beginning to emerge in both the research and commercial sectors based on 'content-based' image retrieval, a technique which explicitly ...
Video is composed of audio-visual information. Pro-viding content based access to video data is e... more Video is composed of audio-visual information. Pro-viding content based access to video data is essential for the sucessful integration of video into computers. Organizing video for content based access requires the use of video metadata. This paper explores the nature video ...
Uploads
Papers by Ramesh Jain