Professional dance is characterized by high impulsiveness, elegance, and aesthetic beauty. In ord... more Professional dance is characterized by high impulsiveness, elegance, and aesthetic beauty. In order to reach the desired professionalism, it requires years of long and exhausting practice, good physical condition, musicality, but also, a good understanding of choreography. Capturing dance motions and transferring them to digital avatars is commonly used in the film and entertainment industries. However, so far, access to high-quality dance data is very limited, mainly due to the many practical difficulties in capturing the movements of dancers, making it prohibitive for large-scale data acquisition. In this paper, we present a model that enhances the professionalism of amateur dance movements, allowing movement quality to be improved in both spatial and temporal domains. Our model consists of a dance-to-music alignment stage responsible for learning the optimal temporal alignment path between dance and music, and a dance-enhancement stage that injects features of professionalism in ...
IEEE Transactions on Visualization and Computer Graphics
Colormapping is an effective and popular visualization technique for analyzing patterns in scalar... more Colormapping is an effective and popular visualization technique for analyzing patterns in scalar fields. Scientists usually adjust a default colormap to show hidden patterns by shifting the colors in a trial-and-error process. To improve efficiency, efforts have been made to automate the colormap adjustment process based on data properties (e.g., statistical data value or histogram distribution). However, as the data properties have no direct correlation to the spatial variations, previous methods may be insufficient to reveal the dynamic range of spatial variations hidden in the data. To address the above issues, we conduct a pilot analysis with domain experts and summarize three requirements for the colormap adjustment process. Based on the requirements, we formulate colormap adjustment as an objective function, composed of a boundary term and a fidelity term, which is flexible enough to support interactive functionalities. We compare our approach with alternative methods under a quantitative measure and a qualitative user study (25 participants), based on a set of data with broad distribution diversity. We further evaluate our approach via three case studies with six domain experts. Our method is not necessarily more optimal than alternative methods of revealing patterns, but rather is an additional color adjustment option for exploring data with a dynamic range of spatial variations.
Journal of Computer-Aided Design & Computer Graphics, 2021
Traditional automatic color optimization methods face the challenge of expressing global features... more Traditional automatic color optimization methods face the challenge of expressing global features in dynamic data ranges. To solve this problem, an adaptive colormap optimization method based on inserting colors is proposed, which includes a process of estimating color inserting position and an inserting color optimization procedure. Firstly, color inserting positions are selected based on color discriminability and data histogram distribution. By keeping the color inserting positions, corresponding embedding colors are estimated through a novel energy optimization equation under the guidance of visual discriminability and the consistency to the original colormap. On the basis of the algorithm, an interactive visual data exploratory system is provided, which includes supporting global data perception and local ROI analysis. The effectiveness and applicability of the algorithm is evaluated via a user study and a case study, based on 6 colormaps with different color features and 8 datasets with different data distributions. The results demonstrate that proposed method can produce high quality visual data information compared with other algorithms, providing a condition for further data analysis.
Understanding semantic similarity among images is the core of a wide range of computer vision app... more Understanding semantic similarity among images is the core of a wide range of computer vision applications. An important step towards this goal is to collect and learn human perceptions. Interestingly, the semantic context of images is often ambiguous as images can be perceived with emphasis on different aspects, which may be contradictory to each other. In this paper, we present a method for learning the semantic similarity among images, inferring their latent aspects and embedding them into multi-spaces corresponding to their semantic aspects. We consider the multi-embedding problem as an optimization function that evaluates the embedded distances with respect to the qualitative clustering queries. The key idea of our approach is to collect and embed qualitative measures that share the same aspects in bundles. To ensure similarity aspect sharing among multiple measures, image classification queries are presented to, and solved by users. The collected image clusters are then conver...
Vast amounts of data are produced with the development of smart cities and urban computing techno... more Vast amounts of data are produced with the development of smart cities and urban computing technologies. The data is often captured from multiple sensors, with heterogeneous structures and highly decentralized connections. Integrated data representation and smart computational models are required for more complex tasks in urban computing. We dwell deeply on two fundamental questions-can we provide an integrated data representation for the whole cyber-physical-social system? And, can we provide an integrated framework to choose the appropriate data for understanding a specific urban event? A holography data representation and the quasi-holography computational model have been proposed to address these problems. We describe case studies using the quasi-holography computational model, and discuss further problems to solve regarding our model.
Haze is a hazardous atmospheric phenomenon that threatens human health and leads to severe econom... more Haze is a hazardous atmospheric phenomenon that threatens human health and leads to severe economic problems. A number of weather factors are relevant to the emergence and evolvement of haze. In this paper we present a visual analytics system for haze study, including its evolution and correlations to a number of weather factors. Specifically, we introduce a haze event detection algorithm based on common haze identification rules in meteorology using the PM 2.5 concentration data. We develop a comparative visualization to consistently overview trends of scalar variables and wind directions, in which wind patterns are extracted via clustering streamlines at user-given sampling time. To study the correlation between wind and PM 2.5 , we decompose time steps into time intervals according to the temporal similarity of streamlines. Additionally, we develop a 1D function dissimilarity measurement to study the temporal correlation between PM 2.5 concentrations and relevant weather factors, such as wind strength, relatively humidity and planetary boundary layer. Furthermore, we employ particle advection using pathline computation within the wind field to locate the origins and destinations of particles seeded in user-interested areas. We applied our system to study of a number of hazes occurring in January 2013 in Beijing, (China). Interpretations and evaluations from domain experts demonstrate the effectiveness of our system in facilitating haze study.
Understanding semantic similarity among images is the core of a wide range of computer graphics a... more Understanding semantic similarity among images is the core of a wide range of computer graphics and computer vision applications. However, the visual context of images is often ambiguous as images can be perceived with emphasis on different attributes. In this paper, we present a method for learning the semantic visual similarity among images, inferring their latent attributes and embedding them into multi-spaces corresponding to each latent attribute. We consider the multi-embedding problem as an optimization function that evaluates the embedded distances with respect to qualitative crowdsourced clusterings. The key idea of our approach is to collect and embed qualitative pairwise tuples that share the same attributes in clusters. To ensure similarity attribute sharing among multiple measures, image classification clusters are presented to, and solved by users. The collected image clusters are then converted into groups of tuples, which are fed into our group optimization algorithm that jointly infers the attribute similarity and multi-attribute embedding. Our multi-attribute embedding allows retrieving similar objects in different attribute spaces. Experimental results show that our approach outperforms state-of-the-art multi-embedding approaches on various datasets, and demonstrate the usage of the multi-attribute embedding in image retrieval application.
Figure 1: Given a segmented image as input we produce a stereo pair (or a motion parallax animati... more Figure 1: Given a segmented image as input we produce a stereo pair (or a motion parallax animation) by hallucinating plausible 3D geometry for the scene. First, segments are depth-sorted using simple depth and occlusion cues. Next, the geometry and texture of each object is completed using symmetry and convexity priors. Finally, we infer a depth placement for each object. We urge the reader to view the companion video before reading the paper, and to examine our anaglyphs in full size using red/cyan glasses. All of the results in this paper are available in the supplementary material.
Bas-relief is an art form part way between sculpture and drawing. In this paper, we present an al... more Bas-relief is an art form part way between sculpture and drawing. In this paper, we present an algorithm for generating a bas-relief from a single image, inspired by the process that artists use to create reliefs. We do not aim to recover exact depth values for objects in the image, which is a tricky computer vision problem, requiring assumptions that are rarely satisfied. Instead, we determine layers based on relative depth ordering of objects (and their parts) in the image, and use this information to construct surfaces in the 3D relief model. Feature lines are extracted and used to build a new region-based representation of the input image. During surface construction, a base surface is first generated; it is then augmented using both intensity and gradient information from the original image. To prevent depth errors arising due to augmentation, a feedback process is used to refine the output. Our experimental results show the generated bas-reliefs have smooth boundaries with appropriate height relationships, a key property of bas-reliefs created by artists. We demonstrate that our algorithm works well for a range of input images, including human faces, flowers and animals.
Proceedings of the ACM SIGGRAPH / Eurographics Symposium on Computer Animation, 2017
Motion capture technology has enabled the acquisition of high quality human motions for animating... more Motion capture technology has enabled the acquisition of high quality human motions for animating digital characters with extremely high delity. However, despite all the advances in motion editing and synthesis, it remains an open problem to modify pre-captured motions that are highly expressive, such as contemporary dances, for stylization and emotionalization. In this work, we present a novel approach for stylizing such motions by using emotion coordinates de ned by the Russell's Circumplex Model (RCM). We extract and analyze a large set of body and motion features, based on the Laban Movement Analysis (LMA), and choose the e ective and consistent features for characterizing emotions of motions. These features provide a mechanism not only for deriving the emotion coordinates of a newly input motion, but also for stylizing the motion to express a di erent emotion without having to reference the training data. Such decoupling of the training data and new input motions eliminates the necessity of manual processing and motion registration. We implement the two-way mapping between the motion features and emotion coordinates through Radial Basis Function (RBF) regression and interpolation, which can stylize freestyle highly dynamic dance movements at interactive rates. Our results and user studies demonstrate the e ectiveness of the stylization framework with a variety of dance movements exhibiting a diverse set of emotions.
We introduce a novel method for enabling stereoscopic viewing of a scene from a single pre-segmen... more We introduce a novel method for enabling stereoscopic viewing of a scene from a single pre-segmented image. Rather than attempting full 3D reconstruction or accurate depth map recovery, we hallucinate a rough approximation of the scene’s 3D model using a number of simple depth and occlusion cues and shape priors. We begin by depth-sorting the segments, each of which is assumed to represent a separate object in the scene, resulting in a collection of depth layers. The shapes and textures of the partially occluded segments are then completed using symmetry and convexity priors. Next, each completed segment is converted to a union of generalized cylinders
yielding a rough 3D model for each object. Finally, the object depths are refined using an iterative ground fitting
process. The hallucinated 3D model of the scene may then be used to generate a stereoscopic image pair, or to produce images from novel viewpoints within a small neighborhood of the original view. Despite the simplicity of our approach, we show that it compares favorably with state-of-the-art depth ordering methods. A user study was conducted showing that our method produces more convincing stereoscopic images than existing semi-interactive and automatic single image depth recovery methods.
Bas-relief is an art form part way between sculpture and drawing. In this paper, we present an al... more Bas-relief is an art form part way between sculpture and drawing. In this paper, we present an algorithm for generating a bas-relief from a single image, inspired by the process that
artists use to create reliefs. We do not aim to recover exact depth values for objects in the image, which is a tricky computer vision problem, requiring assumptions that are
rarely satisfied. Instead, we determine layers based on relative depth ordering of objects (and their parts) in the image, and use this information to construct surfaces in the 3D relief model. Feature lines are extracted and used to build a new region-based representation of the input image. During surface construction, a base surface is first generated; it is then augmented using both intensity and gradient information from the original image. To prevent depth errors arising due to augmentation, a feedback process is used to refine the output. Our experimental results show the generated bas-reliefs have smooth boundaries with appropriate height relationships, a key property of bas-reliefs created by artists. We demonstrate that our algorithm works well for a range of input images, including human faces, flowers and animals.
Professional dance is characterized by high impulsiveness, elegance, and aesthetic beauty. In ord... more Professional dance is characterized by high impulsiveness, elegance, and aesthetic beauty. In order to reach the desired professionalism, it requires years of long and exhausting practice, good physical condition, musicality, but also, a good understanding of choreography. Capturing dance motions and transferring them to digital avatars is commonly used in the film and entertainment industries. However, so far, access to high-quality dance data is very limited, mainly due to the many practical difficulties in capturing the movements of dancers, making it prohibitive for large-scale data acquisition. In this paper, we present a model that enhances the professionalism of amateur dance movements, allowing movement quality to be improved in both spatial and temporal domains. Our model consists of a dance-to-music alignment stage responsible for learning the optimal temporal alignment path between dance and music, and a dance-enhancement stage that injects features of professionalism in ...
IEEE Transactions on Visualization and Computer Graphics
Colormapping is an effective and popular visualization technique for analyzing patterns in scalar... more Colormapping is an effective and popular visualization technique for analyzing patterns in scalar fields. Scientists usually adjust a default colormap to show hidden patterns by shifting the colors in a trial-and-error process. To improve efficiency, efforts have been made to automate the colormap adjustment process based on data properties (e.g., statistical data value or histogram distribution). However, as the data properties have no direct correlation to the spatial variations, previous methods may be insufficient to reveal the dynamic range of spatial variations hidden in the data. To address the above issues, we conduct a pilot analysis with domain experts and summarize three requirements for the colormap adjustment process. Based on the requirements, we formulate colormap adjustment as an objective function, composed of a boundary term and a fidelity term, which is flexible enough to support interactive functionalities. We compare our approach with alternative methods under a quantitative measure and a qualitative user study (25 participants), based on a set of data with broad distribution diversity. We further evaluate our approach via three case studies with six domain experts. Our method is not necessarily more optimal than alternative methods of revealing patterns, but rather is an additional color adjustment option for exploring data with a dynamic range of spatial variations.
Journal of Computer-Aided Design & Computer Graphics, 2021
Traditional automatic color optimization methods face the challenge of expressing global features... more Traditional automatic color optimization methods face the challenge of expressing global features in dynamic data ranges. To solve this problem, an adaptive colormap optimization method based on inserting colors is proposed, which includes a process of estimating color inserting position and an inserting color optimization procedure. Firstly, color inserting positions are selected based on color discriminability and data histogram distribution. By keeping the color inserting positions, corresponding embedding colors are estimated through a novel energy optimization equation under the guidance of visual discriminability and the consistency to the original colormap. On the basis of the algorithm, an interactive visual data exploratory system is provided, which includes supporting global data perception and local ROI analysis. The effectiveness and applicability of the algorithm is evaluated via a user study and a case study, based on 6 colormaps with different color features and 8 datasets with different data distributions. The results demonstrate that proposed method can produce high quality visual data information compared with other algorithms, providing a condition for further data analysis.
Understanding semantic similarity among images is the core of a wide range of computer vision app... more Understanding semantic similarity among images is the core of a wide range of computer vision applications. An important step towards this goal is to collect and learn human perceptions. Interestingly, the semantic context of images is often ambiguous as images can be perceived with emphasis on different aspects, which may be contradictory to each other. In this paper, we present a method for learning the semantic similarity among images, inferring their latent aspects and embedding them into multi-spaces corresponding to their semantic aspects. We consider the multi-embedding problem as an optimization function that evaluates the embedded distances with respect to the qualitative clustering queries. The key idea of our approach is to collect and embed qualitative measures that share the same aspects in bundles. To ensure similarity aspect sharing among multiple measures, image classification queries are presented to, and solved by users. The collected image clusters are then conver...
Vast amounts of data are produced with the development of smart cities and urban computing techno... more Vast amounts of data are produced with the development of smart cities and urban computing technologies. The data is often captured from multiple sensors, with heterogeneous structures and highly decentralized connections. Integrated data representation and smart computational models are required for more complex tasks in urban computing. We dwell deeply on two fundamental questions-can we provide an integrated data representation for the whole cyber-physical-social system? And, can we provide an integrated framework to choose the appropriate data for understanding a specific urban event? A holography data representation and the quasi-holography computational model have been proposed to address these problems. We describe case studies using the quasi-holography computational model, and discuss further problems to solve regarding our model.
Haze is a hazardous atmospheric phenomenon that threatens human health and leads to severe econom... more Haze is a hazardous atmospheric phenomenon that threatens human health and leads to severe economic problems. A number of weather factors are relevant to the emergence and evolvement of haze. In this paper we present a visual analytics system for haze study, including its evolution and correlations to a number of weather factors. Specifically, we introduce a haze event detection algorithm based on common haze identification rules in meteorology using the PM 2.5 concentration data. We develop a comparative visualization to consistently overview trends of scalar variables and wind directions, in which wind patterns are extracted via clustering streamlines at user-given sampling time. To study the correlation between wind and PM 2.5 , we decompose time steps into time intervals according to the temporal similarity of streamlines. Additionally, we develop a 1D function dissimilarity measurement to study the temporal correlation between PM 2.5 concentrations and relevant weather factors, such as wind strength, relatively humidity and planetary boundary layer. Furthermore, we employ particle advection using pathline computation within the wind field to locate the origins and destinations of particles seeded in user-interested areas. We applied our system to study of a number of hazes occurring in January 2013 in Beijing, (China). Interpretations and evaluations from domain experts demonstrate the effectiveness of our system in facilitating haze study.
Understanding semantic similarity among images is the core of a wide range of computer graphics a... more Understanding semantic similarity among images is the core of a wide range of computer graphics and computer vision applications. However, the visual context of images is often ambiguous as images can be perceived with emphasis on different attributes. In this paper, we present a method for learning the semantic visual similarity among images, inferring their latent attributes and embedding them into multi-spaces corresponding to each latent attribute. We consider the multi-embedding problem as an optimization function that evaluates the embedded distances with respect to qualitative crowdsourced clusterings. The key idea of our approach is to collect and embed qualitative pairwise tuples that share the same attributes in clusters. To ensure similarity attribute sharing among multiple measures, image classification clusters are presented to, and solved by users. The collected image clusters are then converted into groups of tuples, which are fed into our group optimization algorithm that jointly infers the attribute similarity and multi-attribute embedding. Our multi-attribute embedding allows retrieving similar objects in different attribute spaces. Experimental results show that our approach outperforms state-of-the-art multi-embedding approaches on various datasets, and demonstrate the usage of the multi-attribute embedding in image retrieval application.
Figure 1: Given a segmented image as input we produce a stereo pair (or a motion parallax animati... more Figure 1: Given a segmented image as input we produce a stereo pair (or a motion parallax animation) by hallucinating plausible 3D geometry for the scene. First, segments are depth-sorted using simple depth and occlusion cues. Next, the geometry and texture of each object is completed using symmetry and convexity priors. Finally, we infer a depth placement for each object. We urge the reader to view the companion video before reading the paper, and to examine our anaglyphs in full size using red/cyan glasses. All of the results in this paper are available in the supplementary material.
Bas-relief is an art form part way between sculpture and drawing. In this paper, we present an al... more Bas-relief is an art form part way between sculpture and drawing. In this paper, we present an algorithm for generating a bas-relief from a single image, inspired by the process that artists use to create reliefs. We do not aim to recover exact depth values for objects in the image, which is a tricky computer vision problem, requiring assumptions that are rarely satisfied. Instead, we determine layers based on relative depth ordering of objects (and their parts) in the image, and use this information to construct surfaces in the 3D relief model. Feature lines are extracted and used to build a new region-based representation of the input image. During surface construction, a base surface is first generated; it is then augmented using both intensity and gradient information from the original image. To prevent depth errors arising due to augmentation, a feedback process is used to refine the output. Our experimental results show the generated bas-reliefs have smooth boundaries with appropriate height relationships, a key property of bas-reliefs created by artists. We demonstrate that our algorithm works well for a range of input images, including human faces, flowers and animals.
Proceedings of the ACM SIGGRAPH / Eurographics Symposium on Computer Animation, 2017
Motion capture technology has enabled the acquisition of high quality human motions for animating... more Motion capture technology has enabled the acquisition of high quality human motions for animating digital characters with extremely high delity. However, despite all the advances in motion editing and synthesis, it remains an open problem to modify pre-captured motions that are highly expressive, such as contemporary dances, for stylization and emotionalization. In this work, we present a novel approach for stylizing such motions by using emotion coordinates de ned by the Russell's Circumplex Model (RCM). We extract and analyze a large set of body and motion features, based on the Laban Movement Analysis (LMA), and choose the e ective and consistent features for characterizing emotions of motions. These features provide a mechanism not only for deriving the emotion coordinates of a newly input motion, but also for stylizing the motion to express a di erent emotion without having to reference the training data. Such decoupling of the training data and new input motions eliminates the necessity of manual processing and motion registration. We implement the two-way mapping between the motion features and emotion coordinates through Radial Basis Function (RBF) regression and interpolation, which can stylize freestyle highly dynamic dance movements at interactive rates. Our results and user studies demonstrate the e ectiveness of the stylization framework with a variety of dance movements exhibiting a diverse set of emotions.
We introduce a novel method for enabling stereoscopic viewing of a scene from a single pre-segmen... more We introduce a novel method for enabling stereoscopic viewing of a scene from a single pre-segmented image. Rather than attempting full 3D reconstruction or accurate depth map recovery, we hallucinate a rough approximation of the scene’s 3D model using a number of simple depth and occlusion cues and shape priors. We begin by depth-sorting the segments, each of which is assumed to represent a separate object in the scene, resulting in a collection of depth layers. The shapes and textures of the partially occluded segments are then completed using symmetry and convexity priors. Next, each completed segment is converted to a union of generalized cylinders
yielding a rough 3D model for each object. Finally, the object depths are refined using an iterative ground fitting
process. The hallucinated 3D model of the scene may then be used to generate a stereoscopic image pair, or to produce images from novel viewpoints within a small neighborhood of the original view. Despite the simplicity of our approach, we show that it compares favorably with state-of-the-art depth ordering methods. A user study was conducted showing that our method produces more convincing stereoscopic images than existing semi-interactive and automatic single image depth recovery methods.
Bas-relief is an art form part way between sculpture and drawing. In this paper, we present an al... more Bas-relief is an art form part way between sculpture and drawing. In this paper, we present an algorithm for generating a bas-relief from a single image, inspired by the process that
artists use to create reliefs. We do not aim to recover exact depth values for objects in the image, which is a tricky computer vision problem, requiring assumptions that are
rarely satisfied. Instead, we determine layers based on relative depth ordering of objects (and their parts) in the image, and use this information to construct surfaces in the 3D relief model. Feature lines are extracted and used to build a new region-based representation of the input image. During surface construction, a base surface is first generated; it is then augmented using both intensity and gradient information from the original image. To prevent depth errors arising due to augmentation, a feedback process is used to refine the output. Our experimental results show the generated bas-reliefs have smooth boundaries with appropriate height relationships, a key property of bas-reliefs created by artists. We demonstrate that our algorithm works well for a range of input images, including human faces, flowers and animals.
Uploads
Papers by Qiong Zeng
yielding a rough 3D model for each object. Finally, the object depths are refined using an iterative ground fitting
process. The hallucinated 3D model of the scene may then be used to generate a stereoscopic image pair, or to produce images from novel viewpoints within a small neighborhood of the original view. Despite the simplicity of our approach, we show that it compares favorably with state-of-the-art depth ordering methods. A user study was conducted showing that our method produces more convincing stereoscopic images than existing semi-interactive and automatic single image depth recovery methods.
artists use to create reliefs. We do not aim to recover exact depth values for objects in the image, which is a tricky computer vision problem, requiring assumptions that are
rarely satisfied. Instead, we determine layers based on relative depth ordering of objects (and their parts) in the image, and use this information to construct surfaces in the 3D relief model. Feature lines are extracted and used to build a new region-based representation of the input image. During surface construction, a base surface is first generated; it is then augmented using both intensity and gradient information from the original image. To prevent depth errors arising due to augmentation, a feedback process is used to refine the output. Our experimental results show the generated bas-reliefs have smooth boundaries with appropriate height relationships, a key property of bas-reliefs created by artists. We demonstrate that our algorithm works well for a range of input images, including human faces, flowers and animals.
yielding a rough 3D model for each object. Finally, the object depths are refined using an iterative ground fitting
process. The hallucinated 3D model of the scene may then be used to generate a stereoscopic image pair, or to produce images from novel viewpoints within a small neighborhood of the original view. Despite the simplicity of our approach, we show that it compares favorably with state-of-the-art depth ordering methods. A user study was conducted showing that our method produces more convincing stereoscopic images than existing semi-interactive and automatic single image depth recovery methods.
artists use to create reliefs. We do not aim to recover exact depth values for objects in the image, which is a tricky computer vision problem, requiring assumptions that are
rarely satisfied. Instead, we determine layers based on relative depth ordering of objects (and their parts) in the image, and use this information to construct surfaces in the 3D relief model. Feature lines are extracted and used to build a new region-based representation of the input image. During surface construction, a base surface is first generated; it is then augmented using both intensity and gradient information from the original image. To prevent depth errors arising due to augmentation, a feedback process is used to refine the output. Our experimental results show the generated bas-reliefs have smooth boundaries with appropriate height relationships, a key property of bas-reliefs created by artists. We demonstrate that our algorithm works well for a range of input images, including human faces, flowers and animals.