Traditional approaches to data visualization have often focused on comparing different subsets of... more Traditional approaches to data visualization have often focused on comparing different subsets of data, and this is reflected in the many techniques developed and evaluated over the years for visual comparison. Similarly, common workflows for exploratory visualization are built upon the idea of users interactively applying various filter and grouping mechanisms in search of new insights. This paradigm has proven effective at helping users identify correlations between variables that can inform thinking and decision-making. However, recent studies show that consumers of visualizations often draw causal conclusions even when not supported by the data. Motivated by these observations, this article highlights recent advances from a growing community of researchers exploring methods that aim to directly support visual causal inference. However, many of these approaches have their own limitations which limit their use in many real-world scenarios. This article therefore also outlines a set of key open challenges and corresponding priorities for new research to advance the state of the art in visual causal inference.
Arthroscopy, Sports Medicine, and Rehabilitation, 2021
The purpose of this study was to determine the inter-rater reliability of arthroscopic video qual... more The purpose of this study was to determine the inter-rater reliability of arthroscopic video quality, determine correlation between surgeon rating and computational image metrics, and facilitate a quantitative methodology for assessing video quality. Methods: Five orthopaedic surgeons reviewed 60 clips from deidentified arthroscopic shoulder videos and rated each on a four-point Likert scale from poor to excellent view. The videos were randomized, and the process was completed a total of three times. Each user rating was averaged to provide a user rating per clip. Each video frame was processed to calculate brightness, local contrast, redness (used to represent bleeding), and image entropy. Each metric was then averaged over each frame per video clip, providing four image quality metrics per clip. Results: Interrater reliability for grading video quality had an intraclass correlation of .974. Improved image quality rating was positively correlated with increased entropy (.8142; P < .001), contrast (.8013; P < .001), and brightness (.6120; P < .001), and negatively correlated with redness (À.8626; P < .001). A multiple linear regression model was calculated with the image metrics used as predictors for the image quality ranking, with an R-squared value of .775 and root mean square error of .42. Conclusions: Our study demonstrates strong inter-rater reliability between surgeons when describing image quality and strong correlations between image quality and the computed image metrics. A model based on these metrics enables automatic quantification of image quality. Clinical Relevance: Video quality during arthroscopic cases can impact the ease and duration of the case which could contribute to swelling and complication risk. This pilot study provides a quantitative method to assess video quality. Future works can objectively determine factors that affect visualization during arthroscopy and identify options for improvement.
Arthroscopy, Sports Medicine, and Rehabilitation, 2021
The purpose of this study was to determine the inter-rater reliability of arthroscopic video qual... more The purpose of this study was to determine the inter-rater reliability of arthroscopic video quality, determine correlation between surgeon rating and computational image metrics, and facilitate a quantitative methodology for assessing video quality. Methods: Five orthopaedic surgeons reviewed 60 clips from deidentified arthroscopic shoulder videos and rated each on a four-point Likert scale from poor to excellent view. The videos were randomized, and the process was completed a total of three times. Each user rating was averaged to provide a user rating per clip. Each video frame was processed to calculate brightness, local contrast, redness (used to represent bleeding), and image entropy. Each metric was then averaged over each frame per video clip, providing four image quality metrics per clip. Results: Interrater reliability for grading video quality had an intraclass correlation of .974. Improved image quality rating was positively correlated with increased entropy (.8142; P < .001), contrast (.8013; P < .001), and brightness (.6120; P < .001), and negatively correlated with redness (À.8626; P < .001). A multiple linear regression model was calculated with the image metrics used as predictors for the image quality ranking, with an R-squared value of .775 and root mean square error of .42. Conclusions: Our study demonstrates strong inter-rater reliability between surgeons when describing image quality and strong correlations between image quality and the computed image metrics. A model based on these metrics enables automatic quantification of image quality. Clinical Relevance: Video quality during arthroscopic cases can impact the ease and duration of the case which could contribute to swelling and complication risk. This pilot study provides a quantitative method to assess video quality. Future works can objectively determine factors that affect visualization during arthroscopy and identify options for improvement.
SummaryImmCellFie is a user-friendly, web-based platform for comprehensive analysis of metabolic ... more SummaryImmCellFie is a user-friendly, web-based platform for comprehensive analysis of metabolic functions inferred from transcriptomic or proteomic data. It enables researchers to leverage the powerful mechanistic insight provided by complex genome-scale metabolic models with little to no bioinformatics training required. The platform has been integrated with a series of useful tools and richly annotated scientific visualizations for interactive exploration by the user. ImmCellFie pushes beyond simple statistical enrichment and incorporates complex biological mechanisms to quantify cell activity.Graphical abstract
The role of empathy and perspective-taking in preventing aggressive behaviors has been highlighte... more The role of empathy and perspective-taking in preventing aggressive behaviors has been highlighted in several theoretical models. In this study, we used immersive virtual reality to induce a full body ownership illusion that allows offenders to be in the body of a victim of domestic abuse. A group of male domestic violence offenders and a control group without a history of violence experienced a virtual scene of abuse in first-person perspective. During the virtual encounter, the participants’ real bodies were replaced with a life-sized virtual female body that moved synchronously with their own real movements. Participants' emotion recognition skills were assessed before and after the virtual experience. Our results revealed that offenders have a significantly lower ability to recognize fear in female faces compared to controls, with a bias towards classifying fearful faces as happy. After being embodied in a female victim, offenders improved their ability to recognize fearful ...
We present MIBFV, a method to produce real-time, multiscale animations of flow datasets. MIBFV ex... more We present MIBFV, a method to produce real-time, multiscale animations of flow datasets. MIBFV extends the attractive features of the Image-Based Flow Visualization (IBFV) method, ie dense flow domain coverage with flow-aligned noise, real-time animation, implementation simplicity, and few (or no) user input requirements, to a multiscale dimension. We generate a multiscale of flow-aligned patterns using an algebraic multigrid method and use them to synthesize the noise textures required by IBFV. We demonstrate our approach with ...
2019 IEEE Workshop on Visual Analytics in Healthcare (VAHC), 2019
With the increase in collection of temporal event data, especially electronic health record (EHR)... more With the increase in collection of temporal event data, especially electronic health record (EHR) data, numerous different visualization and analysis techniques have been developed to assist with the interpretation of such data. As datasets grow increasingly large in both number of event sequences and number of event types, two problems arise: how to group event types, and how to describe selection bias that can occur when selecting cohorts. This poster summarizes two papers, conditionally accepted to VAST, that introduce a dynamic and interactive algorithm for hierarchical event grouping, a scented scatter-plus-focus visualization that supports hierarchical exploration, a tree-based cohort provenance visualization, and a set of visualizations that provide per-dimension selection bias information for pairs of cohorts [2, 4]. These methods are integrated into the web-based interactive medical analysis tool Cadence
Journal of the American Medical Informatics Association, 2014
Objective This study investigates the use of visualization techniques reported between 1996 and 2... more Objective This study investigates the use of visualization techniques reported between 1996 and 2013 and evaluates innovative approaches to information visualization of electronic health record (EHR) data for knowledge discovery. Methods An electronic literature search was conducted May–July 2013 using MEDLINE and Web of Knowledge, supplemented by citation searching, gray literature searching, and reference list reviews. General search terms were used to assure a comprehensive document search. Results Beginning with 891 articles, the number of articles was reduced by eliminating 191 duplicates. A matrix was developed for categorizing all abstracts and to assist with determining those to be excluded for review. Eighteen articles were included in the final analysis. Discussion Several visualization techniques have been extensively researched. The most mature system is LifeLines and its applications as LifeLines2, EventFlow, and LifeFlow. Initially, research focused on records from a s...
2021 IEEE International Conference on Big Data (Big Data)
Roadway safety, especially in rural areas, is one of the most critical components in transportati... more Roadway safety, especially in rural areas, is one of the most critical components in transportation planning. In collaboration with North Carolina Department of Transportation (NCDOT), UNC Highway Safety Research Center (HSRC), and DOT Volpe National Transportation Systems Center, UNC Renaissance Computing Institute (RENCI) developed a roadside feature detection solution leveraging multiple convolutional neural networks. The solution used an iterative active learning (AL) computer vision model training pipeline integrated into an AI tool to detect safety features such as guardrails and utility poles in geographically distributed NC rural roads. We utilized transfer learning by adopting the Xception neural network architecture [1] as the feature extraction backbone which was then used in an iterative AL process supported by a web-based annotation tool. The annotation tool not only allowed for the collection of annotations through an iterative AL process for multiple safety features, it also enabled visual analysis and assessment of model prediction performance in the geospatial context. AL techniques were used to direct human annotators to label images that would most effectively improve the model aimed at minimizing the number of required training labels while maximizing the model’s performance. The iterative AL process combined with a common feature extraction backbone allowed fast model inference on millions of images in the AL sampling space. This enabled a rapid transition between AL rounds while also reducing the computing requirements for each round. Model feature extraction weights were then fine-tuned in the last round of AL to obtain the best accuracy. Since only about 2.7% of 2.6 million unlabeled images in the AL sampling space contain guardrails, there is a significant class imbalance problem that must be addressed in our AL sampling strategies for the guardrail classification model. In this paper, we present our AI tool processing pipeline and methodology and discuss our AL results and future work. Our AI tool can be used to detect roadside safety features and be extended to also locate them for assessing roadside hazards.
2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), 2020
Haptic feedback, an important aspect of learning in virtual reality, has been demonstrated in con... more Haptic feedback, an important aspect of learning in virtual reality, has been demonstrated in contexts such as surgical training. However, deploying haptic feedback in other educational practices remains understudied. Haptically-enabled science simulations enable students to experience abstract scientific concepts through concrete and observable lessons in which students can physically experience the concepts being taught through haptic feedback. The present study aims to investigate the effect of an educational simulation on the understanding of basic physics concepts related to buoyancy. Specifically, we hypothesize that a simulation with visual and haptic feedback will improve participant learning transfer.
2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), 2020
For educational purposes, virtual reality (VR) is often used to produce life-like experiences. Ho... more For educational purposes, virtual reality (VR) is often used to produce life-like experiences. However, the use of haptic feedback in educational practices for science and mathematics remained under-studied. Haptically-enabled science simulations (HESSs) enable students to physically experience the concepts being taught via haptic feedback. We present a study on the effect of a buoyancy HESS designed to aid in the understanding of basic physics concepts. We hypothesize that introducing both visual and haptic feedback of the underlying simulated forces will improve participants understanding. We investigate this hypothesis with a 2 (haptics: yes, no) × 2 (visuals: yes, no) between subjects design user study, where all participants were randomly assigned to one of the four conditions. Participants were given a pretest of buoyancy knowledge, then instructed to interact with the buoyancy simulation, then given a post-test of buoyancy knowledge. The present study is still in the process of data collection, with 40 out of 60 participants. Preliminary results highlight a significant improvement in performance of participants in the haptic-and-visual condition, while no significant differences were observed in other conditions.
Proceedings of the 2021 International Conference on Multimodal Interaction, 2021
We developed a haptically-enhanced physics simulation to investigate the effects of haptics on th... more We developed a haptically-enhanced physics simulation to investigate the effects of haptics on the understanding of conceptual concepts related to forces—specifically those related to buoyancy. We evaluated the effects of haptic force feedback, as well as traditional visual representations of forces, on learning via a between-participant user study. Participants completed a buoyancy assessment before and after interacting with the simulation. Haptics enhanced performance regardless of prior knowledge. However, the combined effect of haptics with visual cues differed based on participant prior knowledge. Participants with high prior knowledge significantly improved performance when given both abstract visual cues and haptic feedback combined. Participants with low prior knowledge significantly improved when given haptic feedback alone, and the combination of haptics with visual cues did not improve performance. Our results suggest that the prior knowledge of users and the visual cues used impact the effectiveness of haptically-enhanced simulations with respect to learning outcomes.
Traditional approaches to data visualization have often focused on comparing different subsets of... more Traditional approaches to data visualization have often focused on comparing different subsets of data, and this is reflected in the many techniques developed and evaluated over the years for visual comparison. Similarly, common workflows for exploratory visualization are built upon the idea of users interactively applying various filter and grouping mechanisms in search of new insights. This paradigm has proven effective at helping users identify correlations between variables that can inform thinking and decision-making. However, recent studies show that consumers of visualizations often draw causal conclusions even when not supported by the data. Motivated by these observations, this article highlights recent advances from a growing community of researchers exploring methods that aim to directly support visual causal inference. However, many of these approaches have their own limitations which limit their use in many real-world scenarios. This article therefore also outlines a set of key open challenges and corresponding priorities for new research to advance the state of the art in visual causal inference.
Arthroscopy, Sports Medicine, and Rehabilitation, 2021
The purpose of this study was to determine the inter-rater reliability of arthroscopic video qual... more The purpose of this study was to determine the inter-rater reliability of arthroscopic video quality, determine correlation between surgeon rating and computational image metrics, and facilitate a quantitative methodology for assessing video quality. Methods: Five orthopaedic surgeons reviewed 60 clips from deidentified arthroscopic shoulder videos and rated each on a four-point Likert scale from poor to excellent view. The videos were randomized, and the process was completed a total of three times. Each user rating was averaged to provide a user rating per clip. Each video frame was processed to calculate brightness, local contrast, redness (used to represent bleeding), and image entropy. Each metric was then averaged over each frame per video clip, providing four image quality metrics per clip. Results: Interrater reliability for grading video quality had an intraclass correlation of .974. Improved image quality rating was positively correlated with increased entropy (.8142; P < .001), contrast (.8013; P < .001), and brightness (.6120; P < .001), and negatively correlated with redness (À.8626; P < .001). A multiple linear regression model was calculated with the image metrics used as predictors for the image quality ranking, with an R-squared value of .775 and root mean square error of .42. Conclusions: Our study demonstrates strong inter-rater reliability between surgeons when describing image quality and strong correlations between image quality and the computed image metrics. A model based on these metrics enables automatic quantification of image quality. Clinical Relevance: Video quality during arthroscopic cases can impact the ease and duration of the case which could contribute to swelling and complication risk. This pilot study provides a quantitative method to assess video quality. Future works can objectively determine factors that affect visualization during arthroscopy and identify options for improvement.
Arthroscopy, Sports Medicine, and Rehabilitation, 2021
The purpose of this study was to determine the inter-rater reliability of arthroscopic video qual... more The purpose of this study was to determine the inter-rater reliability of arthroscopic video quality, determine correlation between surgeon rating and computational image metrics, and facilitate a quantitative methodology for assessing video quality. Methods: Five orthopaedic surgeons reviewed 60 clips from deidentified arthroscopic shoulder videos and rated each on a four-point Likert scale from poor to excellent view. The videos were randomized, and the process was completed a total of three times. Each user rating was averaged to provide a user rating per clip. Each video frame was processed to calculate brightness, local contrast, redness (used to represent bleeding), and image entropy. Each metric was then averaged over each frame per video clip, providing four image quality metrics per clip. Results: Interrater reliability for grading video quality had an intraclass correlation of .974. Improved image quality rating was positively correlated with increased entropy (.8142; P < .001), contrast (.8013; P < .001), and brightness (.6120; P < .001), and negatively correlated with redness (À.8626; P < .001). A multiple linear regression model was calculated with the image metrics used as predictors for the image quality ranking, with an R-squared value of .775 and root mean square error of .42. Conclusions: Our study demonstrates strong inter-rater reliability between surgeons when describing image quality and strong correlations between image quality and the computed image metrics. A model based on these metrics enables automatic quantification of image quality. Clinical Relevance: Video quality during arthroscopic cases can impact the ease and duration of the case which could contribute to swelling and complication risk. This pilot study provides a quantitative method to assess video quality. Future works can objectively determine factors that affect visualization during arthroscopy and identify options for improvement.
SummaryImmCellFie is a user-friendly, web-based platform for comprehensive analysis of metabolic ... more SummaryImmCellFie is a user-friendly, web-based platform for comprehensive analysis of metabolic functions inferred from transcriptomic or proteomic data. It enables researchers to leverage the powerful mechanistic insight provided by complex genome-scale metabolic models with little to no bioinformatics training required. The platform has been integrated with a series of useful tools and richly annotated scientific visualizations for interactive exploration by the user. ImmCellFie pushes beyond simple statistical enrichment and incorporates complex biological mechanisms to quantify cell activity.Graphical abstract
The role of empathy and perspective-taking in preventing aggressive behaviors has been highlighte... more The role of empathy and perspective-taking in preventing aggressive behaviors has been highlighted in several theoretical models. In this study, we used immersive virtual reality to induce a full body ownership illusion that allows offenders to be in the body of a victim of domestic abuse. A group of male domestic violence offenders and a control group without a history of violence experienced a virtual scene of abuse in first-person perspective. During the virtual encounter, the participants’ real bodies were replaced with a life-sized virtual female body that moved synchronously with their own real movements. Participants' emotion recognition skills were assessed before and after the virtual experience. Our results revealed that offenders have a significantly lower ability to recognize fear in female faces compared to controls, with a bias towards classifying fearful faces as happy. After being embodied in a female victim, offenders improved their ability to recognize fearful ...
We present MIBFV, a method to produce real-time, multiscale animations of flow datasets. MIBFV ex... more We present MIBFV, a method to produce real-time, multiscale animations of flow datasets. MIBFV extends the attractive features of the Image-Based Flow Visualization (IBFV) method, ie dense flow domain coverage with flow-aligned noise, real-time animation, implementation simplicity, and few (or no) user input requirements, to a multiscale dimension. We generate a multiscale of flow-aligned patterns using an algebraic multigrid method and use them to synthesize the noise textures required by IBFV. We demonstrate our approach with ...
2019 IEEE Workshop on Visual Analytics in Healthcare (VAHC), 2019
With the increase in collection of temporal event data, especially electronic health record (EHR)... more With the increase in collection of temporal event data, especially electronic health record (EHR) data, numerous different visualization and analysis techniques have been developed to assist with the interpretation of such data. As datasets grow increasingly large in both number of event sequences and number of event types, two problems arise: how to group event types, and how to describe selection bias that can occur when selecting cohorts. This poster summarizes two papers, conditionally accepted to VAST, that introduce a dynamic and interactive algorithm for hierarchical event grouping, a scented scatter-plus-focus visualization that supports hierarchical exploration, a tree-based cohort provenance visualization, and a set of visualizations that provide per-dimension selection bias information for pairs of cohorts [2, 4]. These methods are integrated into the web-based interactive medical analysis tool Cadence
Journal of the American Medical Informatics Association, 2014
Objective This study investigates the use of visualization techniques reported between 1996 and 2... more Objective This study investigates the use of visualization techniques reported between 1996 and 2013 and evaluates innovative approaches to information visualization of electronic health record (EHR) data for knowledge discovery. Methods An electronic literature search was conducted May–July 2013 using MEDLINE and Web of Knowledge, supplemented by citation searching, gray literature searching, and reference list reviews. General search terms were used to assure a comprehensive document search. Results Beginning with 891 articles, the number of articles was reduced by eliminating 191 duplicates. A matrix was developed for categorizing all abstracts and to assist with determining those to be excluded for review. Eighteen articles were included in the final analysis. Discussion Several visualization techniques have been extensively researched. The most mature system is LifeLines and its applications as LifeLines2, EventFlow, and LifeFlow. Initially, research focused on records from a s...
2021 IEEE International Conference on Big Data (Big Data)
Roadway safety, especially in rural areas, is one of the most critical components in transportati... more Roadway safety, especially in rural areas, is one of the most critical components in transportation planning. In collaboration with North Carolina Department of Transportation (NCDOT), UNC Highway Safety Research Center (HSRC), and DOT Volpe National Transportation Systems Center, UNC Renaissance Computing Institute (RENCI) developed a roadside feature detection solution leveraging multiple convolutional neural networks. The solution used an iterative active learning (AL) computer vision model training pipeline integrated into an AI tool to detect safety features such as guardrails and utility poles in geographically distributed NC rural roads. We utilized transfer learning by adopting the Xception neural network architecture [1] as the feature extraction backbone which was then used in an iterative AL process supported by a web-based annotation tool. The annotation tool not only allowed for the collection of annotations through an iterative AL process for multiple safety features, it also enabled visual analysis and assessment of model prediction performance in the geospatial context. AL techniques were used to direct human annotators to label images that would most effectively improve the model aimed at minimizing the number of required training labels while maximizing the model’s performance. The iterative AL process combined with a common feature extraction backbone allowed fast model inference on millions of images in the AL sampling space. This enabled a rapid transition between AL rounds while also reducing the computing requirements for each round. Model feature extraction weights were then fine-tuned in the last round of AL to obtain the best accuracy. Since only about 2.7% of 2.6 million unlabeled images in the AL sampling space contain guardrails, there is a significant class imbalance problem that must be addressed in our AL sampling strategies for the guardrail classification model. In this paper, we present our AI tool processing pipeline and methodology and discuss our AL results and future work. Our AI tool can be used to detect roadside safety features and be extended to also locate them for assessing roadside hazards.
2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), 2020
Haptic feedback, an important aspect of learning in virtual reality, has been demonstrated in con... more Haptic feedback, an important aspect of learning in virtual reality, has been demonstrated in contexts such as surgical training. However, deploying haptic feedback in other educational practices remains understudied. Haptically-enabled science simulations enable students to experience abstract scientific concepts through concrete and observable lessons in which students can physically experience the concepts being taught through haptic feedback. The present study aims to investigate the effect of an educational simulation on the understanding of basic physics concepts related to buoyancy. Specifically, we hypothesize that a simulation with visual and haptic feedback will improve participant learning transfer.
2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), 2020
For educational purposes, virtual reality (VR) is often used to produce life-like experiences. Ho... more For educational purposes, virtual reality (VR) is often used to produce life-like experiences. However, the use of haptic feedback in educational practices for science and mathematics remained under-studied. Haptically-enabled science simulations (HESSs) enable students to physically experience the concepts being taught via haptic feedback. We present a study on the effect of a buoyancy HESS designed to aid in the understanding of basic physics concepts. We hypothesize that introducing both visual and haptic feedback of the underlying simulated forces will improve participants understanding. We investigate this hypothesis with a 2 (haptics: yes, no) × 2 (visuals: yes, no) between subjects design user study, where all participants were randomly assigned to one of the four conditions. Participants were given a pretest of buoyancy knowledge, then instructed to interact with the buoyancy simulation, then given a post-test of buoyancy knowledge. The present study is still in the process of data collection, with 40 out of 60 participants. Preliminary results highlight a significant improvement in performance of participants in the haptic-and-visual condition, while no significant differences were observed in other conditions.
Proceedings of the 2021 International Conference on Multimodal Interaction, 2021
We developed a haptically-enhanced physics simulation to investigate the effects of haptics on th... more We developed a haptically-enhanced physics simulation to investigate the effects of haptics on the understanding of conceptual concepts related to forces—specifically those related to buoyancy. We evaluated the effects of haptic force feedback, as well as traditional visual representations of forces, on learning via a between-participant user study. Participants completed a buoyancy assessment before and after interacting with the simulation. Haptics enhanced performance regardless of prior knowledge. However, the combined effect of haptics with visual cues differed based on participant prior knowledge. Participants with high prior knowledge significantly improved performance when given both abstract visual cues and haptic feedback combined. Participants with low prior knowledge significantly improved when given haptic feedback alone, and the combination of haptics with visual cues did not improve performance. Our results suggest that the prior knowledge of users and the visual cues used impact the effectiveness of haptically-enhanced simulations with respect to learning outcomes.
Uploads
Papers by David Borland