2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
We introduce Ego4D, a massive-scale egocentric video dataset and benchmark suite. It offers 3,025... more We introduce Ego4D, a massive-scale egocentric video dataset and benchmark suite. It offers 3,025 hours of dailylife activity video spanning hundreds of scenarios (household, outdoor, workplace, leisure, etc.) captured by 855 unique camera wearers from 74 worldwide locations and 9 different countries. The approach to collection is designed to uphold rigorous privacy and ethics standards with consenting participants and robust de-identification procedures where relevant. Ego4D dramatically expands the volume of diverse egocentric video footage publicly available to the research community. Portions of the video are accompanied by audio, 3D meshes of the environment, eye gaze, stereo, and/or synchronized videos from multiple egocentric cameras at the same event. Furthermore, we present a host of new benchmark challenges centered around understanding the first-person visual experience in the past (querying an episodic memory), present (analyzing hand-object manipulation, audiovisual conversation, and social interactions), and future (forecasting activities). By publicly sharing this massive annotated dataset and benchmark suite, we aim to push the frontier of first-person perception.
As head-mounted displays (HMDs) commonly present a single, fixed-focus display plane, a conflict ... more As head-mounted displays (HMDs) commonly present a single, fixed-focus display plane, a conflict can be created between the vergence and accommodation responses of the viewer. Multifocal HMDs have long been investigated as a potential solution in which multiple image planes span the viewer's accommodation range. Such displays require a scene decomposition algorithm to distribute the depiction of objects across image planes, and previous work has shown that simple decompositions can be achieved in real-time. However, recent optimal decompositions further improve image quality, particularly with complex content. Such decompositions are more computationally involved and likely require better alignment of the image planes with the viewer's eyes, which are potential barriers to practical applications. Our goal is to enable interactive optimal decomposition algorithms capable of driving a vergence- and accommodation-tracked multifocal testbed. Ultimately, such a testbed is necessa...
Introduction Cooperation refers to a behaviour that benefits another individual12. Such behaviour... more Introduction Cooperation refers to a behaviour that benefits another individual12. Such behaviours are costly. To be repeated and preserved through time (ie 'selected for' in evolutionary terms) the benefits must outweigh these costs in some way. One of the most easily understood ...
The International Telecommunications Union Radiocommunication Sector has recommended a broadcasti... more The International Telecommunications Union Radiocommunication Sector has recommended a broadcasting standard Recommendation ITU-R BT.2020 (Rec. 2020) for ultrahigh-definition television that is aimed at providing a better visual experience. It recommends a color gamut that exceeds previous broadcasting standards and is only achievable by laser-based display technologies. Quantum dot (QD)-enabled liquid crystal displays (LCDs) provide one alternative with potential to meet Rec. 2020's standard color gamut while taking advantage of existing manufacturing capacity. We examined how existing QD and LCD technology could be optimized to meet the Rec. 2020 color standard. Our analysis revealed that up to 94% gamut coverage can be achieved.
ABSTRACT Quantum dot technology offers the promise of efficient and relatively inexpensive liquid... more ABSTRACT Quantum dot technology offers the promise of efficient and relatively inexpensive liquid crystal displays (LCDs) with large color gamuts (∼115% NTSC u'v'). Now, for the first time, a variety of high color performance devices are on the immediate horizon for applications ranging from 3“ diagonal hand held smart phones to +85” diagonal TVs . While people clearly prefer color displays, the relative value of different color reproduction systems remains an open question. Here we review how quantum dots enable high color performance and some of the advantages of large color gamut displays.
Advances in display technology depend on our ability to develop miniscule elements that emit a br... more Advances in display technology depend on our ability to develop miniscule elements that emit a broad range of light intensities and colors. Photometry and colorimetry provide tools to help developers and manufacture chose these targets for future and existing technology. Uncertainty remains however as to which metrics provide the best guidelines. We examined the relationship between 8 color metrics and human preferences for displays that differed only in color gamut. We found that (1) volume metrics, computed from display luminance and color capacity, outperformed area metrics computed only from color and (2) of the color metrics we considered, CIECAM'02 saturation performed best.
2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
We introduce Ego4D, a massive-scale egocentric video dataset and benchmark suite. It offers 3,025... more We introduce Ego4D, a massive-scale egocentric video dataset and benchmark suite. It offers 3,025 hours of dailylife activity video spanning hundreds of scenarios (household, outdoor, workplace, leisure, etc.) captured by 855 unique camera wearers from 74 worldwide locations and 9 different countries. The approach to collection is designed to uphold rigorous privacy and ethics standards with consenting participants and robust de-identification procedures where relevant. Ego4D dramatically expands the volume of diverse egocentric video footage publicly available to the research community. Portions of the video are accompanied by audio, 3D meshes of the environment, eye gaze, stereo, and/or synchronized videos from multiple egocentric cameras at the same event. Furthermore, we present a host of new benchmark challenges centered around understanding the first-person visual experience in the past (querying an episodic memory), present (analyzing hand-object manipulation, audiovisual conversation, and social interactions), and future (forecasting activities). By publicly sharing this massive annotated dataset and benchmark suite, we aim to push the frontier of first-person perception.
As head-mounted displays (HMDs) commonly present a single, fixed-focus display plane, a conflict ... more As head-mounted displays (HMDs) commonly present a single, fixed-focus display plane, a conflict can be created between the vergence and accommodation responses of the viewer. Multifocal HMDs have long been investigated as a potential solution in which multiple image planes span the viewer's accommodation range. Such displays require a scene decomposition algorithm to distribute the depiction of objects across image planes, and previous work has shown that simple decompositions can be achieved in real-time. However, recent optimal decompositions further improve image quality, particularly with complex content. Such decompositions are more computationally involved and likely require better alignment of the image planes with the viewer's eyes, which are potential barriers to practical applications. Our goal is to enable interactive optimal decomposition algorithms capable of driving a vergence- and accommodation-tracked multifocal testbed. Ultimately, such a testbed is necessa...
Introduction Cooperation refers to a behaviour that benefits another individual12. Such behaviour... more Introduction Cooperation refers to a behaviour that benefits another individual12. Such behaviours are costly. To be repeated and preserved through time (ie 'selected for' in evolutionary terms) the benefits must outweigh these costs in some way. One of the most easily understood ...
The International Telecommunications Union Radiocommunication Sector has recommended a broadcasti... more The International Telecommunications Union Radiocommunication Sector has recommended a broadcasting standard Recommendation ITU-R BT.2020 (Rec. 2020) for ultrahigh-definition television that is aimed at providing a better visual experience. It recommends a color gamut that exceeds previous broadcasting standards and is only achievable by laser-based display technologies. Quantum dot (QD)-enabled liquid crystal displays (LCDs) provide one alternative with potential to meet Rec. 2020's standard color gamut while taking advantage of existing manufacturing capacity. We examined how existing QD and LCD technology could be optimized to meet the Rec. 2020 color standard. Our analysis revealed that up to 94% gamut coverage can be achieved.
ABSTRACT Quantum dot technology offers the promise of efficient and relatively inexpensive liquid... more ABSTRACT Quantum dot technology offers the promise of efficient and relatively inexpensive liquid crystal displays (LCDs) with large color gamuts (∼115% NTSC u'v'). Now, for the first time, a variety of high color performance devices are on the immediate horizon for applications ranging from 3“ diagonal hand held smart phones to +85” diagonal TVs . While people clearly prefer color displays, the relative value of different color reproduction systems remains an open question. Here we review how quantum dots enable high color performance and some of the advantages of large color gamut displays.
Advances in display technology depend on our ability to develop miniscule elements that emit a br... more Advances in display technology depend on our ability to develop miniscule elements that emit a broad range of light intensities and colors. Photometry and colorimetry provide tools to help developers and manufacture chose these targets for future and existing technology. Uncertainty remains however as to which metrics provide the best guidelines. We examined the relationship between 8 color metrics and human preferences for displays that differed only in color gamut. We found that (1) volume metrics, computed from display luminance and color capacity, outperformed area metrics computed only from color and (2) of the color metrics we considered, CIECAM'02 saturation performed best.
Uploads
Papers by James Hillis