LETTER

doi:10.1038/nature09521

Holographic three-dimensional telepresence using large-area photorefractive polymer

Holography is a technique that is used to display objects or scenes in three dimensions. Such three-dimensional (3D) images, or holograms, can be seen with the unassisted eye and are very similar to how humans see the actual environment surrounding them. The concept of 3D telepresence, a real-time dynamic hologram depicting a scene occurring in a different location, has attracted considerable public interest since it was depicted in the original Star Wars film in 1977. However, the lack of sufficient computational power to produce realistic computer-generated holograms1 and the absence of large-area and dynamically updatable holographic recording media2 have prevented realization of the concept. Here we use a holographic stereographic technique3 and a photorefractive polymer material as the recording medium4 to demonstrate a holographic display that can refresh images every two seconds. A 50 Hz nanosecond pulsed laser is used to write the holographic pixels5. Multicoloured holographic 3D images are produced by using angular multiplexing, and the full parallax display employs spatial multiplexing. 3D telepresence is demonstrated by taking multiple images from one location and transmitting the information via Ethernet to another location where the hologram is printed with the quasi-real-time dynamic 3D display. Further improvements could bring applications in telemedicine, prototyping, advertising, updatable 3D maps and entertainment. 3D display technology is attracting much public attention; events include the recent release of 3D films such as Avatar, the 2008 US election-night hologram reporter interviews from CNN (http:// www.cnn.com/2008/TECH/11/06/hologram.yellin/index.html), and the demonstration of 3D televisions by some manufacturers (http:// www.3dtvsource.com/). As dramatic as these effects are, the technology used is based on polarization stereoscopy (the technique currently used in cinemas and television), digital image fusion (in the case of CNN), or 2D semitransparent screens (for musion; http://www. musion.co.uk/). As such, they have little to do with holography, which is the reproduction of the amplitude and phase of light by diffraction6. Nevertheless, these examples show the great enthusiasm that the public, media and industry share about 3D image rendering, and for a good reason: the human physiology has adapted to observe its environment in three dimensions. Holography, with its ability to reconstitute both the intensity and wave front information of a scene, allows the observer to perceive the light as it would have been scattered by the real object itself7. Furthermore, there is no need for any special eyewear to be worn by the observer. It has been shown that holograms can be computer generated8. Unfortunately, the amount of information needed to produce a high quality hologram is so large that making a real-time video-rate display has been limited by either size or resolution911. To overcome those major issues, different solutions have been tested, such as pupil tracking12, the use of a holographic diffuser screen13,14 or the synthesis of a hologram from real objects15. It has also been shown that the holographic stereographic technique16 (also referred to as integral holography), using the diffraction of light to reproduce both parallax
1

and occlusion clues (but not reproducing the phase), can be used to ease data management. Compared to normal stereograms or anaglyphs, holographic stereography does not require the viewer to use eyewear to perceive the 3D effect. The technique reconstitutes multiple perspectives that the observer can experience by looking at the screen from different angles. Applied to permanent holographic media such as silver halide films or photopolymers, holographic stereography is capable of providing excellent resolution and depth reproduction on large-scale 3D static images (http://www.zebraimaging.com): but dynamic updating capability has been missing until now. Photorefractive inorganic crystals have been used in the past to demonstrate refreshable holographic displays17,18. However, such systems suffer from the disadvantage that crystalline materials are not scalable in size owing to their laborious growth process, and thus are not well suited for display applications. Our group has introduced an updatable holographic 3D display based on the holographic stereographic technique using photorefractive polymeric materials19. Although this was an important step towards dynamic holography, the system had several limitations, including being monochromatic and having a low refresh rate (more than 4 min per image)20,21; the rapid updating needed for video rates or 3D telepresence was not possible. Here we describe a new holographic 3D display based on a novel photorefractive material capable of refreshing images every two seconds, making it the first to achieve a speed that can be described as quasi-realtime. The system is based on a pulsed laser holographic recording system and a new sensitized photorefractive polymer with remarkable holographic properties. Each holographic pixel, known as a hogel5, is written with a single nanosecond laser pulse. As opposed to 2D pixels, hogels contain 3D information from various perspectives. We also demonstrate multi-colour capability using angular multiplexing and

1.0 Diffraction efficiency (%) 0.8 0.6 0.4 0.2 0.0 0 50 100 150 200 Time (s) 250 300

Figure 1 | Example of diffraction efficiency dynamics under single nanosecond pulse writing. The pulsed energy at the sample location was 650 mJ cm22 (sum of both beams). Applied voltage to the photorefractive device was 7 kV and kept constant during the whole measurement.

College of Optical Sciences, The University of Arizona, Tucson, Arizona 85721, USA. 2Nitto Denko Technical Corporation, Oceanside, California 92054, USA.

8 0 | N AT U R E | VO L 4 6 8 | 4 NO V E M B E R 2 0 1 0

2010 Macmillan Publishers Limited. All rights reserved

LETTER RESEARCH
a b c

Figure 2 | Image from a hologram recorded with the pulsed system. Panels a, b, c respectively show images observed by the camera when pointed to the left, straight ahead, and to the right. This shows the 3D nature of the image (a

model aeroplane) by demonstrating parallax and occlusions. Supplementary Movie 1 shows the recording at 50 Hz and the display of the hologram.

full parallax at the same speed as horizontal-parallax-only (HPO). 3D telepresence is demonstrated with the HPO system, whereby 2D images were taken at multiple angles from one place and sent to another location using the Ethernet communication protocol, and then printed with the holographic set-up. To the best of our knowledge this is the first demonstration of holographic 3D telepresence. Our earlier rewritable 3D display19 used a continuous-wave frequencydoubled Nd:YAG laser as the recording source. The recording time was about one second per hogel, and the scanning system (used to shift from one hogel position to the next) needed to be stopped and damped each time to avoid vibration, resulting in a time-consuming process. As a result, the overall recording time for a 4 inch 3 4 inch hologram consisting of 120 hogels was of the order of 3 min. An erase time of 1 min was needed before refreshing the image. The system suffered from high sensitivity to ambient noise (such as vibration and air turbulence) and to thermal expansion, requiring a fully enclosed air damped optical table. In this work we have used a 6-ns pulsed laser system delivering 200 mJ per pulse at a repetition rate of 50 Hz. A single pulse is used to record an individual hogel in the photorefractive polymer. The writing time is reduced by several orders of magnitude, recording a 4 inch 3 4 inch sample in about 2 s, with 1-mm hogel resolution. The overall holographic recording set-up is also insensitive to vibration owing to the short pulse duration. As a result, the system is capable of operation in an industrial environment without any special need for vibration, noise or temperature control. The holographic recording material used here is a dynamic photorefractive polymer with the following structure and composition: a copolymer with a polyacrylic backbone was used to attach pendant groups, tetraphenyldiaminobiphenyl (TPD) and carbaldehyde aniline (CAAN) in the ratio 10:1 (TPD/CAAN)22. This whole copolymer structure is then referred to as PATPD/CAAN. A plasticizer, 9-ethyl carbazole (ECZ), was added to the composite to lower the glass transition temperature and facilitate chromophore alignment in the photorefractive space charge field. The index modulation properties are enhanced by adding a fluorinated dicyanostyrene (FDCST) chromophore. In order to sensitize the photorefractive material to the nanosecond pulse, we doped the polymer with a fullerene derivative, PCBM ([6,6]-phenyl-C61-butyric acid methyl ester). The photorefractive device was made by melting the final composite PATPD/CAAN:FDCST:ECZ:PCBM (49.5:30:20:0.5 wt%) between two ITO (indium tin oxide)-coated glass plates, with 100 mm spacers used to fix the film thickness. The ITO electrodes are use to apply an external electric field to the sample for the photorefractive effect to happen. Long-term stability against both crystallization and dielectric breakdown has been observed with this material, with samples used for months without any sign of degradation. Figure 1 shows the dynamics of the diffracted signal from the photorefractive polymer in a four wave mixing configuration for a single 6-ns writing pulse. A single pulse is emitted at time t 5 0 and saturates the detector (initial vertical line on the figure). The diffraction grating grows for the first 50 s, followed by a slow decay in the dark during

which the diffraction can clearly be observed for several minutes. In the actual display set-up, slow dark decay is prevented and the decay time becomes much shorter because (1) the decay time of photorefractive polymeric materials is much faster under illumination than in the dark, and (2) rewriting a new image erases the previous image. The slow (50 s) rise time dynamic, occurring on timescales much longer than the pulse duration, is due to the photorefractive process that involves several phenomena taking place after the pulse passing, such as photoconduction, charge trapping, and molecular orientation23. We note that our device is self-developing and there is no need for additional material processing, as opposed to the silver halides or photopolymers used in static holography. The 3D image is viewed using an incoherent coloured LED and is clearly visible under ambient room light. Pictures of a hologram recorded with the pulsed system are presented in Fig. 2. Supplementary Movie 1 shows the recording process taking place in about 2 s and camera movement further demonstrates the 3D nature of the image. The hologram fades away after a couple of minutes by natural dark decay, or it can be erased by recording the new 3D images. When recording a new image, the new interference pattern erases the old pattern, resulting in the formation of a new diffraction structure. To achieve a multi-colour holographic 3D display, we used angular multiplexing. Up to three different holograms are written in the material at different angles and read out with different coloured LEDs (redgreen-blue for full colour). The reference beam angles were set more than 10u apart to avoid crosstalk between the colours during readout. In order to precisely superimpose the different coloured diffracted beams, we used Braggs law to determine the required 5u angle between the different object beams, which helps to compensate for the angular dispersion due to writing and reading at different wavelengths. The three holograms are recorded simultaneously, so the recording speed for the coloured holograms is the same as for monochromatic holograms. We used orthogonal polarizations and symmetrical incidence to avoid ghost diffraction resulting from the interference between the different object beams. When beams are orthogonally polarized, they do not interfere to create an intensity modulation pattern and do not form a hologram in the material. A unique property of photorefractive polymers is that beams with incidence angles that are symmetric with
a b

Figure 3 | Pictures of coloured holograms. a, Hologram of two model cars recorded on a 12-inch-diameter photorefractive device in HPO geometry. b, Hologram of a vase and flowers.
4 NO V E M B E R 2 0 1 0 | VO L 4 6 8 | N AT U R E | 8 1

2010 Macmillan Publishers Limited. All rights reserved

RESEARCH LETTER
a
Reference beam HOE

Objective beam

Lens array Photorefractive device

Figure 5 | Telepresence system. a, Picture of a hologram recorded with the 3D telepresence system. Supplementary Movie 2 shows the recording and the display of the telepresence hologram. b, Picture of a functional prototype of a 12 inch 3 12 inch photorefractive device, held by W.-Y.H. Images of M.Y. (a) and W.-Y.H. (b) are used with permission.

Figure 4 | Full parallax recording sketch. a, The lens array focuses the object beam onto the photorefractive device. The reference beam is redirected and focused by the HOE so that the lens array collimates the reference beams and they intersect the object beams in the plane of the device. be, Various perspectives of a full parallax hologram representing a castle and towers. Recording was done by simultaneously writing 100 hogels; perspectives are respectively up right (b), up left (c), down right (d) and down left (e).

respect to the direction of the external field vector do not produce a hologram. The combination of both principles allows the use of three independent pairs of recording beams for the red-green-blue colours. Our photorefractive polymers are mostly transparent in the visible region of the spectrum, allowing us to effectively reproduce full colour in the transmission geometry. We have recorded colour holograms reproducing vivid colours, as shown in Fig. 3. The colour reproduction of the hologram can be changed according to the reading light sources used. The diffraction efficiency for each hologram is 0.5%, which is large enough to be viewed under ambient light conditions. As perspective is mostly provided to human vision through the eyes horizontal separation, HPO is acceptable for most 3D applications. However, full parallax is required for even better rendering or for some specific applications, such as tabletop displays that can be observed from all positions. The challenge of full parallax derives from the number of hogels to be recorded, namely the square of that needed in HPO. For example, for a 4 inch 3 4 inch screen and 1-mm resolution,
8 2 | N AT U R E | VO L 4 6 8 | 4 NO V E M B E R 2 0 1 0

HPO requires 100 hogels whereas full parallax requires 10,000 hogels. Even with the pulsed laser system we introduce here, the recording time becomes more than 3 min for full parallax. To overcome this limitation, we used a lens array together with a holographic optical element to spatially multiplex the recording beams. A sketch of the optical system is presented in Fig. 4a. The focal distance and the diffraction angle of the HOE were carefully designed to match the lens array focal length and lens separation so the reference beams are collimated. By using this system, we demonstrated the simultaneous recording of 100 hogels with a single pulse (Fig. 4b), decreasing the recording time by the same factor. A raster scan driven by high-speed translation stages was used to cover the entire area of the photorefractive device. The surface area illuminated with the multiplexing setup in a single exposure (1 cm2) is the same as in the HPO geometry (1 mm 3 10 cm 5 1 cm2), but the spatial distribution is different (multiple squares instead of a line). Thus, the same laser source with an identical power can be used to obtain equivalent diffraction efficiency. Figure 4c shows resulting image obtained by this full parallax system. Holographic cinematography has previously been demonstrated24, but the need to process the recording media after exposure (using silver halides or photopolymer) prevents real-time operation and creates a large delay between recording and replay. With the new photorefractive polymer we describe here, quasi-real-time recording and replaying is achieved. Hogel generation in holographic stereography can be performed at video rate and does not require much computational power, as opposed to computer-generated holograms. The real-time recording and viewing, together with the lack of a requirement for extensive computational power, opens the door to new applications, including 3D telepresenceas we now demonstrate. Our 3D telepresence set-up consists of 16 Firewire cameras that take simultaneous pictures of a real 3D scene every second. The 16 views are processed into hogel data by the host computer, and sent to the holographic recording controller through a 100 Mbit s21 Ethernet link that is used at less than 10% of its capacity. For a 4 inch 3 4 inch display, 120 hogels are processed for HPO conditions and sent every second using a general-purpose desktop PC. The pulsed holographic recording set-up is used to continuously write the hologram according to the flow of data, so that the 3D images are refreshed continuously. Once a hologram has been written, the system uses the next available hogels to update the information. The hologram is read by light from a colour LED incident at the Bragg angle. The advantage of the holographic transmission geometry we are using is that the viewer is able to see the hologram all the time, as reading and writing occur simultaneously. The writing beams are on the opposite side of the screen with respect to the diffracted read out light; the writing beams are therefore blocked from the viewer, which is

2010 Macmillan Publishers Limited. All rights reserved

LETTER RESEARCH
advantageous for safety issues. Furthermore, the monochromatic writing light is prevented from reaching the viewer with an optical filter. There is no interruption in the process, with a full 3D image being continuously displayed. As described earlier, the natural (dark) decay of the hologram is longer than the decay on illumination and thus there is no noticeable difference in intensity between the previous (old) and the new 3D image. The reference beam intensity in the writing set-up is such that it erases the old image with no residual ghost image being present after refreshing. An example of a telepresence hologram is shown in Fig. 5a. Supplementary Movie 2 shows the recording and display of holographic telepresence. We have provided here the proofs of concept for the material and the techniques of dynamic near-real-time holographic 3D display and telepresence; we expect this development to lead to new applications of holographic 3D technology. As an example, in telemedicine and especially for brain surgery, surgeons at different locations around the world could use the technique to observe in three dimensions, in real time, and to participate in the surgical procedure.
Received 4 May; accepted 14 September 2010. Slinger, C., Cameron, C. & Stanley, M. Computer-generated holography as a generic display technology. Computer 38, 4653 (August 2005). 2. Ketchel, B. P. et al. Three-dimensional color holographic display. Appl. Opt. 38, 61596166 (1999). 3. Debitetto, D. J. Holographic panoramic stereograms synthesized from white light recordings. Appl. Opt. 8, 17401741 (1969). 4. Thomas, J., Norwood, R. A. & Peyghambarian, N. Non-linear optical polymers for photorefractive applications. J. Mater. Chem. 19, 74767489 (2009). 5. Lucente, M. Diffraction-Specific Fringe Computation for Electro-Holography. PhD thesis, MIT (1994). 6. Ackermann, G. K. & Eichler, J. Holography: A Practical Approach (Wiley-VCH, 2007). 7. Benton, S. A. & Bove, V. M. Jr. Holographic Imaging (Wiley-InterScience, 2008). 8. Waters, J. P. Holographic image synthesis utilizing theoretical methods. Appl. Phys. Lett. 9, 405407 (1966). 9. St., Hilaire, P., Lucente, M. & Benton, S. A. Synthetic aperture holography: a novel approach to three dimensional displays. J. Opt. Soc. Am. A 9, 19691978 (1992). 10. Slinger, C. W. et al. Recent developments in computer-generated holography: toward a practical electroholography system for interactive 3d visualization. Proc. SPIE 5290, 2741 (2004). 1. 11. Liu, Y.-Z. et al. High-speed full analytical holographic computations for true-life scenes. Opt. Express 18, 33453351 (2010). ussler, R., Schwerdtner, A. & Leister, N. Large holographic displays as an 12. Ha alternative to stereoscopic displays. Proc. SPIE 6803 (2008). th, 13. Balogh, T. et al. in Proceedings of the TMCE 2008 (eds Horva I. & Rus, Z.) 111 (Faculty of Industrial Design Engineering, Delft University of Technology, 2008). 14. Sang, X. et al. Demonstration of a large-size real-time full-color three-dimensional display. Opt. Lett. 34, 38033805 (2009). 15. Paturzo, M. et al. Synthesis and display of dynamic holographic 3D scenes with real-world objects. Opt. Express 18, 88068815 (2010). 16. Chen, S.-T. & Chatterjee, M. R. Implementation of a spatially multiplexed pixilated three-dimensional display by use of a holographic optical element array. Appl. Opt. 37, 75047513 (1998). 17. Shin, S.-H. & Javidi, B. Speckle-reduced three-dimensional volume holographic display by use of integral imaging. Appl. Opt. 41, 26442649 (2002). 18. Ketchel, B. P. et al. Three-dimensional color holographic display. Appl. Opt. 38, 61596166 (1999). 19. Tay, S. et al. An updatable holographic three-dimensional display. Nature 451, 694698 (2008). 20. Blanche, P.-A. et al. An updatable holographic display for 3D visualization. IEEE J. Display Technol. 4 (special issue on medical displays) 424430 (2008). 21. Peyghambarian, N., Tay, S., Blanche, P.-A., Norwood, R. & Yamamoto, M. Rewritable holographic 3D display. Opt. Photon. News, 19, 2227 (July/August 2008). 22. Thomas, J. et al. Bis-triarylamine polymer-based composites for photorefractive applications. Adv. Mater. 16, 20322036 (2004). 23. Eralp, M. et al. Submillisecond response of a photorefractive polymer under single nanosecond pulse exposure. Appl. Phys. Lett. 89, 114105 (2006). 24. DeBitetto, D. J. A front-lighted 3-D holographic movie. Appl. Opt. 9, 498499 (1970). Supplementary Information is linked to the online version of the paper at www.nature.com/nature. Acknowledgements We acknowledge support from AFOSR, DARPA and the NSF ERC Center on Integrated Access Networks (CIAN). Author Contributions P.-A.B. and A.B. did experimental work. R.V. did modelling and software. C.C., P.W. and M.K. did experimental work. W.L., T.G., D.F., W.-Y.H., B.R., O.S. and J.T. did sample preparation. R.A.N. and Y.Y. are team leaders. N.P. did project planning and management. Author Information Reprints and permissions information is available at www.nature.com/reprints. The authors declare no competing financial interests. Readers are welcome to comment on the online version of this article at www.nature.com/nature. Correspondence and requests for materials should be addressed to N.P. ([email protected]).

4 NO V E M B E R 2 0 1 0 | VO L 4 6 8 | N AT U R E | 8 3

2010 Macmillan Publishers Limited. All rights reserved