On Using Off-the-Shelf Micro Projectors For 3D Metrology
On Using Off-the-Shelf Micro Projectors For 3D Metrology
On Using Off-the-Shelf Micro Projectors For 3D Metrology
Metrology
Martin Lenz, Matthias Rüther and Horst Bischof
Abstract
Modern projectors offer sufficient light intensity, contrast and resolution to perform highly accurate
3D measurements on a large variety of surfaces by projecting coded patterns observed by cameras.
Yet, for mobile measurement systems or inspections of specimen which are hard to reach, common
projecting devices are often too heavy and too bulky.
Recently, a new projection technology has emerged, employing a reflecting Liquid Crystal Display
(LCD) panel called Liquid Cristal on Silicon (LCoS) which allows to build small and light micro
projectors. These new devices have a low power consumption and are comparatively cheap. Since no
previous experience with these devices for metrology applications is available, an off-the-shelf micro
projector is evaluated regarding its capability in a structured light setup.
1 Introduction
In computer vision, 3D reconstruction using stripe projecting devices is a popular technique. Espe-
cially in industrial measurement applications, structured light is a common tool to inspect surfaces of
various samples.
Existing devices, which employ LCD panels or digital micromirror devices (DMD), are well-established
in 3D metrology systems. On the one hand, they are capable of performing structured light regarding
light intensity, resolution and contrast. On the other hand, DMD or LCD based sensors with a size
of at least 150mm × 50mm × 100mm and a weight of more than 1kg, are not suitable for mobile or
robotic applications. Here, a compact, light and power efficient measurement sensor is desirable.
Within the last years, a new technology called Liquid Cristal on Silicon (LCoS) has emerged. This
technology allows the production of small and cheap projecting units for pocket beamers and mobile
phones. Recently, the first micro projectors were commercially available. The big advantage of these
devices besides size and weight is the fact that they have a low power consumption which even per-
mits to run them in battery mode. One big drawback of available pocket beamers is the poor native
resolution.
Until now, no experiences with micro projectors used in computer vision are available. In this work
we evaluate the performance of an off-the-shelf pocket projector in a compact structured light setup
for 3D reconstruction (see Figure 1).
Related work on 3D reconstruction using structured light is described in Section 2 and the recon-
struction principle is sketched in Section 3. A brief overview about projector technology of DMD and
the newer LCoS devices is explained in Section 4. An elaborate analysis of the projector’s switching
frequency, temperature caused effects and its capability in a structured light setup are presented in
(a) LCoS micro projecting unit. (b) Experimental structured light setup with stereo
cameras and a micro projector inspecting an ob-
ject.
Figure 1: Compact prototype of a structured light sensor using a micro projector with LCoS technology as projec-
tion unit.
Section 5.
2 Related Work
At the moment, several commercially available optical 3D measurement systems exist. An overview
about existing systems is presented in this Section.
The Fraunhofer Institute for Applied Optics and Precision Engineering developed a dedicated 3D
measurement system called Kolibri ROBOT based on structured light. The system is non-stationary
and can be carried by an industrial robot arm. Measurement areas of 460mm × 350mm can be in-
spected within 3-5 seconds and an uncertainty of 25µm [5].
A miniaturized version of the Kolibri ROBOT is the Kolibri CORDLESS, a measurement system with
the size of a shoebox and a weight of about 1kg. The sensor consists of a projecting unit and two cam-
eras and offers a hand-held operation mode with a WLAN connection to a stationary base unit, permit-
ting the reconstruction of areas which would be inaccessible for common systems. The measurement
of a single sequence can be done within 0.25 seconds in a measurement area of 220mm × 170mm
and an accuracy of 50µm-150µm [4].
A competitive product, the z-Snapper, is offered by Vialux. A single camera and a LED projector
using the DMD technology (see Section 4.1) is used to perform 3D measurement procedures with
structured light and a phase shift algorithm as explained in Section 3. The sensor is able to run in
battery mode and exhibits an accuracy of 20µm-50µm in a field of view of 350mm [11].
Another small 3D sensor based on the structured light principle is the ATOS system with a dimen-
sion of 490mm × 300mm × 170mm. A variable measurement area from 150mm × 150mm to
2000mm × 2000mm can be inspected within 2 seconds. About 4 million points are sampled with the
area resulting in a point distance of 0.07mm-1mm [3].
The MiniRot-Kombi system from ABW is a projector-camera measurement sensor using fringe pro-
jection. It generates 3D data within one second over a measurement area of 600mm × 400mm. The
projector’s light source is a 150W halogen bulp. The sensor’s dimension is 75mm×200mm×600mm
and its weight is 7.4kg [1].
3 Structured Light
Structured light is a 3D computer vision reconstruction method using one or more cameras and a
projector. In [10], Salvi gives an extensive overview of existing techniques. Depth information of
surfaces can be gathered by consecutively projecting known patterns onto objects while observing
them with cameras (as shown in Figure 2(a)). Point correspondences between the projector and the
observing device can then be extracted by calculating a code for each observed pixel.
In a calibrated camera-projector setup, each image pixel is intersected with the projector pixel column
or row uniquely identified by the pattern code. The limiting factor regarding reconstruction accuracy
in this case is usually represented by the projector’s resolution. Therefore, sub-pixel methods as
explained in [2] can be applied using phase shift procedures in such a way that the effective resolution
can be reduced to 1/28 of the physical resolution. According to [6], a reduction to 1/64 of the actual
line width is possible.
4 Technology
State-of-the-art projectors employ a variety of technologies such as static projections, Liquid Crystal
Display (LCD) panels, Digital Micromirror Devices (DMD) or recently the Liquid Crystal on Silicon
(LCoS) panels. In this section, the main principles of DMDs (as mainly used for 3D metrology tasks)
and the novel LCoS projectors are explained.
A well-established projector technology uses Digital Micromirror Devices (DMDs). This principle
is also known as Digital Light Processing (DLP) and is a reflective display technology consisting
of an array of micro-mirrors representing projector pixels. According to [9], images are created by
redirecting light beams (usually originating from a halogen lamp or recently from an LED) through
the projector’s lens onto the projecting surface (see Figure 3). Depending on the chip resolution, the
mirrors on the array have a size of 10-16 microns and can be tilted from -12 degrees to 12 degrees
separately within a few microseconds. Constantly switching each mirror at a frequency of several
kHz and varying the duty cycle permits the control of the pixel’s intensity by directing the beam of
light to the projector’s lens or into a light trap (which turns off the light for a certain pixel) [8].
The generation of colored images can be achieved by using either a color wheel with a single DMD
Figure 3: Schematic DLP technology with a single chip DLP unit in combination with a color wheel (left) and
a three-chip architecture for RGB (right) as illustrated in [9]. The DMD chip, consisting of an array of tilting
mirrors, is able to control light intensities by redirecting beams of light either to a heatsink (to absorb intensity) or
to the projection lens.
or by using three DMDs in combination with a prism-based color separator. When using only one
DMD, light is absorbed by the color wheel. For that reason, projectors using three DMDs exhibit
higher light intensities but usually are more expensive and bigger in size. In general, DMD projectors
offer good image quality regarding contrast and light intensity. The projected images tend to be very
crisp because of a very narrow pixel arrangement on the chip compared with LCD panels and the
hardware does not degrade after long term operation. A disadvantage of using a single DMD device
is the weak color saturation and the appearance of rainbow effects caused by the rotating color wheel.
Recently, the Liquid Crystal on Silicon (LCoS) technology (which is also known as Direct-driven
Image Light Amplifier or short D-ILA) used for projectors became popular. Actually, this principle
already existed for several years but lately micro projecting units were developed which can be used
in mobile phones or as pocket beamers.
Instead of tiny mirrors that are used to reflect light as explained in Section 4.1, a liquid crystal display
is applied on a reflective mirror layer.
As illustrated in Figure 4 according to [9], images are created by redirecting a light source (usually
an LED) through a polarized beam splitter (PBS) onto the LCoS panel. Each pixel on the panel can
be switched to a transparent mode, reflecting light, or turned off to an opaque mode.
Alike DLP projectors, single panel solutions and three panel designs (for RGB) are available for the
LCoS principle. For lack of space, a single LCoS chip is provided for micro projectors only. In that
case, to produce color, an LCoS chip is used which provides micro color filters by dividing each pixel
into three sub-pixels (RGB) [7].
The big advantage of this technology is the possibility to build extreme small and light micro beamers.
Additionally, the power consumption of LCoS panels and LEDs is comparably low - pocket projectors
even run in battery mode. A disadvantage of this principle is that contrast and light intensity is also
low in micro projectors.
Figure 4: A single LCoS projection unit according to [9]. The LCoS chip consists of a reflective liquid crystal array
capable of switching each pixel between transparent and opaque to control the reflected light intensity.
In order to evaluate the capability of micro projectors in a structured light setup for industrial mea-
surement applications, several experiments are conducted. Generally spoken, a projector used for
structured light has to fulfill some requirements concerning signal quality, signal stability, projecting
speed, light intensity and contrast on varying surfaces. For the purpose of long-term operations, heat-
ing aspects have to be considered as well.
In this section, an off-the-shelf micro projector of the type 3M MPro 110 (see Figure 1(a)) is analyzed
regarding its signal characteristics, speed in switching signals between the minimal and maximal light
intensity, its long-term operability, its brightness and contrast.
In a first step, the projector’s signal quality is analyzed. This is done by observing a steadily projected
gray value on a planar surface with a grayscale CMOS camera. Image patches of 16×16 pixels on the
center of the beamer’s projection area are acquired at a framerate of about 300 frames per seconds.
The mean gray value of the image patch is stored for each received frame and is analyzed over time.
As illustrated in Figure 5.1, a signal oscillation can be observed.
The signal oscillates with a 60 Hertz frequency, leading to the assumption that this effect is caused by
the projector’s refreshing rate. The signal’s amplitude may vary depending on the currently projected
gray value and time as discussed later.
In order to receive a stable signal, an average filter over the length of the fivefold 60 Hertz oscillation
should be computed. When using the micro projector within a structured light setup, this averag-
ing can be done implicitly by using an exposure time of at least 83ms for the image acquisition of
projected patterns.
Since the structured light procedure requires the subsequent projection of varying stripe patterns,
switching times between patterns are crucial for the duration of the entire 3D reconstruction. The
faster frames can be projected, the shorter the reconstruction process duration. This is of utmost im-
portance for two reasons. Firstly, accurate measurements can only be done if there are no vibrations
(a) Periodical oscillation observed over time. (b) Signal in frequency spectrum after a fast
Fourier transformation.
Figure 5: Analysis of a steadily projected gray value with a CMOS camera at a framerate of about 300 frames per
second. The mean value over a 16 × 16 pixel patch is plotted over time as shown in Figure 5(a). After a Fourier
transformation (see Figure 5(b)), a frequency of 60 Hertz, the projector’s refresh rate, is clearly visible.
or movements that could change the relative position between the sensor and an object to inspect.
Secondly, industrial measurement tasks usually require tight cycle times since each lost second is
critical.
To find out the projector’s ability of switching patterns concerning speed, a high-speed CMOS cam-
era is installed, directly looking into the projector’s lens such that the projector’s dynamic range is
observable.
While grabbing 10k frames with the CMOS camera at 2137 frames per second and with a patch size
of 16 × 16 pixels, a black-white sequence is projected. A delay of 0.04 seconds between switching
the projected pattern from maximum to minimum intensity seemed to be sufficient for the projector
to reach its saturation level. Each acquired image patch is averaged and a mean gray value of the pro-
jected light intensity is stored. Figure 6 schematically illustrates the projection of a black-white-black
sequence and defines the nomenclature in the following.
An analysis of the gathered data showed that the mean rising time trising takes about four times
longer than the mean falling time tf alling . A detailed summary of the experiments on the switching
time can be found in Table 1. Switching from the projector’s minimum to its maximum intensity ap-
Table 1: Summary of the experiment on turning the projected pattern from the projector’s maximum intensity to
its minimum intensity with a delay of 0.04 seconds between the switching. An amount of 10k points are sampled
with a CMOS camera at a framerate of 2137 frames per second. The projected signal is subsequently switched 27
times from black to white. The nomenclature is defined according to Figure 6.
255
Imax
mean intensity
grising gfalling
A B C D
Imin
0 trising tfalling t
frames
Figure 6: Schematic illustration of a projected black-white-black sequence observed with a CMOS camera at a
framerate of 2137 frames per second. The mean values of a 16 × 16 pixels patch are plotted as dark dots over
time. Gray values Imax and Imin represent a 3σ threshold around the mean values of the projector’s saturation
level. The lines grising and gf alling describe the linear rising and falling edges between the saturated levels of the
projector, where the gradient of adjacent points is larger than a threshold. Regions A, B, C and D are denoted as
the periods lasting between the saturated levels and the regions where a linear rising and falling could be observed.
proximately lasts 21.7ms. The change from the maximum to the minimum intensity is done in about
5.1ms. These delays must be considered when using the micro projector in a structured light setup to
avoid observing an unstable signal.
5.3 Contrast
When a projector is used for structured light, contrast is a quite important attribute for the segmen-
tation of stripes in the reconstruction procedure. Especially when performing structured light with
phase shift, contrast between fine stripes is crucial. In an experimental evaluation, the contrast sig-
nal which can be observed with a CCD camera is investigated under stripe projections with varying
width.
A structured light setup is built with a 2.5 megapixels CCD gray scale camera and the micro pro-
jector sharing a field of view of about 50mm × 30mm. For the setup’s contrast evaluation, stripe
patterns with varying stripe thicknesses are projected and an image is acquired with the camera and
sufficient exposure time to suppress oscillation effects as mentioned above. Then, the inverted pattern
is projected and grabbed with the camera. A differential image between the regular stripes and the
inverted stripes is computed to receive the observable intensity difference for each pixel in the image.
This procedure is repeated 30 times for 9 different stripe patterns. Each mean signal and its noise is
analyzed. Figure 7 shows a section of the horizontal profile representing the mean signal and the 3σ
boundary when observing stripes at the finest resolution used during the structured light procedure.
The mean intensity difference for patterns width a thickness of 2 projector pixels is 41.47. Experi-
ments show that the mean intensity difference of broader stripe patterns even rises up to 205.8. Signal
noise only exhibits a mean of 1.66 to 2.1 gray values for the varying stripe resolutions. This leads to
the assumption that contrast quality is sufficient for structured light using binary patterns and noise
does not seriously interfere measurements, even for fine stripe resolutions.
Figure 7: Mean profile of the difference of observed intensities between a black and white stripe projection and
its inverted pattern after 30 repetitions. One stripe has a thickness of two projector pixels. The three-σ boundary
shows that the observable contrast is still high enough to perform measurements at a fine stripe resolution.
As mentioned before, a 60 Hertz oscillation of the projector signal could be clearly observed. The
amplitude of the oscillation depends on the projected gray value and the duration of the projection.
Because of that, a long-term test is done to analyze the effect of the oscillation and its influence.
Again, a CMOS camera is used to acquire data at approximately 250 frames per second, in the same
manner as for the signal quality test discussed above. This time, the projection of black and white is
changed in logarithmic time intervals over a period of roughly four hours. Since it is not possible to
monitor the micro projector’s maximum and minimum saturation with the same camera parameters,
the test is run twice - once for the evaluation of the projector while projecting black with opened
camera aperture and once for the white level with closed aperture.
For some reason, the signal’s amplitude rises from 3 to 15 gray values when steadily projecting black
with the micro projector (see Figure 5.4). Since the LCOS panel absorbs the light sourced from an
LED when pixels are set to black, we assume that there might emerge heating problems causing the
rising amplitude when steadily projecting a black pattern over a longer period.
Measuring temperature on the projector unit’s chassis during the long term test shows that the device
heats up from room temperature to 36◦ C within the first 50 minutes of the test, where it evens out.
Changes in temperature are not noticed between the switching from white to black.
Besides to oscillation effects, one can identify that the light intensity slightly drops during the test
sequence over several hours. The mean filtered gray value at the beginning of the test procedure in
the projector’s maximum saturation level is about 190. At the end of the test, almost four hours later,
the mean value is only in the range of 181. A direct correlation between the light intensity’s decrease
and temperature change is not noticeable.
Former experiments show that the projector’s signal oscillation can vary depending on the projected
gray values. For a standard structured light procedure using binary patterns only, the oscillation does
not influence measurements. Reason for that is the fact that only relative intensity changes between
(a) Oscillated signal and average signal in the projecting unit’s
maximum saturation level.
Figure 8: Mean filtered gray values of the long term test with black-white projections and the minimum and
maximum amplitude of the signal, switching in logarithmic time intervals. Due to camera restrictions, the micro
projector’s maximum and minimum saturation level is observed separately in two acquisition procedures while
projecting the same black-white sequences. During projection of black, the signals’s amplitude rises. The ampli-
tude decreases after projecting white light again. The averaged signal shows that the mean gray value of a stable
projection color may change noticeably.
Figure 9: Repeatability of projected gray values. Eight gray values are projected consecutively with the micro
projector onto a white and planar surface. A CCD camera, as used in a structured light setup, observes the mean
gray value of the projection. This step is repeated for one hour resulting in 574 measurements for each projected
gray value. The average values and their 3σ-boundaries are shown. The deviations for brighter gray values are
obviously stronger than those for darker gray values.
the maximum and the minimum projections are calculated which can clearly be distinguished even
with existing oscillations. When using structured light in combination with phase shifted patterns
(as described in Section 3), certain gray values must be projected to simulate a phase over several
projector pixels instead of pure binary stripes. This circumstance requires steady projections which
should be repeatable for several measurements.
As discussed before, the oscillation effect can be smoothed by simply exposing long enough with
observing cameras. If the micro projector’s signal exhibits an oscillation with a strong amplitude,
the signal indeed is not jammed by an oscillation when adjusting the camera’s exposure time, but
the mean intensity might be different from a former measurement with the same signal and a lower
amplitude.
Therefore, an experiment stating the beamer’s repeatability of certain gray values is done. For this
reason, a setup similar to a structured light setup is used to observe the micro beamer’s projection
on a white and planar surface. A list of certain gray values is subsequently projected onto the planar
surface and observed by a CCD camera. After storing the mean gray value of the projection area in
the camera’s image, the next gray value in the list is processed. This procedure is repeated for eight
gray values for the duration of one hour resulting in 574 measurements for each gray value. Figure 9
illustrates the experimental result.
As results show, the oscillation in brighter projections causes a clearer deviation from the actual
projection intensity. For the structured light procedure using phase shift, deviations of about 10 gray
values are still small enough to receive feasible results. Neighboring projected gray values of a phase
shift pattern still offer variabilities of at least 70 gray values which should even be distinguishable
with stronger amplitudes in the projector’s signal.
To evaluate the micro projector’s capability for industrial measurements, a structured light setup is
assembled. Therefore, the 3M MPro110 beamer (native resolution: 640 × 480) is set up with a focal
length of 26.8mm and a 2.5 megapixel grayscale CCD camera with a focal length of 16mm and an
opening angle of approximately 15 degrees between the two devices. This results in a measurement
volume of about 500mm × 375mm × 20mm. Intrinsic and extrinsic parameters are computed in a
calibration process for both devices.
In order to evaluate the structured light procedure’s accuracy with the micro beamer, a nominal planar
surface is reconstructed. A structured light algorithm with phase shifting is used. A plane is fitted
to the dense point cloud using a RANSAC algorithm with all points. The standard deviation from
all points to the plane can be used as a quality measure for the reconstruction. This procedure is
repeated using different exposure times during the acquisition phase to demonstrate the principle of
smoothing the signal oscillation with an observing camera. For comparability issues, the projected
stripe patterns are adapted concerning their intensity to measure the same mean gray value levels with
varying exposures. For detailed results see Table 2.
When using the recommended exposure time of at least 83ms, the projector’s signal oscillation can
Camera exposure time [ms] Plane fitting inliers [%] std.dev. from plane [µm]
15 95.80 0.0354
35 93.22 0.0326
85 97.41 0.0267
Table 2: Structured light results of reconstructing a planar surface (approximately 10.5cm2 ) with a point density
of 940 points per cm2 . The same region is reconstructed with varying camera exposure times and adapted pattern
intensities to suppress the signal oscillation. The standard deviation of the distances between all points and the
plane acts as error measure for the reconstruction.
be suppressed. This permits better reconstruction results since unavoidable oscillations are smoothed
and do not influence measurements critically. This mainly effects an accuracy gain for the phase shift
procedure since the repeatability of certain gray values is more stable in this case.
6 Conclusion
In this work, a cheap off-the-shelf LCoS micro projector is investigated regarding its capability in a
structured light setup. Several experiences could be gathered when dealing with the 3M MPro110
projector.
A big advantage is the micro projector’s size and weight. This circumstance allows an easy integra-
tion of the projection unit into a compact 3D measurement system. This provides the opportunity to
build non-stationary structured light sensors which permit reconstructions of regions that are difficult
to access.
A disadvantage of the projector is the unstable signal caused by a 60Hz oscillation. Because of this,
some workarounds need to be done to receive reliable measurement results with a structured light
setup. The oscillation can be handled by simply setting the observing camera’s exposure time to at
least 83ms. This results in a longer image acquisition procedure. The mean acquisition time of a sin-
gle pattern, consists of switching to the desired stripe pattern (in worst case: 21.7ms) and acquiring
an image (at least 83ms), lasts for not less than 104.7ms in theory.
Apart from the signal oscillation, the micro projecting unit seemed to be suitable for structured light
applications regarding contrast and light intensity. The beamer’s light source, a 5 Watt LED, is suffi-
ciently bright - especially for short distances between the projecting unit and the projected area.
The reconstruction of a planar surface shows the capability of the micro projector acting as projecting
unit in a structured light setup. When using the proposed exposure time of 85ms for the observing
device during the acquisition phase, admissible accuracies can be achieved at the expense of the re-
construction time. A planar surface of approximately 10.5cm2 is reconstructed with a deviation of
26.7µm and with a density of 940 points per cm2 .
In the future, micro projecting units for pocket beamers or mobile phones will probably capture the
market. Then, tiny and affordable projecting units will be present, sooner or later exhibiting higher
native resolutions which offers the possibility for high accuracy measurements with the help of such
devices in extremely compact sensors.
References
[1] ABW GmbH. MiniRot-Kombi - Optisches 3D-Aufnahme-System, February 2009. http:
//www.abw-3d.de/produkte/projektoren/PDFs/MiniRot-Kombi.pdf.
[2] C. Brenner, J. Boehm, and J. Guehring. Photogrammetric calibration and accuracy evaluation of
a cross-pattern stripe projector. Society of Photographic Instrumentation Engineers, 3641:164–
172, 1999.
[3] GOM - Gesellschaft fuer Optische Messtechnik. ATOS. Website, February 2009. http://
www.gom.com/DE/messsysteme/atos/system/varianten/atos.iii.html.
[4] Gunther Notni, Fraunhofer Institute for Applied Optics and Precision Engineering. kolibri
CORDLESS - Handheld cordless optical 3D measurement system. Website, February
2009. http://www.iof.fraunhofer.de/departments/optical-systems/
_media/KOLIBRI_CORDLESS_05_2007_E.pdf.
[5] Gunther Notni, Fraunhofer Institute for Applied Optics and Precision Engineering. kolibri
ROBOT - Optical 3D measurement system handled by robot. Website, February
2009. http://www.iof.fhg.de/departments/optical-systems/_media/
KOLIBRI_ROBOT_05_2007_E.pdf.
[6] A. Knapitsch. Adaptive structured light in a multi camera and multi projector setup. Master’s
thesis, Institute for Computer Graphics and Vision, Graz University of Technology, October
2008.
[7] H.S. Kwok and H.C. Huang. Liquid crystal on silicon microdisplays. In Solid-State and In-
tegrated Circuits Technology, 2004. Proceedings. 7th International Conference on, volume 3,
pages 1987–1990 vol.3, 2004.
[8] Orin Packer, Lisa C. Diller, Jan Verweij, Barry B. Lee, Joel Pokorny, David R. Williams, Den-
nis M. Dacey, and David H. Brainard. Characterization and use of a digital light projector for
vision research. Vision Research, 41:427–439, 2000.
[9] Bernhard E. Riecke, Hans-Guenther Nusseck, and Joerg Schulte-Pelkum. Selected technical
and perceptual aspects of virtual reality displays. Technical report, Max Planck Institute for
Biological Cybernetics, October 2006.
[10] Joaquim Salvi, Jordi Pages, and Joan Batlle. Pattern codification strategies in structured light
systems. Pattern Recognition, 37:827–849, 2004.
[11] ViALUX GmbH. z-Snapper compact. Website, February 2009. http://www.vialux.de/
z-snapper.pdf.