RTI HLT Processing Guide v14 Beta

Download as pdf or txt
Download as pdf or txt
You are on page 1of 23

REFLECTANCE TRANSFORMATION IMAGING:

GUIDE TO

HIGHLIGHT IMAGE PROCESSING


Document version 1.4 for RTIBuilder v. 2.0.2beta
Find updates and related materials at http://CulturalHeritageImaging.org/Learn/

Reflectance Transformation Imaging: Guide to Highlight Image Processing v1.4


2011 Cultural Heritage Imaging. All rights reserved.
This work is licensed under the Creative Commons Attribution-Noncommercial-No Derivative Works 3.0
United States License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/us/
or send a letter to Creative Commons, 171 Second Street, Suite 300, San Francisco California 94105 USA.
Attribution should be made to Cultural Heritage Imaging, http://CulturalHeritageImaging.org.

Acknowledgments
The RTIBuilder software described in this guide is copyright Universidade do Minho and Cultural Heritage
Imaging 2007 2011 and is made available under the Gnu General Public License version 3.
If you wish to receive a copy of the source code, please send e-mail to [email protected]
The Highlight RTI methodology described here was developed at Cultural Heritage Imaging by Mark
Mudge, Marlin Lum and Carla Schroer with technical guidance from Tom Malzbender of Hewlett Packard
Labs. The idea for Highlight RTI and its original execution came from a collaboration between Tom
Malzbender, Mark Mudge, Marlin Lum and Carla Schroer. Many others commented, tweaked, gave ideas
and were generally helpful in the development of the capture methodology (described in detail in the
guide Reflectance Transformation Imaging: Guide To Highlight Image Capture) It is impossible to name
everyone who contributed, but we would especially like to acknowledge our Portuguese collaborators at
the University of Minho, Alberto Proena, Luis Paulo Santos, and Joo Barbosa, as well as Professor James
Davis at UC Santa Cruz.
The RTIBuilder tool, designed for cultural heritage and natural science applications, enables users to
manage all aspects of building Reflectance Transformation Images (RTIs). A team at the Universidade do
Minho in Braga, Portugal were the primary developers of the RTIBuilder software. The work was funded by
Cultural Heritage Imaging (http://culturalheritageimaging.org) and the Universidade do Minho. Cultural
Heritage Imaging also provided software requirements, design advice, testing, general feedback and
documentation.
The 2.0.2 release of RTIBuilder includes the hemispherical Harmonics (HSH) fitter. The HSHfitter was
developed at the University of California, Santa Cruz, Computer Science Department in collaboration with
Cultural Heritage Imaging, which provided funding for this work from the US Institute of Museum and
Library Services (IMLS) National Leadership Grant Program (Award Number LG-25-06-010706). Tom
Malzbender of Hewlett Packard Labs also collaborated.
HSHfitter is copyright University of California, Santa Cruz and Cultural Heritage Imaging 2007 2009 and is
made available under the Gnu General Public License version 3.
The primary authors of this User Guide are Carla Schroer and Judy Bogart. Many others reviewed and
provided feedback. Partial funding for the development of the guide was provided by a 21st Century
Museum Professionals grant from the Institute of Museum and Library Services (award number:
MP-00-10-0031-10). Its contents are solely the responsibility of the authors and do not necessarily
represent the official position or policies of the Institute of Museum and Library Services. In addition much
time was donated for the development of this guide, including by various users who provided testing and
feedback on the contents.

Processing Captured Data


This document describes how to process image data captured using the highlight capture method, as
described in the companion document, RTI: Guide to Highlight Image Capture.
When you have finished acquiring the original images from a capture sequence, you must perform a set of
post-processing operations to create the final RTI image file.
X

You will review the captured images for usability, add identifying metadata to each one, and convert it
from camera-raw to DNG format for archiving, and then to JPEG format for processing.

You will process the JPEG image set to produce an RTI file. Use the RTI Builder application to perform
these operations:
Z

Compute the ancillary data on the exact light positioning for each image, which is computed from
the highlights on the reflective spheres.

When this is done, crop out the reflective spheres from the images.

Generate the final PTM or RTI file.

Processing raw files


The image-capture process results in a sequence of image files in camera-raw format. We recommend that
you convert these images to DNG (Digital Negative format), and apply any necessary exposure or
white-balance adjustments to the entire image set before converting them to JPEG format, which you will
pass to the RTI Builder.

File and folder conventions


The RTI Builder software cannot process folder names or file names with spaces. In addition to adopting a
naming convention without spaces, you must not store files in a folder whose name contains spaces. In
Windows, for example, you cannot use the My Pictures\ folder, which is a frequent default for image
files; similarly, the Desktop\ folder is actually in Documents and Settings\, which contains spaces.
The captured camera-raw files should reside on the local hard drive of the computer you will use for
processing. A specific folder structure is expected.
X

Create a top-level project folder, typically naming it with the object name and catalog number.

The project folder must contain these subfolders:


projectFolder/
jpeg-exports/
original-captures/

The JPEG files that you create from the captured images must be placed in the subfolder named
jpeg-exports. For example:
/EtruscanCup_123-456/jpeg-exports/*.jpg

Processing Captured Data

Processing raw files

Basic procedure
This is the general procedure for converting your camera-raw files with Adobe Camera Raw, which is
available in Adobe Bridge CS3 or later, and also in Photoshop and Lightroom. You can use any tool you
wish for applying image adjustments as needed. See Image adjustment below.
NOTE: It is not required that you use Adobe Camera Raw to produce your archive files and the needed JPEG
files. It is our preferred workflow, and is described in detail here. Whatever workflow you use must produce
archivable capture files (we highly recommend DNG format for these), to which you can apply any image
adjustment identically to all images in the set. In addition, you must be able to produce a set of JPEG
images to use with RTI Builder.
1. Transfer the original camera-raw files, if necessary, to your workstation. Place them in your chosen
working folder, and consider backing them up.
2. Open the files in the processing application (such as Adobe Bridge CS3).
Z

Delete any images that are unusable, such as any where the flash did not go off, or where part of
the object is obscured or shadowed by a hand, arm, or the flash attachment.

Stamp images with identifying metadata and keywords, such as the object name, collection
name, catalog number, and so on, using conventions established for you organization.

3. Save the adjusted images as DNG.


Z

If your institutional policy allows it, select the "Embed Original Raw File" option.

Adjust the file names and numbering according to your organizations naming conventions.

The DNG image set is now your primary capture data.


4. Export the files in JPEG format to a folder named jpeg-exports/ within your project folder, for use
with the RTI Builder application.
Z

Use the maximum size you wish to process.

To change the size, click the listing of the file size in the center bottom of the ACR window.

Do not change the file name. You should be able to trace back the connection from the JPEG to
the DNG image from which it was created.

Image adjustment
Image adjustments may or may not necessary. You can apply adjustments using the Adobe Camera Raw
tool; this is not a standalone product, but is accessible from both Adobe Photoshop and Adobe Photoshop
Lightroom.
If you do apply them, you must apply exactly the same adjustments to every image in the sequence. You
can do this in Adobe Camera Raw by adjusting one image, then click Select All and Synchronize in the
upper left corner.
X

Do not make any image adjustments other than the recommended ones.

Do not crop out the reflective spheres at this stage -- they are still needed.

Processing Captured Data

Processing raw files

When all adjustments are complete, save the files as DNG and JPEG; see Saving adjusted image files on
page 7.
When you have done so, you must click Done in the lower right corner of the Adobe Camera Raw main
window to make sure that all of your image adjustments are recorded before you exit the application.
Create a
preset

We recommend that you create a "Zeroed Out Settings" custom preset, and apply it to the entire image
set. Consider making this your default preset for Adobe Camera Raw. To do this, set all of the settings to 0,
then save as a named preset using the flyout menu for the Basic image adjustments panel. In particular,
make sure all sharpening options are set to 0.
Check the settings in all of the tabs. The first three tabs (Basic, Tone Curve, and Detail) have default settings
that are non-zero; the radius setting on the Detail tab cannot be less than 0.5. In other tabs, default
settings are already zero.

Processing Captured Data

White
balance

Exposure

Processing raw files

If necessary, perform a white-balance adjustment based on the gray card or color card.
X

Depending on your capture setup, the gray card or color card might be in the capture images or in a
separate shot.

Apply the same adjustment to the entire image set.

Adjust the exposure if necessary.


X

Check the histograms for the lightest and the darkest images, and make sure that no whites are blown
out, and no shadows are too dark. (The lightest image is one taken at the highest light angle, and the
darkest one is one taken at a low incident angle.)

Apply the same adjustment to the entire image set.

Processing Captured Data

Processing raw files

Saving adjusted image files


When you have applied all changes, you must save the adjusted images in DNG format, and then again in
JPEG format.
Check the size of the images before saving, to make sure you aren't accidentally down sizing everything, or
interpolating up. If you are using the Adobe Camera Raw tool to apply adjustments, the size of an image
you are currently viewing appears below it, at the bottom center of the screen. You can click the link to
show details.

In the sizes pull-down options, a minus sign (-) marks a down-res option, and a plus sign (+) marks an
interpolated option. Choose the actual file size, which is not marked by either plus or minus.
When you have checked that the images are shown at actual size, select all images and click Save Images.

Processing Captured Data

Processing raw files

This brings up the Save Options dialog, in which you choose where to save the files and how to name
them.

Processing Captured Data

Processing raw files

We recommend saving to the same location, which should be the projects original-captures
folder. When you have finished creating the DNGs, you can delete the original raw files.

Select the file name and numbering style according to your organizations naming conventions.
Typically, you combine a catalog number or object name with a sequence number (to distinguish
images in the RTI sequence). Do not use spaces in the filenames; if your original filenames have spaces,
use this opportunity to remove them.

For Format, choose Digital Negative. If your institutional archiving policy allows it, we recommend that
you save DNG with embedded RAW. To do this, select the "Embed Original Raw File" option.
This doubles the image size; however, if you do not embed the raw data, the data you archive is the
result of conversion to 16-bit TIFF (the basis of the DNG format).

Click Save to save the DNG files with the chosen options. When the save-as-DNG operation is complete,
make sure all image are still selected and save again. This time, create a set of JPEG images in the
jpeg-exports folder.
X

Choose the projects jpeg-exports folder as the location.

Do not rename the files. Select theDocument Name option, so that the JPEG files match the DNG
files. For the file extension, choose .jpg.

For Format, choose JPEG. For Quality, choose Maximum.

Click Save to save the JPEG files with the chosen options. When the operation is complete, click Done in
the Adobe Camera Raw main window, to make sure all of your image adjustments are saved.
You are now ready to use RTI Builder to process the image sequence into an RTI file.

Processing Captured Data

Using RTI Builder

10

Using RTI Builder


The RTI Builder is an interface to a set of tools that process your image set to produce the final RTI file. It is
written in the Java programming language, and requires that you have Java version 6 or higher installed
on your computer.

Installing RTI Builder


Download the installer file for your platform:
X

For Windows, rtibuilder_2_0_2_beta_setup.exe


Double-click the setup file to run the installer, and follow instructions.

For Intel Macs running Mac OS 10.5 or later: RTIbuilder_2_0_2_beta.dmg


Double-click the image file. In the resulting window, drag the RTI Builder APP file to the Applications
folder.

Download the PTM Fitter plug-in


In this version of RTI Builder, you can choose to create your RTI file using either the Polynomial Texture Map
(PTM) fitting algorithm (ptmfitter), or the Hemispherical Harmonics (HSH) fitting algorithm (hshfitter).
The hshfitter plug-in is included in the RTI Builder package, the ptmfitter plug-in is not; license
restrictions make it incompatible with the Gnu General Public License version 3 that is used with RTI
Builder and the HSH fitter. You can use it, but you must download it separately, and browse to its download
location on your hard drive the first time you want to use it.
If you plan to use the PTM fitting algorithm, download the ptmfitter plug-in from HP Labs:
http://www.hpl.hp.com/research/ptm/downloads/agreement.html

Processing Captured Data

Using RTI Builder

11

Usage overview
This section provides a brief overview of how to use RTI Builder to process your image set into an RTI file.
Each step is described in more detail in following sections.
The basic steps are:
1. Create a project
When you invoke the RTI Builder, you create a named project and point to the location of your image
set.
Choose the processing sequence algorithm for the type of RTI file you want to create; it can be
produced with the Polynomial Texture Map (PTM) algorithm, or with the Hemispherical Harmonics
(HSH) algorithm. Both types of RTI files can be viewed with the RTIViewer.
2. Find light positions
To do this, you must first identify the spheres in one of the images, then run the tool that detects the
highlights and uses them to calculate the light positions and generate a light-position file.
3. Crop the images
The RTI should show only the target object; there is a tool to help you crop the image so that the
reflective spheres are not included in the final RTI. The cropping tool automatically applies the same
crop to the entire image set.
4. Generate the RTI
The final tool processes the complete image set to produce the final PTM or RTI file.
5. Reprocess an existing data set
After producing a PTM or RTI file, you can reload the existing project to create additional PTM or RTI
files with different settings.

Processing Captured Data

Using RTI Builder

12

Create a project
To start the RTI Builder:
X

In Windows, double-click the executable Java archive, RTIbuilder-202-beta.jar

In Mac OS, click the application file, RTIbuilder-202-beta.app

In the first screen, you can either create a new project, or open an existing project to create a different PTM
or RTI file from the same image data.

To create a new project:


1. Enter a name for your project.
2. Select one of the Highlight Based operation sequences, using the HSH Fitter or PTM Fitter algorithm.
NOTE: If you choose to use PTM Fitter, you must download the plug-in from HPLabs. See Installing RTI
Builder on page 10.
3. Click Start New Project. (For information on opening an existing project, see Reprocess an existing
data set on page 23.)
4. In the next screen (see following figure), click Open Folder at the right of the window, and browse to
the top-level project folder. This is the parent of the jpeg-exports/ subfolder that contains the
exported JPEG image files.
The import takes some time, depending on the size and number of your images, and the speed of your
computer; it can take several minutes. When it is finished, the entire image set is displayed, and the
currently selected image in the set is shown in the upper left panel.

Processing Captured Data

Using RTI Builder

13

5. You can use the Project Properties section to change existing metadata for the set, or to add or remove
properties if you wish. Each property associates a field name with a value. These properties are stored
in the project log file; they are not included in the metadata for each image.
Z

Simply type in new values for existing metadata fields.

Use the Add button to create new properties.

Select a property field name and click Remove to delete it from the metadata.

When you have finished editing the metadata, click Save to apply the change to the project log
file.

6. Use the removal section to remove any images that should not be used to calculate the RTI. These
images are removed only from the set of images to be processed; the image files are not deleted.

Processing Captured Data

Using RTI Builder

Select the image or images you want to remove.

Type in the reason for the removal.

Click Remove Picture to apply the change to the image set.

14

7. When you have finished, click Next to begin the process of finding the light positions from the
highlights in the reflective spheres.

Find light positions


There are two steps to finding the light positions, identifying spheres and detecting highlights.
You must identify the area around a reflective sphere in one of the images, so that the software can detect
the highlights and use them to compute the light position for each image. This is an interactive process.
1. In the larger image at the upper left, use the mouse to select a rectangular region that includes an
entire sphere. You will adjust the size and placement to be more exact in a later step.
The region shows up in light green. Click and hold the mouse button down to drag out region,
releasing the button when you have the region you want.
2. When the green region is approximately where you want it, click Add area.
3. The area you have defined turns red. Once an area has turned red, you can click and drag the corner
handles to resize it, or click in the middle of the region and drag to move it.

Processing Captured Data

Using RTI Builder

15

Ensure that the highlighted area is larger than the sphere, and that the sphere edge is completely
included within the box.

Resize handles in corners of defined area

New area in green

NOTE: Currently, the program that generates the light positions from the highlights uses only a single
sphere. It is possible to detect the highlights on additional spheres; future versions of the software will
support multiple spheres, in order to take advantage of new research software that requires a
minimum of two spheres in the image. For this reason, we recommend placing two spheres in the
scene at the time of image capture.
4. When the area fits fairly closely around and completely encompasses the sphere, click Detect
spheres.
The processing takes some time. When it is completed, the upper left panel shows the detected
sphere.

Processing Captured Data

Using RTI Builder

16

1. Use the Image Scale to fit the image in the panel.


2. You might want to resize or move the red circle designating the sphere so that it fits the edge as
closely as possible. You can use the Ball Center and Radius controls, or click and drag the red handles
at the right edge and center of the image.

Move and resize handles

3. If you move or resize the sphere, click Set New Center to apply the change.
4. To see the actual sphere image with the detection outline superimposed on it, scroll to the last image
in the thumbnails view. Select the one where the sphere appears black, and check the fit there. If

Processing Captured Data

Using RTI Builder

17

necessary, use the adjustments to get the red outline as close as possible to the actual outline of the
sphere. You can move the sphere by dragging, or use the handles to change the size and position of
the outline. If you make any changes, click Set New Center to apply the changes.

When you have completely identified a sphere, the software can detect the highlights and compute
the light positions.
When you have finished, click Next.
5. Click Highlight detection.
This is a lengthy operation. When it has completed, the red circles around the sphere images change
to small, red + marks at the center of each highlight.

Processing Captured Data

Using RTI Builder

18

6. If the highlight has not been detected properly, click Redo Process to retry the highlight detection
process.
The tool generates the following files:
X

A blended image showing all the highlights in the set, in the assembly_files/ subfolder. You can
scroll to the bottom of the thumbnail images in the RTI Builder to see this image.

A light position (LP) file with the extension .lp, which is used to generate the RTI. This is written into
the assembly_files/ subfolder that contains the JPEG image files.

An XML log file named <projectName>.xml, written to the top-level project folder.

Processing Captured Data

Using RTI Builder

19

Crop the images


After the light positions have been computed, you can choose to crop all of the images to show only the
target object (not the reflective spheres or extraneous background). You specify the crop area in a single
image, and the software automatically performs the same cropping operation on the entire image set.
The software passes the cropped images to the fitter algorithm, along with the light-position file, in order
to generate the final PTM or RTI file.
1. Click Next to show the cropping screen. (The information on the right reflects your choice of the PTM
or HSH fitter algorithm, but the cropping process is the same.)

2. In the left bottom pane, select Use Crop.


3. You can use rectangular or free-hand crop areas to crop down the image to the portion you want to
include in your finished RTI.
Z

If you select Rectangular, use the mouse to draw out the crop region, which appears in green. You
can adjust the region using the resize handles in the corners.

If you select Free, you can create a closed polygon to define the crop area. Click on the starting
point of the shape. Each mouse click draws a dotted-line segment in the image. The final click
should be on the starting point.

4. Adjust the crop area as closely as possible around the target image. To start over, click Clear Crop.

Processing Captured Data

Using RTI Builder

20

Generate the RTI


The right side of the screen allows you to generate the final RTI, using either the PTM or HSH fitting
algorithm.

PTM options
If you are using the PTM algorithm, the first time you run RTI Builder, you must enter the path to the
ptmfitter file; that is, the location where it is installed on the local computer. (This tool is downloaded
separately from HP Labs; see Installing RTI Builder on page 10.) The software remembers the location,
and you do not need to set it again.
1. You can choose to resize the images to make a smaller PTM. If you select Resize, enter a New Width or
New Height value (see following figure). The aspect ratio is automatically maintained.

Processing Captured Data

Using RTI Builder

21

2. Choose LRGB or RGB as the PTM Type. LRGB create a smaller file size, but the color is not as accurate.
This is particularly noticeable for shiny materials like silver or gold.
3. Choose a name for the PTM file, if you wish. The default is the project name, followed by _nnnn (the
number of pixels in the horizontal direction of the images).
4. Click Execute.
If you chose to resize the image, the tool generates the smaller images before calling the PTMfitter tool.
The box at the upper right of the screen displays the output of the fitter tool; when the operation is
complete, a notification dialog appears.
The generated PTM file is written to the finished_files/ subfolder of the project folder.
You can generate several PTMs with different image cropping, size, and color type (LRGB/RGB) options.
After creating a PTM, simply change the parameters and generate another. If you do this, however, be
careful to use unique names to distinguish the results. RTI Builder does not check for an existing file with
the same name. If a file with the same name exists, it is overwritten without warning.HSH options

Processing Captured Data

Using RTI Builder

22

HSH options
1. You can choose to resize the images to make a smaller RTI. If you select Resize, enter a New Width or
New Height value (the aspect ratio is automatically maintained).

2. The Order determines how many variables are used to store the mathematical description of the
surface normal. Order 1 uses 4 variables, order 2 uses 9 variables, and order 3 uses 16 variables.
When you use more variables, there is more high-frequency data per pixel. The higher the order, the
more information is included per pixel, and the closer the approximation of the virtual illumination in
the Viewer is to the actual object. The higher order, however, also increases the file size.
3. Fill in the other options to control how the HSH fitting algorithm is applied.
4. Click Execute to generate the RTI file.
The generated RTI file is written to the finished_files/ subfolder of the project folder.
You can generate several RTIs with different options. After creating an RTI, simply change the parameters
and generate another. If you do this, however, be careful to use unique names to distinguish the results.
RTI Builder does not check for an existing file with the same name. If a file with the same name exists, it is
overwritten without warning.

Processing Captured Data

Using RTI Builder

23

Reprocess an existing data set


If you want to go back and work on a data set where you have already generated an RTI Builder project:
1.

Start RTI Builder.

2. Select one of the Highlight Based operation sequences, using the HSH or PTM fitting algorithm. If you
are switching from one of these methods to the other, you must make this selection before opening
the existing project file.
NOTE: If you choose to use PTM Fitter, you must download the plug-in from HPLabs. See Installing RTI
Builder on page 10.
3. Click Open Existing Project, browse to the existing project, and select the XML project file. This is at
the top level of the project folder, next to the jpeg_exports/ subfolder.
NOTE: Currently, if you browse to a folder other than a project file, or select a file other than a valid project
XML file, the system does not detect and report the error, but appears to do nothing.
4. Your choice of operation sequence, or fitting algorithm, determines the processing pipeline that is used
to process the image data and produce a PTM or RTI file. If, when you choose to reprocess an existing
data set, you choose to do so using a different algorithm, you see this dialog:

Confirm your choice by clicking Yes.


5. RTI Builder shows the final screen, in which you crop the image and choose generation options. Even if
you are using the same fitting algorithm, you can make different choices by cropping differently,
resizing, or selecting different options.
6. When you have made all of your generation choices, provide a unique name for the finished file.
Make sure this is a unique name; RTI Builder does not check for an existing file with the same name. If a
file with the same name exists, it is overwritten without warning.

You might also like