A Robotic Excavator for Autonomous Truck Loading

ANTHONY STENTZ, JOHN BARES, SANJIV SINGH AND PATRICK ROWE

The Robotics Institute, Carnegie Mellon University, Pittsburgh, PA 15213

Abstract. Excavators are used for the rapid removal of soil and other materials in mines, quarries, and construction sites.
The automation of these machines offers promise for increasing productivity and improving safety. To date, most research
in this area has focussed on selected parts of the problem. In this paper, we present a system that completely automates the
truck loading task. The excavator uses two scanning laser rangefinders to recognize and localize the truck, measure the soil
face, and detect obstacles. The excavators software decides where to dig in the soil, where to dump in the truck, and how to
quickly move between these points while detecting and stopping for obstacles. The system was fully implemented and was
demonstrated to load trucks as fast as human operators.

Keywords: autonomous excavation, robotic excavator, integrated robotic system, laser rangefinder, software architecture,
manipulator, dig planning

1. Introduction pass by even a second translates into an enormous gain

across the entire job. The operators performance peaks
early in the work shift and degrades as the shift wears on.
The surface mining of metals, quarrying of rock, and con-
Scheduled idle times, such as lunch and other breaks, also
struction of highways require the rapid removal and han-
diminish average production across a shift. All of these fac-
dling of massive quantities of soil, ore, and rock. Typically,
tors are areas where automation can improve productivity.
explosive or mechanical techniques are used to pulverize
the material, and digging machines such as excavators load Safety is another opportunity. Excavator operators are
the material into trucks for haulage to landfills, storage ar- most likely to be injured when mounting and dismounting
eas, or processing plants. As shown in Figure 1, an excava- the machine. Operators tend to focus on the task at hand
tor sits atop a bench and loads material into trucks that and may fail to notice other site personnel or equipment en-
queue up to its side. The operator is responsible for desig- tering the loading zone. Automation can improve safety by
nating where the truck should park, digging material from removing the operator from the machine and by providing
the face and depositing it in the truck bed, and stopping for complete sensor coverage to watch for potential hazards
people and obstacles in the loading zone. entering the work area.
Numerous researchers have addressed aspects of auto-
mated earthmoving (Singh, 1997). The lowest and most
common level of automation has been teleoperation. Typi-
cally, the operator is removed from the scene for reasons of
safety. Teleoperated excavators are used in applications
that pose a danger to humans, such as the uncovering of
buried ordnance (Nease and Alexander, 1993) and waste
(Burks et al., 1992; Wohlford et al., 1990), or excavation
around buried utilities. A higher level of autonomy is
achieved by systems that share control of the excavation
cycle with a human operator. Typically, these systems
(Bradley et al., 1993; Bullock and Oppenheim, 1989; Hua-
Figure 1. Excavator loading a truck with soil in a typical mass
ng and Bernold, 1994; Lever et al., 1994; Rocke, 1994; Sa-
excavation work scenario. kai and Cho, 1988; Salcudean et al., 1997; Sameshima and
Tozawa, 1992; Seward et al., 1992) concentrate on the pro-
The opportunities for automation are immense. Typical- cess of digging. An operator chooses the starting location
ly, loading a truck requires several passes, each of which for the excavators bucket and a control system takes over
takes 15 to 20 seconds. Reducing the time of each loading the process of filling the bucket using force and/or joint po-
sition feedback to accomplish the task. At the next level of
autonomy are systems that automatically select where to
dig. Such systems measure the topology of the terrain using
Left scanner
ranging sensors (Feng et al., 1992; Singh, 1995; Takahashi Right
et al., 1995) and compute dig trajectories that maximize ex- scanner
cavated volume. At the highest level of autonomy are sys-
tems that sequence digging operations over a long period
(Bullock et al., 1990; Romero-Lois et al., 1989; Singh,
1998).
The prior work addresses many subproblems important
for autonomous truck loading, however in order to field a
fully automated system that performs at the level of its
manually operated equivalent, a much broader set of prob-
lems must be solved than just digging. Sensors are needed
to sense the dig face, recognize and localize the truck, and Figure 2. Sensors mounted on excavator.
detect obstacles in the workspace. Perception algorithms
are needed to process the sensor data and provide informa-
tion about the work environment to the planning algo- Scan axis Distance
measuring
rithms. Planning and control algorithms are needed to sensor
decide how to work the dig face, deposit material in the
truck, and move the bucket between the two. Rotating Pan axis
reflector
We have developed a complete system for loading trucks
fully autonomously with soft materials such as soil. The
Autonomous Loading System (ALS) was implemented and Scan plane Vehicle cab
demonstrated on a 25-ton hydraulic excavator and succeed-
ed in loading trucks as fast as an expert human operator.
The rest of the paper describes the ALS and presents results Obstacle or terrain
from experimental trials. Figure 3. Two axis scanning sensor configuration.

The excavator uses its scanners in the following fashion

2. System Overview
when loading a truck (Figure 4). While the excavator digs
its first bucket, the left scanner pans left from the dig face
across the truck both to detect obstacles and to recognize,
The Autonomous Loading System uses two scanning laser
localize, and measure the dimensions of the truck. Using
rangefinders that are mounted on either side of the boom
this information, a desired location in the truck to dump the
(see Figure 2) to sense the dig face, truck, and obstacles in material is planned, and the bucket swings toward the
the workspace. Two scanners are needed for full coverage truck. During this swing motion, the right scanner pans left
of the workspace and to enable concurrent sensing opera- across the dig face to measure its new surface, and the next
tions. Each sensor has a sample rate of 12 kHz, and a mo- location to dig is calculated. The right scanner continues to
torized mirror sweeps the beam circularly in a vertical pan toward the truck. After the soil is dumped into the
plane. Additionally, each scanner can pan at a rate of up to truck, the right scanner pans back across the dig face to de-
120 degrees per second, enabling this circle to be rotated tect obstacles in the way of the implements. The excavator
about the azimuth, as shown in Figure 3. The scanner posi- swings back to the next dig point. During this swing, the
left scanner pans across the truck to measure the soil distri-
tioned over the operators cab is called the left scanner,
bution in the truck bed, and the next desired dump location
and it is responsible for sensing the workspace on the left is calculated. This process repeats for each subsequent
hand side of the excavator. The right scanner, which is loading pass until the truck is full, with the exception that
located at a symmetric position on the right side of the truck recognition is only necessary for the first pass for
boom, is responsible for sensing the workspace on the right each new truck. Typically, six passes are needed to load our
hand side of the excavator. twenty-ton truck with our excavator testbed.
 Left sensor left position right
scan plane sensor system sensor

sensor
commands
left sensor right sensor
Truck interface interface

left scanline position data

right scanline
Left scanner Right processor sensor data processor
scanner to obst. det.
sensor data sensor data sensor data sensor
data
truck dump point dig point
recognizer planner planner
Right sensor
scan plane truck info. dump pt. dig pt.
Dig face sensor dumping digging obstacle
motion motion motion
planner planner detector
Figure 4. Top view of sensor configuration. sensor planner
states
states commands
Information from the scanners is processed using an on- machine
controller excavator
board array of four MIPs processors. The software archi- interface
tecture is shown in Figure 5. The boxes are software mod-
Figure 5. ALS software architecture.
ules that can run on one of the system processors. Circles
are hardware components such as sensors. Lines represent 3. Hardware Subsystem
communication channels. The sensor interfaces receive
data from the two scanners and control the panning motion
The ALS hardware subsystem consists of the servo-con-
of the devices. Sensor data from the interfaces are passed to trolled excavator, on-board computing system, perception
scanline processors, where they are converted from spher- sensors, and associated electronics. In this paper we focus
ical, sensor coordinates to Cartesian, world coordinates us- on the perceptual sensors which provide the data from
ing corresponding data from the position system. These which the truck is identified, the dig location is chosen, ob-
three-dimensional range points are then made available to stacles are avoided, and ultimately the mass excavation
whatever perception software modules require them. process is achieved.
With the target application of earthmoving, we focussed
One consumer of this processed sensor data is the truck
on developing a laser based scanning system that would be
recognizer, which recognizes the truck and measures both able to penetrate a reasonable amount of dust and smoke in
its dimensions and location. Two others are the dig point the air. The laser itself would need to be able to accurately
planner, which plans a sequence of dig points for eroding measure range from a variety of target materials (e.g., met-
the dig face, and the dump point planner, which plans a se- als, wood, dirt, rock, snow, ice, and water), colors and tex-
quence of dump points for loading soil into the truck bed. tures. We also needed a system that would be robust to dust
The digging motion planner controls the excavator during and dirt accumulating on the protective exit window
digging at the specified location. The dumping motion (glass or plastic which protects the laser and optics from
planner dumps the bucket of soil into the truck and returns weather and dirt, though permits the beam to pass).
to the dig face. The sensor motion planner controls the pan- Over the past decade, a variety of laser based scanners
ning for both scanners to coordinate scanner and excavator have been produced. With the exceptions of the Dornier
(Shulz, 1997) and Schwartz (Schwartz) scanners, most
motion, following the scenario described above. The obsta-
have either been research devices or limited to indoor us-
cle detector processes sensor data from the scanner that is
age. None that we know of addresses the problems of dust
sweeping in advance of excavators motion and stops the penetration or a partially occluded (i.e., dirty) exit window.
machine if an obstacle is detected in its path. The machine We have developed two different time-of-flight scanning
controller interface communicates commands to the low ladar systems that are impervious to ambient dust condi-
level machine joint controller, which executes the com- tions. The first uses a last-pulse technique that observes
mands and sends excavator state information back to the the waveform of the returned light and rejects early returns
planning modules. that can arise from internal reflections off of a dirty exit
window, or from a dust cloud obscuring the target (see Fig- al for construction and mining environments. Though the
ure 6). In general, the next-to-last pulse returns are due to trailing edge detection technique forgoes some range accu-
dust in the scene and are indicative of what a normal first racy, we believe it is a superior approach for environments
pulse rangefinder would see. For instance, in Figure 6, a where the dust may frequently surround the target.
first pulse rangefinder would detect the dirty exit window
and would be unable to see the target. Even if the win-
dow were clean, the first pulse unit would still see the
dust cloud instead of the target. Since reflections off the
exit window are rejected with the last pulse technique, the
unit can be environmentally sealed using an inexpensive
transparent cover that does not have to be optically perfect
or clean. Another advantage is that the laser system can
also report when multiple returns occur, giving a warning
that dust is present. This is important because overall rang-
ing reliability and accuracy is decreased in dusty condi-
tions, so an autonomous machine might need to adopt a
slower, more conservative motion strategy.
Dust cloud Target
Dirty exit window

Figure 8. Last pulse vs. trailing edge detection when target is

Laser within dust cloud.
source
The television monitor pictured in Figure 8 shows range
points plotted from a single scanline for both the last pulse
Return Threshold
signal and trailing edge scanners. Range increases from the left to
the right of the monitor. The top monitor screen shows
scans of the rear of a dump truck. The bottom screen shows
Target range (using last pulse) scans of the same truck but shrouded in a heavy dust cloud.
Figure 6. Last pulse detection concept. Note that the last pulse device is unable to separate the dust
cloud from the truck and reports the front of the cloud. The
trailing edge device correctly reports range to the truck re-
Dust cloud Target gardless of the presence of dust.
Dirty exit window
It is important to note that both dust penetrating tech-
niques are physically limited by very heavy dust levels that
attenuate the return target signal below the point of detect-
Laser
source
ability.

4. Software Subsystem
Return Threshold
signal
The software subsystem consists of several software mod-
ules that process sensor data, recognize the truck, select
Target range (w/last pulse) (w/trailing edge) digging and dumping locations, move the excavators
joints, and guard against collision. In this section, the algo-
Figure 7. Trailing edge detection of target when target is rithms employed by key software modules in the software
obscured in dust cloud. architecture are described.
There is, however, a limitation to last-pulse rangefind-
ing. When the target is within the dust cloud, the receiver 4.1. Truck Recognition
electronics can have difficulty separating the dust and tar-
get returns (see Figure 7). We have built a second dust pen- In order to properly load a truck, an excavator operator
etrating scanner system that identifies that target by must verify that it is a loadable vehicle, determine its loca-
locating the trailing edge of the last return signal as is tion, and determine its dimensions. This information is es-
shown in Figure 7. Like the last pulse system, this device is sential for calculating a loading strategy and for planning
also robust to occlusions on the exit window making it ide- the sequence of joint motions that implements this strategy.
In some scenarios, such as surface mining, the loaders are matches. At each level in the tree, constraints are used to
serviced by a mine-owned fleet of haulage trucks. An auto- prune the search and to check for consistency with previ-
mated system could acquire this information by equipping ously hypothesized matches. The interpretation that match-
each truck with a global positioning system (GPS) sensor es most of the model regions and survives the verification
and an identification transponder. However, in other sce- stage is selected as the correct one. In order for the truck
narios such as highway construction, the loaders are ser- recognizer to recognize a class of truck models rather than
viced by a variety of independently-owned, on-highway just a single model, the model in Figure 10 uses parameter
trucks of varying dimensions, so equipping each and every ranges rather than single parameter values. Ranges are used
truck with such sensors could be infeasible. For such sce- on the sizes of the planar regions in the model, the locations
narios, an automated system could acquire the necessary of their centroids relative to each other, and the angles be-
information using rangefinder data. tween the planes. These parameter ranges are checked for
consistency at every level in the interpretation process to
prune the search. This specification allows the truck recog-
nizer to identify trucks of varying sizes and truck bed
shapes.
For each complete interpretation (i.e. an attempt to match
all model regions to scene regions), the truck recognizer
performs a verification. The verification consists of finer-
grained consistency checking of truck parameters, and the
identification of the four corner points in the sensor data
that define the opening of the truck bed. For the selected in-
terpretation, the corner points are used to calculate the po-
sition and orientation of the truck bed. This information is
Figure 9. Raw range data of a truck. passed to other modules in the system for producing a
dumping strategy. Figure 10 shows the model matched to
the planar regions segmented from the raw sensor data.

4.2. Coarse-to-Fine Dig Point Planning

Automated earthmoving operations such as leveling a

mound of soil are distinguished from typical planning
problems in two important ways. First, soil is diffuse and
therefore a unique description of the world requires a very
large number of variables. Second, the interaction between
the robot and the world is very complex and only approxi-
mate models that are also computationally tractable are
available. The large state space and complex robot-world
interaction imply that only locally optimal planners (i.e. per
Figure 10. Truck model fit to segmented data.
dig) can be created. In order to deal with the practical issues
The truck recognizer uses sensor data to automatically of excavating large volumes of earth in applications, we
recognize, localize, and dimension haulage trucks. As the have developed a multi-resolution planning and execution
excavator digs its first bucket of soil, the left scanner pans scheme. At the highest level is a coarse planning scheme
across the truck, which is assumed to be parked to the ex- that uses the geometry of the site and the goal configuration
cavators side. The raw sensor data are shown in Figure 9. of the terrain to plan a sequence of dig-regions. In turn,
Each rotation of the mirror returns one vertical scanline of each dig region is searched for the best dig that can be ex-
data, created by intersecting a vertical plane with the truck. ecuted in that region. Finally, the selected dig is executed
Each scanline is processed into line segments which are by a force based closed loop control scheme (Rocke, 1994).
grouped with coplanar line segments from other scanlines Treatment of the problem at three levels meets different ob-
to form planar regions. jectives. The coarse planner ensures even performance over
Using an interpretation tree approach (Grimson, 1990), a large number of digs. The refined planner chooses digs
the simple model for a truck bed, shown in Figure 10, is that meet geometric constraints (reachability and colli-
matched to the segmented data region by region. Depth- sions) and which locally optimize a cost function (e.g. vol-
first search is used to hypothesize model-to-scene region ume, energy, time). At the lowest level is a control scheme
that is robust to errors in sensing the geometry of the ter- given a scanning range sensor that is mounted on the cab.
rain. Figure 11 shows the process of coarse to fine planning The refined planner operates on an abstract representa-
for the excavator. tion of an atomic action (i.e. a single dig). Rather than
searching for a bucket trajectory, the refined planner
searches through compact task parameters within the
Terrain Map bounds specified by the coarse planner. In order to select
the best digging action, the refined planner evaluates can-
didates through the use of a forward model that simulates
the result of choosing an action (in our case the starting lo-
Coarse Refined Closed Loop
Planner Planner Executor
cation of the bucket). An evaluation function scores the tra-
Dig Dig jectory resulting from each action, and the action that meets
Region Goal all constraints and optimizes the cost function is chosen.
This process is shown in Figure 13.

Candidate
Figure 11. Coarse to fine planning strategy. Digs

Forward Dig for each dig:

trajectory, Evaluation
Model swept volume,
work, time

Constraints Best Dig

(kinematics, (starting
coarse plan
2 1
bucket angle,
geometry) location)
3
5 4 6 Figure 13. Operation of refined planner.

8 7 4.3. Template Based Dump Planning

10 9
The truck must be loaded evenly and completely. Because
of uncertainty in soil settlement, the dumping strategy may
need to be revised for each successive bucket load. The
Dig Region dump point planner applies a template-based approach to
Figure 12. Coarse plan for an excavator.
robustly find the low regions of soil distribution in the truck
bed.
The coarse planner takes as input processed sensor data Sensor data are gathered after each bucket of soil has
which it places in a terrain map (a 2-D grid of height val- been dumped in the truck as the excavator is swinging back
ues). The output is a sequence of dig regions, each of which to the dig face. Like the dig point planner, the sensor data
is in turn sent to a refined planner. Figure 12 shows a strat- are placed in a 2-D terrain map. The dump point planner
egy for removing material that was recommended by an ex- also requires information about the location of the truck,
pert excavator operator. Each box indicates a region, and provided by the truck recognizer module, so it can filter out
the number within the box indicates the order in which the any irrelevant sensor data that are outside of the truck bed.
region is provided to the refined planner. In this strategy, The terrain map is then smoothed using a simple Gaussian
material is removed from left to right, and from the top of filter to eliminate any sensor noise. The current grid cell
the face to the bottom. There are several reasons for choos- resolution of the truck bed terrain map is 15 cm, with a typ-
ing this strategy. In most cases, the truck is parked on the ical map containing on the order of 500 cells.
operators left hand side so that the operator has an unob- Occlusion of the deposited soil by the truck bed walls is
structed view of it. By digging from left to right, the imple- a serious problem. Rather than assuming that nothing is in
ments do not need to be raised as high to clear material the unseen regions of the truck bed, the dump point planner
when swinging to the truck. In digging from top to bottom, fills in any unknown grid cells with the average elevation
less force is required from the implements because it is not of the known grid cells. This results in some slight inaccu-
necessary then to work against the weight of the material racies in the perceived soil distribution at first, but they di-
up above. In addition, clearing material away from the top minish as more soil is placed in the truck bed.
minimizes the range shadows cast on the face of the terrain Finally, a specific terrain shape template is convolved
over the entire truck bed terrain map to produce a score for The rules of script were designed with the input of an ex-
each grid cell. This small 5x5 or 7x7 grid cell template pert human excavator operator and implicitly constrain
looks for a certain profile of the material in the truck bed, what the excavator is and is not allowed to do. For example,
such as a slope or a hole. Simple templates of constant ele- if it was advised that moving two particular joints simulta-
vations can be used to find the lowest elevation in the truck neously was a bad idea, then the rules of the script make
bed terrain map as well. The convolution operator produces that motion impossible. The left hand side of the rules are
a score which represents how well the template matched functions of the excavators state, and the right hand side of
the particular region in the truck bed, and the location of the the rules are the commands which the planner sends to the
cell with the best score is returned as the desired dump lo- excavators low level joint controllers. Thus, when the left
cation. hand side of a particular rule evaluates to true, its corre-
sponding command gets sent to the excavator. The rules get
4.4. Script Based Motion Planning re-evaluated at a fixed rate, 10 Hz for example, during the
execution of the excavators motion.
The motion planning software coordinates the motions of Figure 14 shows the script rules for the truck loading
the excavators joints for each loading pass, beginning im- task. The numbers in boldface are one example set of script
mediately after digging a bucket of soil and ending when parameters, which will be described in more detail below.
the bucket has returned to the next dig point. The main ob- The s are the excavators state, in this case the angular
jectives of the motion planner are to plan motions which positions of the joints. The commands are desired angular
place the soil at the desired dump location, avoid all known joint positions. Notice that each joint has its own separate
obstacles in the workspace such as the truck, and execute script. Therefore, only one rule per joint may be active at a
each loading cycle as quickly as possible. time.
Joint 1: Swing Command The script parameters are computed before each loading
1) When digging finishes, wait 1 = 5
pass starts using the information about the trucks location
2) If 2 > 14, swing to truck 1 = 101
3) If 4 > 10, swing to dig 1 = 0
and the desired dig and dump points. There are two types
4) If 1 = 0, stop and execute dig of script parameters, those which appear in the left hand
side of the script rules and affect which commands are sent
Joint 2: Boom Command
1) When digging finishes, raise 2 = 18
by the planner, and the joint commands themselves which
2) If 1 < 60, lower to dig 2 = 6 appear on the right hand side of the rules.
The command script parameters in the right hand side of
Joint 3: Stick Command
1) When digging finishes, wait 3 = -100
the rules are primarily computed by geometric and kine-
2) If 1 > 31, move to spill point 3 = -76 matic means. For example, consider the command of 18
3) If 4 > -30, move to dump point 3 = -92 from step 1 of the boom joints script in Figure 14. That is
4) If 1 < 65, move to dig 3 = -75 the boom angle which is required for the excavators buck-
et to safely clear the top of the truck, and is a kinematic
Joint 4: Bucket Command
1) When digging finishes, curl 4 = -90
function of the height and location of the truck relative to
2) If 1 > 60 and 3 > -89, open 4 = 30 the excavator. Similarly, the stick joint commands are com-
3) If 1 < 60, move to dig 4 = 7 puted using knowledge about the radial distance of the
truck from the excavator, and the swing joints commands
Figure 14. Truck loading script for an excavator.
are found from the desired dig and dump points.
Because of power constraints and joint coupling effects The script parameters in the left hand side of the rules are
of the excavators hydraulic system, as well as the difficul- found through a combination of simple excavator dynamic
ty in accurately modeling the dynamics of such a machine, models and heuristics. These simple dynamic models cap-
more traditional optimal trajectory generation schemes do ture first order effects of the excavators closed loop behav-
not work well. Instead, recognizing the fact that the exca- ior when given desired angular position commands. These
vators motions are highly repetitive and very similar from models provide information about the velocities, accelera-
loading cycle to loading cycle, and that it operates in a rel- tions, and command latencies for each joint, which are used
atively small portion of its total workspace, a script based to intelligently coordinate the different joint motions, re-
approach to motion planning was adopted (Rowe and sulting in faster loading times. As an example, consider the
Stentz, 1997). A script is a set of rules which define the case when the excavator has finished digging, and the
general motions of the excavators joints for a certain task, bucket is raising up out of the ground. The excavator does
in this case loading trucks. These rules contain a number of not need to wait until the bucket has raised to its full clear-
variables, known as script parameters, which get instanti- ance height before swinging to the truck. Instead, it can be-
ated on every different loading pass. gin swinging at some earlier point as the bucket is still
raising, but it must have the knowledge provided by the dy- any point on the underside of the linkage is lower than the
namic models about how much time it will take to swing to elevation of the grid cell that coincides with it, then a pre-
the truck and to raise the bucket so it can safely couple the dicted collision is reported and the excavator is command-
two motions to avoid a collision. ed to stop.

4.5. Obstacle Detection

A major requirement for automated loading is detecting

and stopping for people and other obstacles which pose a
threat for collision. Obstacle detection software has been
developed which uses sensor data to perceive objects in the
excavators workspace, and simple dynamic models to pre-
dict where the excavators linkage will be for a short time
in the future as the excavator swings back and forth be-
tween the dig face and the truck. The predicted excavator
linkage locations are compared to the sensor data, and if
there is an intersection, the excavator is immediately com-
manded to stop. It is crucial that the sensors scan far
enough ahead of the excavators motion, and the prediction
is far enough in the future, for the excavator to have enough Figure 16. Typical dig for truck loading.
time and space to come to a complete stop and avoid hitting
the obstacle. This look-ahead distance is a function of the 5. Results
swing joints maximum velocity and was found through
experimentation to be between 40 to 50 in front of the ex- Figure 16 shows the excavator after digging a bucket of
cavators swing joint. soil, and Figure 17 shows the truck after it has been loaded
The prediction of the excavators location is done using with six buckets of soil. To date, we have autonomously
the simplified models of the excavators closed loop dy- loaded our truck hundreds of times. The typical loading
namic behavior. Not only is the obstacle detection algo- times are 15 to 20 seconds per pass, with six passes needed
rithm predicting what the excavator itself will do, it must to load the truck. This rate is very close to the loading times
also simulate what the motion planner will do using the logged by an expert operator manually loading trucks in the
predicted excavator state. It performs this prediction at the same configuration using the same excavator.
same rate of the script rule base update, 10 Hz for instance.
The final result is a list of predicted excavator linkage
states for some amount of time. The look-ahead time was
found empirically to be between 2 - 3 seconds.

underside
linkage
envelope

Figure 15. Depiction of the points that are calculated on the

underside of the linkage.

For each predicted linkage state, the coordinates of

points on the envelope underneath the linkage are comput- Figure 17. Truck is loaded after six passes.
ed. This is done using the forward kinematics of the exca-
vator and simple linear models of the shapes of the 6. Conclusion
linkages. This is shown in Figure 15. Each point on the un-
derside of the linkage for each predicted linkage state is We have demonstrated an autonomous loading system for
then compared to the 2-D elevation map of sensor data. If excavators which is capable of loading trucks with soft ma-
terial at the speed of expert human operators. The system Romero-Lois, H., Hendrickson, C., and Oppenheim, I. 1989. A
uses two scanning laser rangefinders to recognize the truck, Strategic Planner for Robot Excavation. In Proc. Sixth Inter-
national Symposium on Automation and Robotics in Con-
measure the soil on the dig face and in the truck, and to de- struction.
tect obstacles in the workspace. The system modifies both Rowe, P., and Stentz, A. 1997. Parameterized Scripts for Motion
its digging and dumping plans based on settlement of soil Planning. In Proc. International Conference on Intelligent
Robots and Systems (IROS). Grenoble.
as detected by its sensors. Expert operator knowledge is en- Sakai, T., and Cho, K. 1988. Operation System for Hydraulic
coded into templates called scripts which are adjusted us- Excavator for Deep Trench Works. In Proc. 5th International
ing simple kinematic and dynamics rules to generate very Symposium on Robotics in Construction. Tokyo.
fast machine motions. We believe ours to be the first fully Salcudean, S.E., Tafazoli, S., Lawrence, P.D., and Chau, I. 1997.
Impedance control of a teleoperated mini excavator, In Proc.
autonomous system to load trucks for mass excavation. of the 8th IEEE International Conference on Advanced Robot-
ics (ICAR), Monterey, California.
Acknowledgments Sameshima, M., and Tozawa, S. 1992. Development of Auto
Digging Controller for Construction Machine by Fuzzy Logic
Control. In Proc. of Conference Japanese Society of Mechani-
This paper summarizes the work of the Autonomous cal Engineers.
Loading System team. This team consists of Stephannie Schwartz Electro-optics Inc., Autosense Scanner, http://
www.seord.com/comsen/autosen.htm
Behrens, Scott Boehmke, Howard Cannon, Lonnie Devier,
Seward, D., Bradley, D., Mann, J., and Goodwin M. 1992. Con-
Jim Frazier, Tim Hegadorn, Herman Herman, Alonzo trolling an Intelligent Excavator for Autonomous Digging in
Kelly, Murali Krishna, Keith Lay, Chris Leger, Oscar Lu- Difficult Ground, In Proc. 9th International Symposium on
engo, Bob McCall, Ryan Miller, Richard Moore, Jorgen Automation and Construction, Tokyo.
Shulz, K. 1997. Imaging Ladar Camera for Washing Robots. In
Pedersen, Chris Ravotta, Les Rosenberg, Wenfan Shi, and Proc. of the 14th International Symposium on Automation and
Hitesh Soneji in addition to the authors. Robotics in Construction, Pittsburgh.
Singh, S. 1995. Synthesis of Tactical Plans for Robotic Excava-
tion, Ph.D Thesis, Robotics Institute, Carnegie Mellon Uni-
References versity, Pittsburgh, PA.
Singh, S. 1997. State of the Art in Automation of Earthmoving,
Bradley, D.A., Seward, D.W., Mann, J.E., and Goodwin, M.R. ASCE Journal of Aerospace Engineering, Vol 10, Number 4.
1993. Artificial intelligence in the control and operation of Singh, S. 1998. Multi-Resolution Planning for Earthmoving, to
construction plant-the autonomous robot excavator. Automa- appear in ICRA.
tion in Construction. 2(3). Takahashi, H., Kamata, H., Masuyama, T., and Sarata, S. 1995.
Bullock, D.M., and Oppenheim, I.J. 1989. A Laboratory Study Autonomous shoveling of rocks by using image vision system
of Force-Cognitive Excavation. In Proc. Sixth International on LHD. In Proc., International Symposium on Mine Mecha-
Symposium on Automation and Robotics in Construction. nization and Automation, Golden, Colorado.
June. Wohlford, W.P., Bode, B.D., and Griswold, F.D. 1990. New
Bullock, D.M., Apte, S., and Oppenheim, I.J. 1990. Force and capability for remotely controlled excavation. In Proc. 1990
Geometry Constraints in Robot Excavation. In Proc. Space Winter Meeting of the American Nuclear Society, American
90: Engineering, Construction and Operations in Space. Nuclear Society, pp. 628-629.
ASCE, Albuquerque.
Burks, B.L., Killough, S.M., and Thompson, D.H. 1992. Remote
excavation using the telerobotic small emplacement excava-
tor. 1992 Winter Meeting. International Conference on Fifty
Years of Controlled Nuclear Chain Reaction: Past, Present
and Future, American Nuclear Society, November. pp. 559-
560.
Feng, P., Yang, Y., Qi, Z., and Sun, S. 1992. Research on Control
Method of Planning Level for Excavation Robot. In Proc. 9th
International Symposium on Automation and Robotics in
Construction. Tokyo.
Grimson, W.E.L. 1990. Object Recognition by Computer: The
Role of Geometric Constraints, The MIT Press.
Huang, X.D., and Bernold, L.E. 1994. Control Model for
Robotic Backhoe Excavation and Obstacle Handling. In Proc.
Robotics for Challenging Environments. Albuquerque.
Lever, P., Wang, F., and Chen, D. 1994. Intelligent Excavator
Control for a Lunar Mining System. In Proc. ASCE Confer-
ence on Robotics for Challenging Environments. Albuquer-
que.
Nease, A.D., and Alexander, E.F. 1993. Air Force construction
automation/robotics. In Proc. 10th International Symposium
on Automation and Robotics in Construction (ISARC). Hous-
ton.
Rocke, D.J. 1994. Control system for automatically controlling a
work implement of an earthmoving machine to capture mate-
rial, US Patent 5528843.