Collaborative Autonomy for Manned/Unmanned Teams

Steve Jameson and Jerry Franke Robert Szczerba and Sandy Stockdale
Lockheed Martin - Advanced Technology Lockheed Martin Systems Integration – Owego
Laboratories [email protected]
[email protected]
Abstract
UAVs offer tantalizing capabilities to the warfighter, such as tireless observation, quick recognition, and rapid reaction to
today’s changing battlespace. These trends are important because they aid Warfighter in their duties. Today, unmanned
systems exist that extend the vision and the reach of the Warfighter. However, they spend so much time managing these
assets that they lose effectiveness as a Warfighter. This is a particular problem if the warfighter’s role is one demanding
continuous sensory and mental workload, such as the Co-Pilot/Gunner (CPG) of an Apache Longbow attack helicopter.
Autonomy, the ability of vehicles to conduct most of their operation without human supervision, can help relieve the burden
of providing continuous oversight of the UAV’s operation. This moves the Warfighter’s role from control to command,
enabling them to perform their duties more effectively and successfully. Collaboration, the ability of teams of vehicles to
coordinate their activities without human oversight, moves unmanned systems to the level of a true force multiplier, giving
a single human warfighter the power of multiple coordinated, intelligent platforms.

Introduction1
Lockheed Martin has developed a general architecture for
Collaborative Autonomy that provides both the Autonomy
and the Collaboration necessary to achieve this force
multiplication. This architecture provides the capability for
individual unmanned vehicles to operate with unparalleled
degrees of intelligence and autonomy, and for groups of
unmanned vehicles to operate together effectively as a
team, providing greater effectiveness than an equal number
of vehicles operating independently. Collaborative
Autonomy allows the human warfighter to command the
unmanned vehicles as an active member of a warfighting
team, rather than as a detached controller (Figure 1).
Central to the architecture are state-of-the-art software
Figure 1. Unmanned Vehicle Teams on the digital
components for Mission Planning, Collaboration,
battlefield can act as a force multiplier if they
Contingency Management, Situational Awareness,
have the autonomy and collaboration capabilities
Communications Management, Resource Meta-Controller,
necessary to operate in teams without extensive
and Vehicle Management. Lockheed Martin is currently
human supervision.
employing and expanding this architecture to turn state-of-
the-art unmanned vehicles into transformational and the unmanned vehicle – the Manned/Unmanned Team –
warfighting teams. to perform tasks more effectively or more safely than a
human warfighter can alone. This trend is certain to
Background
continue, since UVs have proven their effectiveness
The U.S. Military relies heavily on the use of unmanned
repeatedly in conflicts from Bosnia and Kosovo to
vehicles (UVs) for a variety of tasks, including surveillance
Afghanistan and Iraq. For a variety of reasons, it is not
and reconnaissance, explosive ordnance disposal, and to an
likely that we will see unmanned vehicles operating with
increasing degree for strike against terrorist and other
full autonomy in most military applications in the
targets. In all cases, the unmanned vehicle operates under
foreseeable future, and so the paradigm of
the direct supervision or control of a human warfighter. The
Manned/Unmanned Teaming will continue to be the
goal of the unmanned vehicles is to provide a force
dominant approach to the deployment of UVs in military
multiplier for the human warfighter that enables the human
applications.
1 One of the domains of particular interest is the teaming of
Presented at the American Helicopter Society 61th Annual Forum, Unmanned Air Vehicles (UAVs) with human pilots in a
Grapevine, TX, June 1-3, 2005. Copyright © 2005 by the American
Helicopter Society International, Inc. All Rights Reserved.
scout or attack helicopter such as an Apache Longbow. The
Distribution Statement A: Approved for Public Release, Distribution US Army, US Navy, and DARPA have pursued the
Unlimited. development of manned/unmanned teaming with human
helicopter pilots on several programs, including the Army’s already has a demanding workload. This collaborative
Airborne Manned/Unmanned System Technology autonomy approach enables an unmanned vehicle team to
Demonstration (AMUST-D) and Hunter Standoff Killer be truly transformational by enabling the following five
Team (HSKT) ACTD, the Navy’s Intelligent Autonomy critical attributes (Figure 2):
Future Naval Capability (IA-FNC) program, and the
DARPA/Army Unmanned Combat Armed Rotorcraft • Intelligent – Autonomous Mission Planning and
(UCAR) program, with Lockheed Martin as a participant. Execution rapidly finds and implements the best
To meet the demanding requirements of achieving a robust solution to complex tactical problems, ensuring
force multiplier capability while limiting human workload mission success on a dynamic battlefield.
demands, Lockheed Martin has developed an architecture • Collaborative – Collaboration and Teaming
and a set of technologies for Collaborative Autonomy which capabilities produce a lethal warfighting team that
provides: shares information, responsibilities, and tasks. It
interacts with human warfighters and other systems as
• A high degree of autonomy for each individual a team and not as separate individuals.
vehicle, enabling robust and sophisticated capabilities • Aware – Comprehensive, shared, and predictive
with limited human intervention Situational Awareness overcomes the “fog of war” to
• Collaborative team operations, enabling multiple enable precision engagements with precision
vehicles to operate as a team with the human information.
warfighter; allowing a single human to command • Responsive – Holistic Contingency Management
multiple vehicles with no more workload than a single ensures survival and mission effectiveness of UVs
vehicle. teams in the face of the unexpected, including a
In this paper, we provide an overview of the Collaborative “reflexive response” capability that allows intelligent
Autonomy approach, describe the details of the “short-circuiting” of higher level planning functionality
Collaborative Autonomy components, and describe a for rapidly changing battlefield conditions.
prototype implementation of the Collaborative Autonomy • Agile – Tactile Maneuvering exploits terrain and
architecture as a Manned/Unmanned teaming avoids obstacles, enabling the unmanned vehicle to
demonstration. survive and surprise.

Approach to Collaborative Autonomy Collaborative Autonomy is implemented by a Mission

Lockheed Martin has developed and demonstrated a Management system that provides the high levels of
revolutionary approach to Collaborative Autonomy for intelligence necessary for autonomous and collaborative
heterogeneous teams of manned and unmanned vehicles mission operations. Autonomy lets teams of unmanned
[1]. The approach is specifically oriented to allow a team of vehicles operate with only top-level human guidance and no
unmanned vehicles to be commanded by a warfighter such need for detailed supervision. Collaboration is essential for
as the CPG (Co-Pilot Gunner) of an Apache Longbow, who team effectiveness.

Figure 2. The Five Key Attributes of Mission Management.

Collaborative Autonomy Architecture external to the team. This approach is more extensible and
The Collaborative Autonomy architecture is segmented into scalable.
seven major components (Figure 3):
The architecture is extensible because the components are
• Mission Planning – develops plans for the team and for decoupled; analysis and development can be performed by
individual vehicles different disciplines with relative independence.
• Collaboration – manages team formation and Additionally, this means that novel algorithms can be added
interaction among team members with a minimum disturbance of existing components.
• Contingency Management – detects, assesses, and Collaboration is an integral part of the system architecture –
responds to unexpected events that an entire component is dedicated to it and many
• Situational Awareness – creates Common Relevant components have collaborative concepts at their core. For
Operating Picture (CROP) for team example, mission planning is hierarchical in nature so
• Communications Management – Manages the teams can be formed and reformed with tasks allocated and
interaction with the vehicle’s communications systems. reallocated to team members. Contingency management is
• Air Vehicle Management – Manages the air vehicle’s hierarchical in nature, supporting the concept of issues
flight systems, sensors, and weapons. being addressed at a team level. These team issues are
addressed poorly in conventional approaches.
• Resource Meta-Controller – Manages processing
resources and dynamically allocates them to different The architecture is scalable, because collaboration has been
components as necessary. incorporated at the core of key components. The
These components work in concert to achieve objectives architecture is intended to work with multiple instances of
itself so the team vehicles work synergistically and
without violating constraints. This system architecture
autonomously. Therefore, intermittent communication
offers substantial advantages over existing approaches, such
as recognizing the need to partition components requiring between instances or complete loss of an instance is
handled gracefully.
distinct disciplines for analysis, development, and operation
as well as the need for autonomy to be collaborative both
with other autonomous systems of the team and systems

Collaborative Autonomy
Mission Management
Resource
Meta-Controller

Functional Modules
Contingency
Mission Planning
Management

Situational
Awareness Intelligent Agents

Collaboration

Knowledge/Data
Air Vehicle Management Models
Communications Management

Air Vehicle

Flight Control System Weapons/Sensors Communications

DARRS025..ppt
Figure 3. Collaborative Autonomy is achieved through a Mission Management system made up
of a set of intelligent components that implement higher-level functions on top of the vehicle
systems.
Mission Planning capabilities), and vehicle/team and external asset capability
Mission Planning onboard the autonomous system performs information (e.g. payload availability and mobility models)
pre-mission and dynamic in-mission replanning for the are used. Mission planning generates mission plans
collaborative team. Mission planning develops including travel plans, sensor plans, communications plans,
collaborative synchronized plans for sensor employment, and weapon plans. At the team level, task objectives and
flight paths, communications, and engagements. constraints are generated for lower level mission planning
to honor. It then accepts, combines, and deconflicts those
Generally, most existing mission planning systems are
plans when lower level mission planning responds.
monolithic in nature. These systems are very good at
planning for specific situations that are predetermined, but Collaboration
are poor reacting to unforeseen events. Unfortunately, it is Collaboration, i.e. the ability of multiple vehicles to interact
unrealistic for a mission planning system to have planners to carry out a team mission, is inherent in the Collaborative
to handle all situations and all contingencies. To address Autonomy architecture. Most components, including
these shortcomings, Lockheed Martin has developed a Mission Planning, Contingency Management, Situational
revolutionary approach to handle this problem via the Awareness, and Communications Management are
Mission Planning Toolkit [2]. In this toolkit, planning designed to facilitate the collaborative operations of a team
algorithms are broken down into their smallest functional of vehicles.
subcomponents, called “primitives”. The algorithm
The Collaboration component embodies several functions
primitives are then collected into a library of modules, each
(Figure 4) that are uniquely required in order to support this
with specific inputs, outputs and functionalities. The toolkit
operation. These include:
is used to construct a specific planning system on the fly,
based on the current situational awareness. This allows for • Sharing Information and tasks
the dynamic construction of mission planners, as opposed • Allocating Roles and Responsibilities
to just mission plans, to handle unpredicted events. For • Coordinating Task Execution
anticipated or common mission scenarios, reconstructed • Dynamically forming teams
planner templates can be used for an even faster response • Interacting with external assets
times. • Interacting with human Warfighter.
During operation, the Mission Planning Toolkit works in a Two main technology elements of the Collaboration
hierarchical fashion with mission plans at the highest level component are the Grapevine information sharing
– such as Teams A and B recon area ZEBRA, team plans at technology, and the Dynamic team formation and
the next level, and individual vehicle plans at the lowest management.
level. These plans optimize and/or account for factors such
as: Grapevine information sharing [3] handles the aspects of
collaboration that deal with information sharing and
• High level mission objectives and constraints coordination between unmanned team members, between
• Resource allocation for the number of vehicles unmanned systems and the human Warfighter, and between
• Payload configuration for different mission objectives the unmanned team and external systems such as C4ISR
• Collaborative use of onboard sensors and external ISR and Networked Fires. On every unmanned vehicle, the
assets to detect, identify, and geo-locate vehicle and Collaboration component sets up intelligent agents known
dismounted infantry targets of interest as Proxies to represent each other manned or unmanned
• Communication events that support the teams' entity that vehicle needs to communicate with. Each Proxy
information dissemination and synchronization agent contains a set of criteria that are used to select and
requirements prioritize information for dissemination to the entity
• Routes that support the planned use of sensors and represented by the Proxy, known as the Client. The set of
communications while minimizing threat exposure Proxy agents are continually evaluating the information
• Target engagement planning and weapon deployment available to the Collaborative Autonomy system and
sequencing. selecting and prioritizing information for dissemination to
other manned and unmanned team members. The Proxy
Mission planning accepts objectives and constraints for
agent’s criteria are updated in response to changing
planning missions as well as alerts indicating that
conditions, such as new team members, changes in team
replanning is required. Geographic information (e.g. terrain,
member roles, or changes in mission tasking, and can also
obstacle, and cultural), environmental information (e.g.
be updated to reflect explicit requests for information from
weather), situational information (e.g. threat locations and
a human Warfighter or external system.
 SA Lead

Warfighter
• C4ISR Gatew ay
• Netw orked Fires
• Information
Sharing
• Command
Response
• C2 Handoff
• Status
• Plans
External • CROP
Asset
Gatew ay

Cooperative
Engagement
Sharing Information
Assigning Team Responsibilities
Warfighter Interaction
Team Lead Coordinating Tasks
External System Interaction
DARRS024..ppt

Figure 4. Collaboration performs the functions necessary to enable a team of unmanned vehicles
to function as a team of human warfighters.
One important aspect of the Grapevine is the sharing of Contingency Management
Situational Awareness information to form a Common A key challenge to successful autonomous operations is
Relevant Operational Picture (CROP) across the team. The detection and reaction to unplanned events that affect the
Collaboration component handles the information sharing execution of the vehicle system’s mission. Contingency
operations needed to construct the CROP, and the Management watches for unexpected influences that affect
Situational Awareness component does the information team plan success, such as payload failure, modified orders,
fusion and deconfliction necessary to assemble the shared new operational constraints, changing environmental
information into a CROP. conditions and other unexpected changes in the battlespace
(see Figure 5). It works with the Mission Planning
Dynamic Team Formation accommodates the formation
component to generate an effective response to the
and reformation of unmanned vehicle teams as required to
contingency so the mission can be continued.
meet the mission requirements. At the beginning of the
mission, the Collaboration component identifies the set of The Contingency Management component is implemented
team members required to meet the mission requirements, based on Lockheed Martin’s MENSA technology [4].
and these vehicles exchange information to set up a team. MENSA takes each new or updated mission plan and
Setting up a team includes determination and distribution to applies algorithms to identify plan dependencies and
all team members of the team membership, team structure, constraints. Based on these dependencies and constraints, it
and allocation of roles within the team. An important sets up monitoring agents to check for conditions that
element of the team is the allocation of roles to team violate those dependencies and constraints. During
members to perform responsibilities on behalf of the team, execution of the plan, these agents continually monitor
such as coordinating interaction with the human warfighter. available information to determine if their assigned
When a team member is lost or damaged, new team conditions are met. If the conditions are met, the agent
members become available, or when the mission changes, signals that the contingency has occurred and the reasons
the team members interact to reform the team and reallocate about the impact of that contingency on the mission plan. If
roles. Reforming the team can include splitting the team necessary, the Mission Planning component is requested to
into two smaller teams to accomplish separate mission modify the plan to take into account the contingency.
tasks, or merging two or more teams into a single combined
For example, vehicle health updates are related to vehicle
team.
operational capabilities (such as maximum endurance)
before being compared to the requirements of the executing
 Failure by Needed Loss of Contact
External Asset With Operator(s)

Weather and Other

Environmental Factors

Loss/Failure
of Teammate

Vehicle System
or Payload
Failure
Unexpected Developments
in the Battlespace
Changes in Orders and
Operational Constraints
DARRS026..ppt

Figure 5. Contingency Management Handles Unexpected Influences that Affect Mission Plan
Success.
plan to determine if the vehicle can perform its mission as Contingency Management detects a contingency, assesses
planned. Pop-up threats are assessed with respect to their the impact and identifies a plan violation, then:
influence on the planned route that the vehicles will take
1. The affected vehicle locally performs a replan which
through threatened territory. This mission-centric approach
may resolve the problem
to contingency management focuses computational
2. If there are tasks that could not be re-planned locally,
resources toward those problems that have real mission
contingency management then collaborates with other
impact and reduces the number of false alarms and
team members to reallocate tasks
unnecessary replans that occur.
3. If there is a reallocation failure, a team replan is
Contingency Management implements contingency triggered
monitoring and plan impact analysis for most contingency 4. If a team replan cannot resolve the situation,
types, including air vehicle flight capability degradation, contingency management alerts the controlling element
pop-up threats and targets of opportunity, friendly and (typically a manned asset) of a team planning failure
neutral movement within the battle space, loss of team and awaits updated guidance.
members, and mission equipment failures. Contingency
Situational Awareness
Management can also determine when an emergency
The Situational Awareness (SA) component gathers data on
mission abort is required and provides the controlling
the external tactical and environmental situation and
element with control over the level/type of contingency
processes it into a CROP, which the other Mission
monitoring performed. Contingency Management takes in
Management components use to make their decisions. A
mission plans and information regarding the changing
pilot or crewmember needs good situational awareness to
situation (e.g. new objectives, new constraints, new
perform effectively in a manned system. Intelligent
obstacles, new threats, new targets, and changes in
autonomous systems also require complete, timely, specific,
vehicle/team capabilities). It issues alerts when plans will
and relevant information to make good “decisions”.
no longer satisfy objectives and constraints. At the team
level, it takes in alerts of contingencies that cannot be The Situational Awareness (SA) module is implemented by
handled at a vehicle level and issues alerts to team mission leveraging Lockheed Martin’s technology for Level 1 Data
planning for replanning. Fusion [6] and Battlefield Assessment, originally developed
on the Rotorcraft Pilot’s Associate program. SA performs
Our contingency management approach features a team-
multiple levels of assessment of the data [5] from onboard
wide contingency resolution escalation process where
 C4ISR Netw ork
Situational Aw areness

Level 1 Level 3
Predictive • Mobility
Object
Battlespace • Intent
Assessment
Awareness

CROP With Selected

Images Level 2 Level 4
Situation Process
Assessment Refinement
Cues, Sensor
Requests, Tracks
Environmental SA

Sensor Coverage SA
Lead
• Threat
Clustering ID Intervisibility
13 • Threat Priorities
Team Shared
2 2 CROP
• Obstacles
• Weather
Threat Relationships

DARRS023..ppt

Figure 6. Situational Awareness provides a comprehensive assessment of all battlespace information to

enable the Collaborative Autonomy functions to operate with precision information.
sensors and external data sources (Figure 6) to produce the components to make autonomous decisions that guide
CROP. Level 1 Object Assessment consists of fusing data vehicle behavior.
from onboard sensors, teammate sensors, and external data
Air Vehicle Management
sources such as C4ISR networks to produce a set of tracks
Air vehicle management (AVM) provides the link between
representing friendly and threat entities in the battlespace.
the Collaborative Autonomy components and the vehicle
In addition to this fusion, SA deconflicts data from each of
systems. It translates tasks from the Mission Planner into
the teammates to ensure that each vehicle’s CROP is
commands for the vehicle sensors, weapons, and flight
consistent.
systems and acts as the point of entry for information from
Level 2 Situation Assessment consists of evaluating the these vehicle systems into Mission Management.
fused track picture in the CROP to assess friendly and
AVM refines route plans to minimize overall exposure to
threat sensor coverage and intervisibility, potential threat
threats factoring in terrain masking, collision risks and
organizations, and the priority associated with different
vehicle dynamics. AVM provides reflexive obstacle and
threats. Level 3 Predictive Battlespace Awareness [7]
threat response capability to enhance overall system
determines likely threat mobility and future locations, and
survivability. AVM quickly maneuvers the vehicle out of
assesses likely threat intent. Level 4 Process Refinement
harm’s way while the more deliberative system autonomy
determines when the information being produced by
generates a re-plan to achieve mission objectives. AVM
Situational Awareness does not meet the requirements of
generates trajectory commands based on a library of
the mission, and takes action to generate additional
maneuver primitives, including agile maneuvers that fully
information such as requesting Mission Planning to task
span the available flight envelope, providing enhanced
sensors, from other vehicles, or from the C4ISR networks.
maneuvering effectiveness for survivable threat response.
In addition to this multi-level processing of sensor AVM accepts travel plans (e.g. flight plans), threat
information, Situation Awareness collects and maintains warnings from onboard sensors, and obstacle warnings
other types of information such as weather data, from obstacle sensors and generates maneuvers to the
environmental information, and obstacle maps. This vehicle actuator systems.
information is also used by Mission Planning and other
As part of plan refinement, the Mission Planning generates Management is shown in Figure 7.
routes between mission “hard points” that minimize the
Resource Meta-controller
total exposure to known external threats by factoring in the
Resource Meta-controller (RMC) is a software
threat type, location, lethality radius, terrain elevation
infrastructure component providing processing and memory
profiles, and the vehicle’s exposure given its position,
resources for other components. RMC operates in concert
speed, and attitude. The route plan is then refined within the
with operating system level resource management
route plan constraints to further reduce threat exposure,
functions. RMC performs system management functions
generating detailed flight trajectories that adjust speed,
such as processor switchover, memory zeroize, pre- and
aspect angles, and altitude above the local terrain to reduce
post-mission data exchange and fault isolation. RMC
the total risk of exposure to external threats. The plan
manages computational resources by performing resource
refinement component of AVM produces trajectories that
utilization monitoring, resource allocation to agents,
meet the route plan goals and constraints, while reducing
resource reclamation and reallocation, resource tracking,
the exposure to risk from external threats and terrain
and resource scheduling and optimization. The RMC Agent
collisions.
Supervisor manages agents by agent creation and
Communications Management destruction, agent registration and monitoring, job
Communications Management provides and manages data assignment and status reporting, and agent suspension and
links to connect team members with each other and with resumption. RMC provides other components with access
external assets (e.g., ISR and Networked fires) over to data by managing publication/subscription interchanges,
battlefield networks. Communications software manages managing data retention and performing structured queries
this system by: implementing the communications plan upon request.
provided by mission planning using available system data
Implementation
links, predicting and monitoring communication Quality of
Mission Management is the component that provides the
Service (QoS) and optimizing performance of the data
intelligence for the unmanned team to make collaborative
links. Communications may also request Mission Planning
and autonomy decisions. The architecture for the Mission
to modify plans to keep QoS at effective levels. The
Management segment was instantiated in the
relationship between Collaboration and Communications

Comm HW Comm Collaboration Mission

Manager SW Services Management
• Encryption • Message • Team
• LPI/LPD Formatting Management Mission
Waveforms • Netw ork • Information Planning
• Modulation/ Management Dissemination/
Demodulation • Flow Control Routing
• Signal • Hardw are • Information
Transmission Control Situational
Prioritization
Aw areness
• BIT • Hardw are • Proxy/Gateway
Interface Protocols
• Error • Negotiation
Management Protocols Contingency
Management

DARRS022..ppt

Figure 7. Communications Management and Collaboration interact to ensure that needed

information is exchanged among team members and with external systems.
Manned/Unmanned (MUM) Teaming Demonstration that representative system for evaluations and performance
verified the high levels of intelligence necessary for analysis, forming a robust test bed that accommodates a
autonomous and collaborative mission operations can be number of changes into the environment and assesses the
achieved. Autonomy lets the vehicle operate with only top- capability of the manned/unmanned team to respond to the
level human guidance and no need for detailed supervision. changes. The demonstration showed that the combination of
Collaboration is essential for team effectiveness. manned and unmanned team attack assets provided a new
level of situational awareness and operational flexibility not
The MUM Teaming demonstration showcased critical
currently available.
concepts including autonomy and collaborative operations,
Human Machine Interface (HMI) and workload A representation of the MUM teaming demonstration is
management approaches and technologies, and manned shown in Figure 8.
unmanned system interaction. The MUM Demonstration
The demonstration culminated in an exercise where an
created a high fidelity simulation-driven test bed to develop
independent team injected a number of changes into the
and evolve MUM concepts and to mature the autonomy and
simulation environment and assessed the capability of the
collaborative technology. The simulation showcased the
manned/unmanned team to respond in an autonomous and
robustness of the MUM approach by responding to
collaborative manner to the changes. The results of the
contingency behaviors and by providing a new level of
MUM teaming demonstration were exciting and
situational awareness and operational flexibility. The
convincing. The HMI approach, including spoken language
simulation was an evolutionary development of increasing
systems voice command and response, provided a natural
fidelity over time to demonstrate autonomous and
communication modality for the MUM team commander.
collaborative operations and assess the technical feasibility
The team-based mission management, collaborative
of achieving this capability.
autonomy algorithms, and system implementation clearly
The MUM Teaming demonstration contained a team of showed the advantages and possibilities in team based
simulated rotary wing vehicles commanded from either a operations with highly intelligent vehicles. The Advanced
ground command console or a manned aircraft, represented Tactical Combat Model (ATCOM) simulation results
as a simulated Apache Longbow. The testbed also utilized the dynamic re-planning and contingency response
incorporated command and control nodes, Command, available in the MUM Teaming Demonstration. Finally, the
Control, Communications, Computers, Intelligence, demonstration showed that the combination of manned and
Surveillance, and Reconnaissance (C4ISR) network, and unmanned team attack assets provided a new level of
threats, in addition to the human machine interfaces and situational awareness and operational flexibility not
information exchanges between these simulated currently available.
components. The simulation environment provides a

C4ISR NET / GIG

Team of Unmanned
Air Team Commander Vehicles

Threats

HQ

Ground Team
Commander

Figure 8. MUM Teaming Demonstration.

The Air Team Commander is a member of a team Acknowledgments
containing unmanned vehicles (UVs) and provides mission Some of the work described in this paper was funded under
redirects to the UVs, receives/dispositions target the OTA portion of the DARPA Unmanned Combat Armed
engagement requests, and receives vehicle status/mission Rotorcraft (UCAR) program (MDA972-02-9-0011). The
plans. The team commander had high fidelity controls and authors wish to recognize the following individuals who
on-board displays to control the UV team. These have made recent contributions to this system:
capabilities include: tactile vest, spoken language system,
Draper Laboratory – Brent Appleby, Mark Homer, Leena
and multi-purpose display pages with a digital map. The
Singh, Lee Yang, and their team
Ground Team Commander is used with the unmanned
vehicles to ensure that the vehicles are prepared for the Lockheed Martin Advanced Technology Laboratory -
mission and execution of the initial and final portions of the David Cooper, Rich Dickinson, Chris Garrett, Adria
mission. Hughes, Mike Orr, Brian Satterfield, Mike Thomas, and
Vera Zaychik
The simulation included software-in-the-loop execution of
critical team-based mission planning, autonomy, Lockheed Martin Simulation Training and Support – Ken
collaboration and contingency management functions. The Stricker and Brian Vanderlaan
system included a six degree-of-freedom air vehicle model.
The system implemented reflexive maneuver elements of Lockheed Martin Systems Integration – Owego – Erin
the air vehicle management system and collaborative team Accettullo, Rick Crist, Steve DeMarco, Dave Garrison, Carl
Herman, Adam Jung, Ateen Khatekhate, John Moody,
searches and target engagements with weapons release
authority provided by the manned element. The system Donn Powers, Greg Scanlon, Mike Scarangella, Keith
explored key aspects of the HMI solution, which included Sheppard, Tom Spura, Peter Stiles, and Joel Tleon
spoken language system voice command and response, UCAR Government Team - Bob Boyd, Marsh Cagle-West,
tactile vest for alerting, mission controls and displays Steve MacWillie, Steve Rast, Randy Scrocca, CW4 Matt
integrated with existing systems, and workload Thomas, and Don Woodbury
management functions, including negotiated intervention.
References
A distributed team was responsible for the demonstration: [1] R. Szczerba, D. Garrison, and, N. Ternullo,
Lockheed Martin Systems Integration - Owego, Lockheed "Autonomous UAV Team Planning for
Martin – Advanced Technology Laboratory, Draper Reconnaissance Missions", Proceedings of the 59th
Laboratory, Lockheed Martin – Simulation Training and Annual Forum of the American Helicopter Society,
Support, Naval Aeromedical Research Lab (NAMRL), and Phoenix, AZ, May 2003.
Cepstral. [2] R. J. Szczerba, P. Galkowski, I. Glickstein, and N.
Ternullo, "Robust Algorithm for Real-Time Route
Conclusion
Through this work, an architecture and a set of technologies Planning," IEEE Transactions on Aerospace and
have been developed with the potential for tremendous Electronic Systems, Vol. 36, No. 3, pp. 869-878, July
warfighter benefit as it matures and transitions into 2000.
[3] "Information Sharing in Teams of Self-Aware
operational use. Our analysis work and the MUM
Demonstration have shown that our approach to Entities," J. Franke, B. Satterfield, S. Jameson, 2003
Collaborative Autonomy is highly effective in demanding Multi-Robot Systems Workshop, Naval Research Lab,
Washington, DC, March, 2003.
military scenarios. Our work with Army Subject Matter
Experts has validated the appropriateness of the concept [4] "Self-Awareness for Vehicle Safety and Mission
operationally and the feasibility for the warfighter with the Success," J. Franke, B. Satterfield, M. Czajkowski, S.
Jameson, 2002 Unmanned Vehicle Systems
limited workload required in a demanding role such as
copilot/gunner of an Apache attack helicopter. Technology Conference, Brussells, Belgium,
December, 2002.
Both the architecture and the technologies are scalable and [5] White, F.E., “A Model for Data Fusion”, Proc. 1st
extensible, in terms of the size of the unmanned teams, the National Symposium on Sensor Fusion, 1988
degree of capability of the unmanned vehicles, the type of [6] "RPA Data Fusion," Don Malkoff and Angela
manned platforms involved, and the required interaction Pawlowski, 9th National Symposium on Sensor
with external systems. This extensibility has the potential to Fusion, Vol.1, Infrared Information Analysis Center,
provide significant benefit in a wide range of domains, September 1996.
including both rotary and fixed wing UAV’s, Unmanned [7] "Situation and Threat Refinement Approach for
Ground Vehicles (UGVs) teamed with humans, and Combating the Asymmetric Threat," Angela
heterogeneous air/ground teams. We are currently pursuing Pawlowski, Sergio Gigli, and Frank Vetesi, Military
numerous avenues to extend, mature, and transition this Sensing Symposia, National Symposium on Sensor and
technology so that America’s warfighters can get maximum Data Fusion 2002, San Diego, CA, August 13-15,
benefit from the promise of intelligent unmanned systems. 2002.