See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/228902672

Data Fusion for the Apache Longbow: Implementation and Experiences

Article

CITATIONS READS
2 280

7 authors, including:

Steve Jameson Craig Stoneking

BAE Systems Lockheed Martin Corporation
13 PUBLICATIONS 111 CITATIONS 13 PUBLICATIONS 64 CITATIONS

SEE PROFILE SEE PROFILE

All content following this page was uploaded by Craig Stoneking on 29 May 2014.

The user has requested enhancement of the downloaded file.

 Data Fusion for the Apache Longbow: Implementation and Experiences
Steve Jameson1, Craig Stoneking2, David G. Cooper3, Peter Gerken2, Chris Garrett2, and Adria Hughes3
1
Manager, 2Senior Member of the Engineering Staff, 3Member of the Engineering Staff
Lockheed Martin Advanced Technology Laboratories
Cherry Hill, NJ 08002
{sjameson, cstoneki, dcooper, pgerken, cgarrett, ahughes}@atl.lmco.com

Abstract systems, unmanned platforms, and decision aiding

systems in the forward elements of the combat force.
Maintaining Situational Awareness and Tactical
This work is exemplified by the Airborne Manned/
Decision Making are workload-intensive and time-
Unmanned Systems Technology Demonstration
critical challenges for the crew of the Army’s AH-64D
(AMUST-D) and the Hunter Standoff Killer Team
Apache Longbow. The Apache crew faces these
(HSKT) Advanced Concept Technology Demonstration
challenges with extreme mission demands coupled with
(ACTD) programs, led by the U.S. Army Aviation
stressful high-speed, low-level flight. Two technology
Applied Technology Directorate (AATD) [1].
trends show great promise in addressing these problems:
AMUST-D and HSKT will provide airborne warfighters
Decision Aiding and Manned/Unmanned teaming with
and mobile commanders with improved situational
Unmanned Aerial Vehicles (UAV). The US Army
awareness from the cooperative construction of a shared
Aviation Applied Technology Directorate (AATD) is
common operational picture of the battlefield, through
leading the Airborne Manned/Unmanned System
the sharing and fusion of information from all
Technology Demonstration (AMUST-D) and Hunter-
exploitable sensors and intelligence data sources that
Standoff Killer Team (HSKT) programs to develop,
bear on the battlespace. Central to this program is the
deploy and demonstrate these technologies in
development of the Warfighter’s Associate (WA)
operational evaluations. Data Fusion, the capability to
decision aiding system for the Apache Longbow, and
integrate information from multiple sensors and other
the Mobile Commander’s Associate (MCA) system for
sources into a consistent Common Relevant Operational
the A2C2X Blackhawk.
Picture (CROP), lies at the heart of both of these
capabilities. A reliable CROP is required to support the
LM ATL, under contract to AATD, is providing a
automated reasoning processes of Decision Aiding, and
Multi-Sensor Data Fusion system to provide the
to automatically combine sensor data from teamed
necessary Situational Awareness for these decision
UAV’s, reducing the significant human workload that
aiding systems to function. In addition, under
would occur if UAV data were to be manually
AMUST-D, LM ATL has produced a Distributed Data
combined with other fused data representations. For the
Fusion capability for multi-platform operation to
past 11 years, Lockheed Martin Advanced Technology
produce a shared CROP representing the best situational
Laboratories (LM ATL) has been developing a Data
awareness available across a network of communicating
Fusion capability specifically designed to support Army
platforms (Figure 1) [3]. LM ATL’s Data Fusion system
Aviation decision aiding systems. In this paper, we
is integrated into both the MCA and WA decision
describe LM ATL’s effort to design, implement, and
aiding systems [4]. Implementation, integration, and
test a Data Fusion system for the Apache Longbow
evaluation of Data Fusion in WA is challenging because
under the AMUST-D and HSKT programs.
of the highly embedded nature of the processing on the
Apache, the sensor information available to the Apache
Introduction avionics system, limitations of the display, and the very
The Army Aviation community is promoting the tightly focused nature of the Apache Pilot and Co-
development of technologies and systems that support Pilot/Gunner (CPG) tasks.
effective on-the-move command of airborne and
ground-based maneuver forces through shared situation This paper provides a brief overview of the general Data
awareness and decision aiding technologies. The Fusion capability LM ATL has produced. We discuss in
operational concepts for these technologies and systems detail the limitations and challenges inherent in
are characterized by the extensive use of mobile sensing implementing Data Fusion in the Apache Cockpit. We
describe the details of the design and implementation
Presented at the American Helicopter Society 61st Annual Forms, needed to overcome those limitations and challenges,
Grapevine, TX, June 1-3, 2005. Copyright © 2005 by the American
Helicopter Society International, Inc. All rights reserved.
presenting results from testing of the WA implementa-
tion of Data Fusion, both in engineering tests and pilot
 system meets real-time and resource requirements. The
Fusion Dispatch module evaluates incoming data,
determines which algorithm set—embodied in a fusion
Kernel module—should be applied to fuse the data, and
dispatches the appropriate Kernel to process the data
set. The Fusion Control module monitors the
performance and resource usage of Data Fusion, and
applies control measures such as prioritization or down-
sampling of input data, in cases where the Data Fusion
process begins to exceed resource limitations (such as
memory or CPU usage) or fails to meet timing
requirements. This separation of the top-level control
from the fusion algorithms allows the Data Fusion
system to be configured readily to meet different
performance and resource requirements in different
Figure 1. AMUST/HSKT Shared Situation Aware- applications.
ness Architecture.
A set of Input and Output modules manages the real-
feedback. Finally, we describe our vision for the future time interfaces with the sensor systems and the other
applicability of Data Fusion in aircraft like the Apache, components of the RPA system. The input modules read
and the developments needed to make this possible. input from the sensor systems at a sensor specific rate,
using a sensor-specific input protocol, and pre-process
Multi-Sensor Data Fusion the input using tailored sensor-specific routines into a
common intermediate input format and information
From 1993 to 1999, LM ATL participated in the Army’s
content. The output modules take the fused trackfile and
Rotorcraft Pilot's Associate (RPA) Advanced
output it to a client, in a client-specified format, using a
Technology Demonstration program, sponsored by
client-specific protocol. The modular, object-oriented
AATD. LM ATL developed the multi-sensor Data
Data Fusion software architecture, including the use of a
Fusion system [2] that provides a common fused track
common intermediate data input format, permits a
picture to the RPA pilot and the RPA decision aiding single core body of Data Fusion code to be easily
systems. In the RPA Data Fusion system, data
adapted to multiple input and output formats and
representing as many as 200 battlefield entities, from 14
requirements, facilitating portability. On AMUST-D,
different types of onboard and offboard sensors, is
two different versions of the Data Fusion system will be
correlated and fused in real time into a consolidated
deployed on the Longbow Apache and A2C2S
picture of the battlespace. The RPA system, including
Blackhawk helicopters. Both versions will contain the
LM ATL’s Data Fusion system, was successfully flight
same common fusion core, coupled with platform- and
demonstrated on an AH-64/D in August 1999. The Data
sensor-specific input and output modules.
Fusion system (Figure 2) consists of four main
elements.
A Track Management module stores all track data and
maintains the relationships among a set of track
Source- Core Fusion Process Client-Specific databases, one for each sensor providing input, and a
Specific Input Fusion Fusion Output
Modules Dispatch Control Modules Central Trackfile that stores the fused picture.
JSTARS
Track Fusion
Additional databases also provide access to a variety of
Input
Module Kernel data about platform, sensor, and weapon characteristics
MTI Fusion CORBA used by Data Fusion.
Grapevine Kernel Output Module
Input Track
Module
Group Fusion
Management
Shared
A set of Fusion Kernel modules performs the heart of
Kernel
JCDB
Memory the correlation and fusion processing. As with the input
Intel Fusion Output Module
Input
Kernel
and output modules, the modular nature of the fusion
Module
process makes possible the encapsulation of algorithms
Figure 2. LM ATL’s real-time multi-sensor data tailored to a specific input data type into a kernel
fusion system. module. As input data sets are received, the appropriate
Kernel is applied to that data set, with the resulting
A top-level control structure, including Fusion Dispatch output passed to the Track Management module to
and Fusion Control modules, controls the application of update the fused trackfile.
fusion algorithms to the input data and ensures that the
The general functional flow followed by each kernel Entity SENSOR 2
CLASS:
follows a similar set of steps. Figure 3 illustrates the AIR DEFENSE
Air
steps followed for MTI (Moving Target Indicator) data. Land
The Prediction function performs time-alignment across SENSOR 1
Wheeled
CLASS: Tracked
the sets of data being fused to ensure that valid TRACKED
comparisons can be made. The Clustering algorithms Armor Artillery Air Defense Support
break down the battlespace into geographically distinct
clusters to limit the computational complexity of the Resulting Class: Tracked ADU Wheeled ADU
following algorithms. Cost Functions operate on each TRACKED AIR
DEFENSE
cluster to compute a matrix of composite similarity ZSU-23 Resulting ID:
ONE OF
values between each input sensor data item and the 2S-6
THESE 4
SA-13
candidate fused tracks. The Assignment function uses
SA-15
the optimal JVC (Jonker-Volgenant-Castanon)
algorithm to compute matches between sensor data and Figure 4. Example of class fusion in RPA data fusion.
fused tracks. Once the matches are identified, Fusion
algorithms are applied to update the state of the fused Apache Longbow Implementation
trackfile based on the associated sensor data using an
Interacting Multiple Model (IMM) Kalman Filter. Overview of Warfighter’s Associate and WA Data
Fusion
The Warfighter's Associate (WA) system, developed by
Boeing Phantom Works in Mesa, AZ, is the decision
aiding system developed under AMUST-D/HSKT to
support the pilot of the Apache Longbow. WA provides
the following capabilities to the Apache pilot:
Fused Common Picture
at time t
Prediction: Sensor Data
received at time t+1 and
Clustering: Data grouped
into geographic clusters • A Route Planner that allows the pilot to plan a
time-aligned
detailed route given a set of high level waypoints,
terrain, and time and fuel constraints.
• A Route Assessor that continually monitors the
route for exposure to threat, speed constraints, low
fuel, and potential failure to meet time constraints.
• An Attack by Fire (ABF) Planner that selects the
Cost Functions: Similarity Assignment : Associations Fusion: Algorithms produce
Metric applied to clusters formed based on cost values updated common picture at most appropriate location from which to attack a
time t+1
threat based on a variety of tactical heuristics.
Figure 3. Data fusion kernel functional flow. • Display Aids, including Ownship and Threat
intervisibility, and Search Area Covered.
One of the key algorithmic advances of the RPA Data • UAV Management functionality, including a
Fusion system was its ability to effectively combine the TCDL link to the UAV and a variety of UAV
processing of kinematic (position and velocity) management aids.
information with the processing of Class (vehicle type), Integral to many of these functions is the need to have
ID (specific vehicle information), and IFF (friend/ up-to-date, consistent, information on threat and
hostile) information. This processing takes place during friendly entities in the battlespace, which is the role of
the comparison of sensor data with fused tracks, by the Data Fusion.
Cost Functions, and during the updating of the fused
trackfile with associated sensor data by the Fusion The Role of Data Fusion in Warfighter’s Associate
algorithms. The Class Cost Function and Fusion
algorithms compare and combine Class and ID In order for WA to perform intervisibility calculations
information expressed in a class hierarchy (Figure 4). with respect to a threat, and to plan a route to best avoid
Each sensor report or fused track has a class that threat, it is desirable to have an accurate and stable
representation that specifies the confidence of each node estimate of the threat’s location. Data Fusion provides
in the hierarchy. A set of Modified Bayesian Evidence support for these automated decision aiding functions of
Combination algorithms developed by LM ATL is used WA, as well as supporting WA in providing the pilot
to compare, combine, and summarize this information. with better situational awareness, with an operational
LM ATL’s work in this area [2] represented a major picture that is more stable, more accurate, more
advance in Data Fusion technology. complete, and less cluttered. These benefits are realized
through Data Fusion’s scan-to-scan tracking, multi-
sensor fusion, and false target removal capabilities.
Scan-to-Scan Tracking contributes to the stability, false target represents an actual object on the battlefield,
accuracy and completeness of the tactical picture. so that it shows up in subsequent FCR scans, but is an
Without Data Fusion, each time the FCR performs a object of little or no interest, or a target that has already
scan, all of the targets detected in the previous scan are been destroyed. In this case, once the operator
removed from the display and replaced by the results of determines that the object is not of interest, the WA
the new scan. Since the FCR scans only a limited sector system allows the operator to “disallow” the
in any one scan, targets from a previous scan can be lost corresponding target track. When subsequent sensor
from the display when the FCR’s scan sector is moved. reports on the target are input to Data Fusion, it can
Even for a target that is detected by both the previous associate the new report with a disallowed track, and
and the new scan, it can be difficult for the operator to refuse to process it. This way, a false target that is
determine which target from the new scan corresponds persistently detected by the FCR can be excluded from
to a target in the previous scan. Data Fusion alleviates the display.
this problem with scan-to-scan tracking. The first
detection of a target by the FCR establishes a target Design Challenges
track, with an identifying track number. With Design, implementation, integration, and evaluation of
subsequent scans, Data Fusion is able to associate a new Data Fusion in WA are challenging because of the
detection of that target with the established track, sensor data available on the Apache Longbow, the
providing continuity of that track across scans. Targets highly embedded nature of the processing on the
not reinforced by the subsequent scan do not Apache, and the need to conform to existing Pilot and
immediately disappear with the new scan. Rather, Data Co-Pilot/Gunner (CPG) task expectations. Each of these
Fusion uses a time-based scheme for eventually challenges imposed design constraints that resulted in
eliminating tracks that are not reinforced. Each time a specific design decisions during the design and
track is reinforced by a subsequent scan, the position development of the WA Data Fusion.
location from the new detection can be used to refine
the accuracy of the existing track. Sensor Data
Multi-Sensor Fusion increases accuracy and reduces WA Data Fusion was required to accept and fuse data
clutter in the tactical picture. Feedback from pilots, from a wide variety of sources, for which the original
during flight test simulations, emphasized that an data fusion design was well suited, having been
important situation awareness concern that the pilots designed to fuse up to 21 different data sources.
have is the unambiguous determination of how many However, the original design relied on assumptions
targets are present on the battlefield. Without Data about the periodic nature of the input sources for input
Fusion: if the FCR scanned a target whose position had processing and some of the correlation and fusion logic.
already been previously stored from another source, The fusion algorithms rely on the availability of certain
such as TADS, then the pilot would see two symbols for attributes for all sensor reports received.. These include
the target on the Tactical Situation Display—one for position of the detected entity, time of the detection, the
each reporting source. Data Fusion alleviates this class/ID of the entity, and the uncertainty (error ellipse)
problem by recognizing that the reports from the associated with the detection. The sensor data available
different sources refer to the same object on the on the Apache posed design challenges because most
battlefield, and fuses them into a single target track, so sources did not report periodically and none of the
that the pilot sees a single display symbol per real-world sources provided all of the information required by the
target. Fusing reports from multiple sensors reduces original Data Fusion algorithms (Table 1).
clutter and improves the completeness and accuracy of
information on a track, in the same way that scan-to- Almost all of these sensors report neither the time of the
scan tracking does. measurement nor the expected error/uncertainty in the
measurement. This information is particularly important
False Target Removal builds upon scan-to-scan for Data Fusion to make a reasonable estimate of the
tracking to further improve accuracy and reduce clutter positional uncertainty of any track that it forms from
in the tactical picture. Data Fusion assists the sensor data input. This forced us to accept the time of
recognition and elimination of ephemeral and persistent reporting to Data Fusion as the time of measurement,
false targets. An ephemeral false target shows up in one and to assign to each sensor a default error estimate,
FCR scan, but is not reinforced by a subsequent FCR based on discussions with pilots and engineers
scan of the same area. Tracks that are not reinforced by associated with the Apache program, that could be
subsequent scans decay, or “age-out”, and are removed, refined, through experience in simulated and actual
or “dropped”, after a certain period of time. A persistent operations, to achieve reasonable results.
Table 1. The data characteristics of the sensors that are fused on the Apache.
Reports Reports
Reports Reports Reports Reports
Sensor/Source Detection Position Periodicity
Position Velocity Bearing Class
Time Uncertainty
Enhanced GPS/
Inertial Navigation Yes Yes No Yes No No 1 Hz
Unit (EGI)
Fly-Over Store No Yes No No No Yes Crew Initiated
Crew Store No Yes No No No Yes Crew Initiated
Target Acquisition
/Designation No Yes No No No Yes Crew Initiated
System, (TADS)
Fire Control Crew Initiated Scan or
No Yes No Yes No Yes
Radar (FCR) continuous scan
Teammate Fire Teammate aircraft crew
Control Radar No Yes No Yes No Yes initiates scan and
(FCR) via IDM transmission
Radio Frequency Reports at 10Hz, but
No No No No Yes Yes
Interferometer measures at < 1 Hz
Radio Frequency
Hand Over Yes Yes Yes Yes No Yes Wingman Initiated
(RFHO)
Electronic Data
Transfer Cartridge No Yes No No No Yes One Time
(EDTU)

The reason for the importance of periodicity in input Many sensors are expected to report only once on a
sensor data is twofold. First, Data Fusion makes the target, and others (most notably the FCR) report only on
assumption that a given sensor will never report twice activation by the pilot/operator. Without a reliable
on the same target within the same reporting interval, reporting interval for its contributing sensors, the
the length of which is a property of the sensor. For approach to the problem of determining when a track
example, a traditional rotating radar system has a should be declared lost and dropped needed to be
reporting interval equal to the period of rotation of the modified for WA Data Fusion. Through consultation
antenna, and during that period it can reliably be with Boeing engineers and pilots, we worked out an
assumed that the radar will not report twice on a target. approach that is consistent with the way the pilots
This allows Data Fusion to assume that two closely employ the sensors in an operational environment. For
spaced reports from the same sensor in that time interval those sensors that are expected to report only once on a
must be separate targets. For a sensor that does not target, in most cases the target is assumed by the pilots
report periodically, we must identify a time interval to be stationary and will never be dropped. Data Fusion
within which it can reliably be assumed that closely incorporates this assumption. In the case of the FCR, we
spaced reports are from separate targets. Our design implemented a track age-out rate that caused the track to
then accumulates input from that sensor over the go stale in about 30 seconds. This was selected based on
identified time interval and processes the accumulated pilot feedback as the period beyond which they did not
data as though that data represented a periodic reporting feel that the data was sufficiently reliable to be shown
of data. on the cockpit display.

The other reason for the importance of periodicity is to Radio Frequency Interferometer (RFI)
determine when a track can be dropped. In previous One of the most challenging aspects of the Apache
applications, Data Fusion could expect that if a sensor’s sensor data is the nature of the data from the RFI. This
regular reporting interval elapsed without a target’s provides directional indication of radar systems
track being updated by that sensor, then that track could employed by threats such as Surface to Air Missile
be considered lost. Once the track is lost, Data Fusion (SAM) batteries, and a high confidence identification of
continues to maintain the track, but with increasing the threat type. Currently the only information available
uncertainty in the track’s location. When the uncertainty to a pilot is the directional indicator on the display,
in the track location reaches a specified threshold, Data along with the threat type. One of the goals of the Data
Fusion “drops,” (stops reporting on) the track entirely. Fusion development was to implement a capability to
In the current WA application on the Apache, almost convert this directional information over time into an
none of the sensors have a regular reporting interval.
estimate of the target’s position, and – most importantly sensor is used to create a track that contains only
for the pilot – the target’s range, so the pilot can bearing information and ID.
immediately determine the magnitude of the threat 2. Bearing Association – A second LOB
posed by the system. measurement is associated with the LOB track
based on consistency of ID information and
As part of the development of the WA Data Fusion, we geometric feasibility. The geometric feasibility
designed and implemented a capability for RFI test verifies that if the two lines are crossed, the
Multilateration (Figure 5). resulting position represents a feasible target
location, i.e., one within the maximum range of
Step 1: Initial Line of Bearing (LOB) the emitters represented by the ID information.
measurement gives bearing with The result of the association of the two LOBs
error. LOB track is formed. gives a triangulated position with a positional
uncertainty ellipse that is typically fairly large.
This position and uncertainty are used to create a
tentative track that is not yet reported out of Data
Fusion.
3. Track Promotion – When additional LOB
measurements are detected and associated with
Step 2: Second bearing line is the tentative track, an Unscented Filter is used to
associated. Multilateration refine the position and uncertainty. When the
(triangulation) of second line of uncertainty reaches a threshold, the track is
bearing measurement
gives long narrow error ellipse,
promoted to Firm and is reported out of Data
creating tentative track. Fusion to WA or other applications using the
output of Data Fusion.
4. Continued Refinement – As additional LOB
data, or other reports, such as radar reports, are
received on the target, the position is refined
Step 3: Unscented filter further to produce the best possible Target
refines position with Location Error (TLE) from the available data.
additional LOBs, allowing
firm track to be formed
A significant amount of effort was devoted to
development and testing to demonstrate the utility and
performance of the RFI multilateration. During pilot
evaluation we were able to demonstrate that this
capability works well for a single target when the RFI
sensor is providing fine, or high resolution, LOB data.
Based on this testing the pilots specifically recognized
the particular value of the ability to quickly determine
the position of a hostile threat with the passive RFI
sensor. Further maturation is in progress to enable this

Step 4.
capability to work more effectively with less accurate
Multilateration over data and for multiple closely spaced targets so that this
time gives small capability can be integrated and evaluated in operational
TLE. testing.

Figure 5. Generation of positional tracks from Processing Environment

bearing measurements enables WA Data Fusion to Data Fusion was originally developed to run in a
combine RFI and other bearing data with FCR and multiprocessing Unix environment on Silicon Graphics
other positional sensors. processors, in which it could run continuously as its
own process, and rely on the operating system to
The RFI Multilateration process consists of four basic manage fair sharing of the CPU, and memory space
steps: protection, with respect to other processes. The
1. LOB Track Formation – A Line of Bearing VxWorks operating system used on the Apache’s
(LOB) measurement from RFI or any similar General Purpose Processors provides a very different
environment. Under VxWorks, the operating system is
the only real process. All applications, like Data Fusion, Interfacing the “host” Common Operating Environment
run as tasks within that process, and share a common (COE) software to Data Fusion was done via two
data address space. For scheduling purposes, tasks are methods – direct procedure calls and data written to
divided up into real-time tasks and background tasks. common “shared memory” segments. Figure 6
Real-time tasks are scheduled to execute in a round- illustrates these interfaces. Most data transfer occurs
robin fashion, and are expected to guarantee to finish through the shared memory interface segments that
within a small number of milliseconds. As a background include an Input segment for sensor data, an Output
task, called periodically by the WA process, Data segment for the fused trackfile data from Data Fusion, a
Fusion is expected to execute for a few milliseconds at a Status segment to share information about Data Fusion
time before returning and relinquishing the CPU, and status, and a Control segment to issue control flags,
could be preempted at any time, for any length of time, such as a request to delete a specific track.
by one or more real-time tasks.
CÓcallable functions
Data Fusion shares one of the Apache’s 375 MHz G4

Common Operating Environment (COE)

Power PC – based General Purpose Processors with a Shared
Shared Memory
Memory Pointers
Pointers

number of other tasks. As a subtask of WA, Data Fusion Initialize

Initialize // Reset
Reset
gets called to execute, on average, about 10 times per
Iterate
Iterate

WA Data Fusion
second. Each time Data Fusion executes, it is expected
to execute for no more about 8 ms before returning. Request
Request System
System Time
Time

This gives Data Fusion a nominal 8% of the total CPU

time to perform its processing. However, since Data Control Control
Control Parameters
Parameters
Fusion can be preempted at any time by a real-time task,
Status Fusion
Fusion Status
Status Fields
Fields
the actual portion of the CPU available to Data Fusion is
some unknown amount less than 8%. To adapt to this Input Sensor
Sensor Reports
Reports

execution paradigm, we needed to provide a means by Output Fused

Fused Tracks
Tracks

which Data Fusion could execute for a few

milliseconds, save an intermediate state of the Shared memory segments
processing before returning, then pick up where it left
off upon next being invoked, and internally manage Figure 6. The structure for the Data Fusion-to-WA
memory allocation. The details of this solution are interface.
described in a later section.
Data Fusion Software Architecture
Interface/Hardware To integrate LM ATL’s Data Fusion system into WA,
Integrating Data Fusion onto the Apache required two important implementation challenges were
addressing several challenges related to hardware and addressed. The first required Data Fusion to split its
software platform. The Warfighter’s Associate was able processing over a series of short time slices rather than
to leverage much of the work done under MCA to running as a continuous process. The second required
enable Data Fusion to run on Power PC-based Data Fusion to avoid the use of dynamically allocated
hardware. However, WA had to meet the requirement memory.
that Fusion must operate under the real-time operating
system VxWorks. Finally, Data Fusion – being written LM ATL’s Data Fusion system was originally
primarily in C++ – was to interface with software developed to execute as a stand-alone continuously
written in Ada. executing process, with a well defined set of functions
executed in a specified order on the input data to
The decision was made early on that it would be more produce the fused output. Integration onto the Apache
productive and less costly to do the development and required that Data Fusion run as a subtask within the
testing in a simulated VxWorks environment as multiple overall WA execution process, with a strict limitation of
instances of the simulator can be run simultaneously. eight milliseconds on the time that it could execute
We chose the Tornado™ development environment as it before returning control to the WA process. This meant
provided both a VxWorks simulator as well as a cross- that it was in general not possible to complete
compiler that can generate PPC-compatible code. We processing on any given set of sensor data during an
hosted TornadoTM within a development environment execution cycle. The solution to this problem was
that enabled us to compile, test and release software implementation of a state machine control structure.
packages in a manner commensurate with our MCA The state machine keeps track of the current state of
development. Data Fusion execution (Figure 7), and permits it to
resume execution at the appropriate point, with all
required contextual information, once Data Fusion sensors. However, once a fused class ID is determined
execution is resumed. for a track, special care needed to be taken to map the
fused class ID back into an appropriate sensor-specific
Level 1
class taxonomy, to support WA processing logic and
Fusion Processing
State 1 State 2 State 3
display symbology.

While providing new capability, we also worked to

Level 2
meet the goal of minimizing changes required to the
Retrieve Inputs Process Data Output Data
State 1 State 2 State 3
pilots’ expectations of system behavior. One example of
this was the adjustment of the age-out behavior of
unreinforced tracks to approximate that already
Level 3
experienced by current pilots. In another instance, it
Update Trackfile Predict Trackfile Fuse Data turned out that when a track was reported to be
Forward State 1 State 2 State 3 State 4 State 5
unknown, it was meaningful for the pilot to know which
particular sensor data source reported the class. To
maintain this richness of information in its output, Data
Level 4
Fusion was specifically modified so that when a track
Predict Cluster Cost Assign Fuse was established based on data from a single sensor, Data
Fusion reported the track type using the same
Figure 7. Data Fusion’s State Machine control
symbology as the reporting sensor, rather than choosing
structure allows it to maintain its state of execution
a single default symbology for representing all tracks of
over a number of short-duration execution cycles.
unknown type.
The WA processing environment does not support the As a third goal, we worked throughout this process to
routine allocation and deallocation of memory that is ensure that the fundamental correctness and accuracy
standard in an object oriented program, such as Data of the underlying core Data Fusion processing was
Fusion. To minimize changes to previously developed retained as updates to the software were made in
Data Fusion software and allow commonality between support of the other goals.
the WA and MCA Data Fusion implementations, we
implemented a custom memory manager utility that The overall development, integration and testing
allocates a single large block of memory at program process was divided into four major phases.
initialization, and then internally manages the use of this
memory to enable the object-oriented software to Initial Port of Code to VxWorks. In the WA execution
behave as it would normally. In addition to managing environment Data Fusion is called as a subtask of the
memory, the utility also provides statistical profiling for larger WA task. WA, itself, runs as a task within the
memory management to ensure that the software has no Apache’s Common Operating Environment (COE),
memory leaks and to support analyzing analyzing which provides a framework of task management
memory usage. policies and services on top of the underlying VxWorks
operating system. To enable development in our
Integration and Testing Process laboratory environment, we used the TornadoTM
LM ATL worked in collaboration with engineers from VxWorks emulation, from Wind River Systems Inc.
Boeing Phantom Works to adapt and integrate Data Running on our normal Solaris development platforms,
Fusion to run as part of WA within the Apache mission the VxWorks emulation allowed us to initially port Data
systems hardware and software environment. Fusion so that it could be compiled and run in either
VxWorks or Solaris. This allowed us to identify and
Throughout the development and integration process, resolve issues arising from differences between the two
effort was made to strike a balance between three operating systems. Most of these differences had to do
overarching goals. with differences in the interface to the operating system
as represented in the set of system utility functions, (for
As our primary goal, we worked to ensure that Data example, the time function), available as C++ function
Fusion provides the functionality needed to support WA calls. With these issues resolved, most of our
requirements. As an example, when a sensor reports the subsequent development and testing could be done
class of a target to Data Fusion, the class ID is mapped under Solaris, with a reasonable level of confidence that
from the sensor-specific class reporting taxonomy to a it would run identically under VxWorks.
more comprehensive class taxonomy that Data Fusion
uses for proper fusion of class IDs from multiple
Integration with a WA Sub: While Boeing was function, as a part of WA, in operational scenarios with
developing the overall WA software functionality, they a pilot (or engineer) flying the Apache through the
were able to provide a stub version that allowed us to simulated scenario, and operating the sensors in a full
test Data Fusion’s interfaces with WA, and the mock-up of the Apache cockpit. At times, the pilot was
execution of Data Fusion within the WA and COE put into a scenario and given the ability to “play” in a
calling context. Boeing was able to compile into the relatively free-form manner to uncover operational
WA stub program a number of test scenarios that issues that were unanticipated by the engineers.
provided Data Fusion with input data, and consumed the However, as testing proceeded, a fairly comprehensive
Data Fusion output. These scenarios provided a checklist of operational tasks was developed to test and
thorough and detailed exercise of the Data Fusion verify the proper operational function of Data Fusion as
functionality. The ability to create a track of each class, a part of the WA system. For example, a part of this
from each available sensor source was exercised, as routine checklist would call for the pilot to scan for a
well as the ability to fuse reports from multiple sensors, target with the FCR and, having acquired the target with
track moving targets, and manage the ageing and loss of the FCR, the pilot would use the Target Acquisition and
tracks. Logs were collected of the data passing over Designation System (TADS) to find and designate the
both the input and output interfaces between Data target to store its location. While the pilot verified and
Fusion and WA, and the contents of the logs were evaluated the expected behavior of the system
analyzed to verify the correctness of the interface throughout these tasks, LM ATL and Boeing engineers
function. would verify that the FCR track appeared in the correct
geographical location, with the appropriate class, that
Integration with WA in Boeing Labs. Once the WA the TADS store properly fused with the FCR track, and
software was ready, we began integration testing of that the track symbol on the pilot’s display properly
Data Fusion with the full-fledged WA software within reflected the update to the track. The pilots and PVI
the actual target hardware and software environment in engineers provided valuable feedback that led to a
the Boeing integration labs. During this period Data number of operational improvements to Data Fusion
Fusion was tested on a small number of specific and WA behaviors, such as the timing and display of
scenarios provided through Boeing’s simulation tracks going stale, the expected error for certain sources
capability. This most lengthy phase of the integration of track data, and the interaction of moving FCR reports
process was characterized by an iterative cycle of with stationary pre-planned targets.
development and testing. Running in the Boeing lab,
Data Fusion would be observed while simulating a Results
scenario, slightly modified to target a specific aspect of Through this testing process, we were able to validate
the Data Fusion functionality, and bugs or desired that Data Fusion addressed all required functionality
modifications would be noted. Data Fusion and performance to support the WA functionality. More
modifications would then be developed and tested at importantly, we were able to get feedback from Apache
LM ATL, running under the Solaris operating system. pilots involved in the tests that Data Fusion was a useful
The compiled image of the modified Data Fusion could capability that should be brought forward into
then be sent by email to Boeing for targeted retesting operational use in the Apache and other helicopters.
and verification in the simulation lab. While many of There were several features of Data Fusion that pilots
these test-and-develop cycles could be carried out via specifically identified as valuable:
email interactions between LM ATL and Boeing • The retention of FCR-generated tracks across
engineers, there were also periodic visits by LM ATL multiple scans
engineers to the Boeing labs, for face-to-face interaction • The ability of Data Fusion to take reports on a
with the Boeing engineers, and firsthand observation of single target from multiple sources and combine
Data Fusion operation. With this iterative cycle of them into a single track, with a single
discovery and resolution, we were able to resolve the corresponding symbol on the display. That is, they
majority of the software interaction and operator appreciated the use of Data Fusion to reduce “data
usability issues. confusion.”
• The RFI Multilateration capability was described as
Pilot-Driven Operational Focus: Finally, Boeing’s “phenomenal” and something that would be very
Pilot-Vehicle Interface (PVI) engineers, as well as pilots valuable in tactical situations once it is matured to
from both Boeing and the Army, were brought into the an operational capability.
Having undergone thorough testing and evaluation in
iterative test-and-fix cycle. Running Boeing’s the Boeing simulation labs, we look forward to ground
Engineering Development Simulation (EDS) lab, LM
ATL and Boeing engineers could observe Data Fusion
and flight testing in the actual aircraft in the second (SBC) running a Real-Time Linux implementation from
quarter of 2005. TimeSys Corporation. All data to and from the Data
Fusion system are transmitted over CORBA Event
Other AMUST-D/HSKT work Channels using multicast protocols. Despite these
differences, MCA Data Fusion and WA Data Fusion
LM ATL has developed a number of other components
share a common software base, with much of the
to support the AMUST-D and HSKT programs. These
differences between the two versions accomplished via
include a Data Fusion component to support the Mobile
run-time configuration using an XML configuration file.
Commander’s Associate (MCA) decision aiding system
on the A2C2X Blackhawk aircraft. We have provided
Agent-Based Data Discovery
an Agent-Based Data Discovery (ABDD) component
[5] to retrieve and monitor information from the Army ABDD is a component within the MCA that is
Battle Command System (ABCS) C2 applications responsible for controlling the flow of data between the
installed on the A2C2X aircraft. In addition, we data sources within the ABCS onboard the A2C2X and
developed support for and integrated the PC Improved the rest of MCA using LM ATL’s Extensible Mobile
Data Modem (PCIDM) product that uses the Agent Architecture (EMAA). EMAA agents are tasked
SINCGARS radios on the A2C2X to transfer data to retrieve data from the JCDB and the Live Feed Server
between the A2C2X and the Apache. The combination (LFS) within ABCS. The JCDB contains information on
of these components with the WA Data Fusion system blue/red unit data and control measures, while the Live
enables a Shared Situational Awareness capability that Feed Server contains real-time data on friendly force
spans the multiple aircraft participating in the HSKT vehicles. The constant blue track feed from the LFS
experiments. In the remainder of this section, we briefly allows the agents to monitor their positions with respect
describe each of these components. to a set of control measures available from the JCDB.
These control measures include: Named Area of Interest
MCA Data Fusion (NAI), Engagement Areas (EA), Phase Lines (PL) and
Helicopter Routes. By monitoring these control
MCA Data Fusion provides the companion capability to
measures, the agents are able to warn the commander
WA Fusion as part of the MCA decision-aiding system
when blue forces may be vulnerable if they cross a
on the A2C2X. The underlying functionality and
phase line too soon, cross into a potential engagement
software are the same, but because the data available on
area or are near a blue force helicopter route. The agents
the A2C2X is very different from that on the Apache,
can also trigger critical decisions when a red force is
the behavior is somewhat different. The A2C2X has no
found in a named area of interest and allow the
onboard sensors, so MCA Data Fusion is responsible for
commander to send an attack to a specified engagement
combining sensor data from JSTARS and other sources
area.
over Link-16, Friendly position reports received through
the Force XXI Battle Command Brigade and Below
PCIDM Message Service
(FBCB2) system, spot reports and those collected into
the Joint Common Data Base (JCDB), and Operator- At the inception of the AMUST-D program, it became
entered tracks based on video from the UAV. A number clear that a mechanism was needed to enable transfer of
of specific behavior logic requirements were Situational Awareness and other data between the MCA
incorporated to accommodate the needs of processing and WA aircraft, and LM ATL was tasked to provide
data from Link-16 and the UAV. For example, the this mechanism. Enabling data transfer required both a
commander using MCA may request that a strike physical medium, i.e., a set of radios, and a data transfer
aircraft, such as an F/A-18, engage a target. While the medium, i.e., a set of protocols and terminal or modem
engagement is in progress, Data Fusion must ensure that hardware to encode the data for transfer over the radio
it does not drop the target from its trackfile even if no links. Because the Apache had a previously existing
data has been received on the target for several minutes. capability to transfer data using the Improved Data
Another difference is that MCA Data Fusion has a Modem (IDM) over its SINCGARS radio sets, we
requirement to deal with situations in which there may determined that the best approach was to enable the
be as many as 2000 friendly and threat entities, and be MCA A2C2X aircraft to use this same mechanism. We
capable of adapting its output rate and processing to identified a commercially available product, the PCIDM
ensure that it deals as effectively as possible with the from Innovative Concepts Inc., as the best route to
very large number of entities. accomplish this. The PCIDM is a PCMCIA form factor
modem that provides the same messaging capability as
Another difference between MCA and WA Data Fusion the IDM on the Apache. Using this product, we
is in the processing environment. Data Fusion for the developed a PCIDM Message Server software
MCA occurs on a PowerPC Single Board Computer component that enables the MCA software on the
 A2C2X to exchange data with the Apache using the Hess, and many others at Lockheed Martin Systems
Boeing-developed AFAPD message set. Under Integration – Owego in Owego, NY, developed the
AMUST-D, this capability is used by MCA to receive MCA system and worked closely with us in the
FCR target and Apache position data from the Apache, development and integration of the MCA Data Fusion,
and to send other sensor data, target handover requests, ABDD, and PCIDM Message Service components.
and routes to the Apache from the MCA system. Most of all, we would like to acknowledge the
longstanding and tireless leadership, vision, and
Conclusions inspiration of Angela Pawlowski of LM ATL in making
this work possible.
Through the work designing, developing, integrating
and testing Data Fusion for the Warfighter’s Associate
under AMUST-D and HSKT, we have shown that References
useful Data Fusion functionality can be performed in the [1] LTC John C. Wright and Kristopher Kuck, “U.S.
processing environment of the Apache Longbow and Army Modernization: Eyes on the Target,” Rotor
with the sensor data sets available on the Apache. Our and Wing, April 2001.
testing has shown that this Data Fusion functionality is [2] Don Malkoff and Angela Pawlowski, “RPA Data
useful not only to the other decision aiding functions Fusion,” 9th National Symposium on Sensor
that make up the WA system, but to the pilot as well. Fusion, Vol.1, Infrared Information Analysis
We expect to continue to mature this capability to Center, September 1996.
support an eventual operational implementation. [3] Steve M. Jameson and Craig Stoneking, “Army
Aviation Situational Awareness Through Intelligent
Acknowledgments Agent-Based Discovery, Propagation, and Fusion
of Information," American Helicopter Society
The work described in this paper was performed under
Forum 58th, Avionics and Systems Session,
contract DAAH10-01-2-0008 from the US Army
Montreal, Canada, June 11-13, 2002.
Aviation Applied Technology Directorate (AATD). We
[4] William J. Farrell, Steve Jameson, and Craig
would like to acknowledge the support of Keith Arthur,
Stoneking, “Shared Situation Awareness for Army
Balinda Moreland, Mark Ennis, Ray Higgins, and Tim
Applications,” Proceedings of 2003 National
Condon of AATD in this work. Many other people from
Symposium on Sensor and Data Fusion, San Diego,
Lockheed Martin and Boeing supported this effort. Jim
CA, June 2003.
Farrell of LM ATL developed the RFI Multilateration
[5] Peter Gerken, Steve Jameson, Brian Sidharta, and
and IMM Kalman Filter algorithms that were used in
Joyce Barton, “Improving Army Aviation
WA Data Fusion. Alex Artigas, Karl Bielefeldt, Bob
Situational Awareness with Agent-Based Data
Huber, Bob Faerber, Jeff George, and others at Boeing
Discovery,” Presented at the meeting of the
Phantom Works in Mesa, AZ, developed the WA
American Helicopter Society Forum 59, Phoenix,
system and worked closely with us in development,
AZ, May, 2003.
integration, and testing of the WA Data Fusion system.
Chris Bodenhorn, John Kiernan, Peter Stiles, Janice

View publication stats