May 1999



Ronald M. Yannone

Sanders, A Lockheed Martin Company

95 Canal Street

Nashua, NH  03060





A standard model for data fusion has been developed by the U.S. DOD Joint Directors of Laboratories/Data Fusion Subpanel (JDL/DFS).  This panel was established in 1986 as a subpanel to the JDL Technical Panel for C3.  The five levels of fusion are Sub-Object Data Association and Estimation: pixel/signal level data association and characterization at the sensor level (L0), Object Refinement (L1), Situation Refinement (L2), Significance Estimation or Threat Refinement (L3) and Process Refinement: adaptive search and processing - resource management (L4).  The next-generation aircraft will be a multirole strike aircraft weapon system for the Navy, Air Force, Marines and U.S. allies and will encompass all five levels of fusion.  We explore the viable fusion architectures and algorithms in the context of the JDL/DFS definitions that will be required for the aircraft to be successful.


1.0  Introduction


Next-generation fighter aircraft must satisfy stringent mission goals and maximize crew survivability against threat weapon systems that are constantly increasing in their ability to detect, track and fire upon their foe.  The fighter will inherently be multirole, support the requirements and missions for several armed services, be economically affordable, have high reliability, be low observable (LO) and rely heavily upon offboard assets.


The fighter will be part of a “system-of-systems” where every piece provides a critical link in the “information” chain.”  With affordability as the linchpin, and mission success and survivability as requirements; a “compromise” is required as illustrated in Figure 1.




Figure 1. There is a Compromise in the Tug-of-War between Aircraft Affordability and Mission Success & Crew Survivability.



The paper outline is as follows:


·         The Five Levels of Data Fusion

·         Concept of Operations

·         Typical Sensors

·         Typical Countermeasures

·         Mapping the Tasks to Fusion Levels & Algorithms

·         Issues Regarding CM Response

·         Viable Fusion Architectures

·         Algorithm Considerations

·         References


2.0  The Five Levels of Data Fusion


In a paper by Franklin E. White (Space and Naval Warfare Systems Center, San Diego) titled “Managing Data Fusion Systems in Joint Coalition Warfare,” a functional model for data fusion was presented as a common standard for multisensor practitioners to use.  The model proposes five (5) recognizable functional levels as summarized in Table 1.


Table 1.  The Five Levels of Data Fusion.



Fusion Layer & Definition







Sub-Object Data Association and Estimation


Pixel/signal level data association and characterization




Object Refinement

Observation-to-track association, continuous state estimation (e.g., kinematics) and discrete state estimation (e.g., target type and ID) and prediction




Situation Refinement

Object clustering and relational analysis, to include force structure and cross force relations (e.g., an enemy’s order of battle), communications, physical context, etc.





Consequence prediction, susceptibility and vulnerability assessment


Process Refinement

Adaptive search and processing (an element of resource management)


3.0  Concept Of Operations


The Concept of Operations (CONOPS) for next-generation fighter aircraft of the 21st century help us understand the different information required by the crew to survive the treacherous arena they operate in.  Around 23 nations are expected to have advanced surface-to-air missiles (SAMs) in 2005 and around 20 nations are expected to have advanced air-to-air missiles (AAMs) in 2005.  The electronic warfare (EW) tasks for a typical “fighter sweep mission” are shown in Table 2.


Table 2.  Typical EW Tasks for Fighter Sweep Mission.


Mission Phase

EW Tasks



·         Database Management (EOB, Threat and Tactics Tables)

·         Prioritization and Tailoring

·         Expendables Configuration


Take off, Climb, Subsonic Cruise

·         Activation

·         BIT/Status

·         Observables Management

·         Gain Situation Awareness



Supercruise, Fence Check, MEZ Ingress

·         Observables Management

·         Increased Situation Awareness

·         Locate, track, ID, prioritize targets (support targeting)

·         Provide/accept cueing

·         Locate, track, ID, prioritize threats/friendly defenses

·         Avoid/counter SAMs/threats

·         Support route management

·         BIT/Status

·         Autonomous/cooperative





·         Observables Management

·         Support targeting - provide quality track data

·         Increased Situation Awareness/Kill Assessment

·         Provide/accept cueing

·         Avoid/counter threats

·         BIT/Status

·         Autonomous/Cooperative



Disengage, MEZ Egress

·         Observables Management

·         Maintain Situation Awareness

·         Avoid/counter threats

·         Support Route Management

·         BIT/Status

·         Autonomous/Cooperative



Subsonic RTB, Landing

·         Maintain Situation Awareness

·         Observables Management

·         Avoid/counter threats

·         BIT /Status

·         Provide updated EOB

·         Support Integrated Diagnostics


4.0  Typical Sensors


The cost, weight and power constraints for the fighter limit the number of sensors and countermeasures.  Sensors that contribute strongly to the fighter’s survivability and mission success are given in Table 3.


The fire control radar must have strong air-to-ground (surface-moving-target-track [SMTT]) and air-to-air modes, as well as single-target-track (STT) and track-while-scan (TWS) modes and an ATR mode.  The IRST will require STT and TWS modes.  The IRW will require threat missile classification algorithms for AAMs and SAMs, with possible ranging algorithms using the intensity measurements, atmospheric data and stored radiant missile intensity database.  The IRST aids in raid assessment, in conjunction with the fire control radar (when emissions are permitted).


The RWR will need to provide high fidelity RF emitter mode and ID capability.  Offboard sources will include all available sources: low observable (LO) assets (F-22, B-2, F-117) and non-LO assets (F-15E) and air surveillance and reconnaissance support (E-3 AWACS, E-8 JSTARS, RC-135 RIVET JOINT, UAVs and command and control equipment).


Table 3.  Sensors that Strongly Contribute to Fighter Survivability and Mission Success.



Primary Parameters

Secondary Parameters


range, range rate, TTG and ATR

azimuth, elevation, coarse threat class/ID


intensity, azimuth, elevation

range with ownship maneuver and threat weapon (AAM, SAM) release confirmation




intensity data, detection of threat missile (AAM/SAM) launch, possible threat class/ID, azimuth and elevation


slant range estimate for SAMs, coarse threat class/ID, coarse range for AAMs


RF emitter ID and mode

azimuth, elevation, coarse AI range and slant range to SAM


threat optical systems, azimuth and elevation

slant range to ground site optical system


threat IR systems, azimuth and elevation

slant range to ground site IR system


range, range rate, TTG

azimuth, elevation, coarse threat class/ID via




specific SAM and AI class/ID, and locations of friendly craft, threat updates and weather


threat AI location, speed and heading at a point in time as time transpires



targeting imagery, threat class/ID, bomb damage indication (BDI)


azimuth and elevation

Pre-Mission Planning Data

Preferred route(s), CM response(s) to specific threats, anticipated threat(s), EOB



5.0  Typical Countermeasures


There are a host of countermeasures (CMs) available for the crew to utilize, as indicated in
Table 4.


Table 4.  The Crew has an Array of CMs to Select.








·         Applicable against threat AI/SAM/AAA emitters

·         May be towed, onboard or expendable decoy/with coordinated host vehicle maneuver

·         Utilize cooperative CM’s with manned or unmanned friendly vehicle

·         Utilize unmanned vehicle (UAV) and/or unmanned fighter “equivalent”

·         Incorporate RF stealth management



·         Expendable decoy coupled with host vehicle maneuver

·         Incorporate IR stealth management


·         Counter optical trackers


·         Counter IR trackers

Susceptibility Reduction

·         Incorporate total low observability (LO) posture (RF/EO/IR/Visual/Acoustic)

Onboard weapon(s)

·         Use offensive posture with guns, HARMs, AAMs


Cooperative Offensive

·         Use wingman, UAV and/or unmanned fighter “equivalent” to take an offensive posture in a coordinated, or as a stand-alone, offensive asset


Cooperative Defensive

·         Use wingman, UAV and/or unmanned fighter “equivalent” to take a defensive posture in a coordinated, or as a stand-alone, defensive asset


6.0  Mapping the Tasks to Fusion Levels & Algorithms


As we look at some of the typical tasks during the scenario, we can begin to map them into the five levels of data fusion and the general algorithm(s) to consider as shown in Table 5.


Table 5.  Task, Applicable Fusion Level and Algorithm Considerations.



Fusion Level

Algorithm Considerations


Detect threats (IRW)



·         3D Image processing using time, space and multiple IR bands together [See Ref. 1]

Locate threats (All sensors) - associate detections over space and time



·         Nearest neighbor, Viterbi, Multiple Hypothesis Tracking association

Estimate range passively for SAMs (RWR, IRW, OAEO, OAIR, a priori cued IRST)



·         Through use of fighter altitude and elevation data - accuracy improves with time as fighter moves

Estimate range passively for threat AIs (RWR)



·         Through knowledge of host fighter’s RCS, threat emitter ID and mode transition

Estimate range passively for threat AAMs (IRW)



·         Through the use of IRW irradiant intensity, threat class/ID and known threat radiant intensity


Estimate range passively for AI threats (IRST)



·         Through the use Kalman filtering using fighter’s INS data as it maneuvers and an assumed constant velocity and heading threat AI model


Estimate range passively for threat AIs (Offboard data)




·         Through the propagation of initial offboard reports and knowledge of fighter’s relative speed, heading and elapsed time (gets stale with time)

·         Through AI emitter ID & mode switching




Estimate AAM class/ID





·         Through the use of likely AAM class/ID and corresponding range/velocity profiling

·         Through table lookup of likely AAM that go with corresponding AI emitter and inferred AI platform

·         Through the use of FLIR processing

·         Fuse disparate threat class/ID and confidence information using the Dempster-Shafer algorithm  [See Ref.4,pp. 297-298]


Netting a group of threats as a single “entity”



·         Clustering analysis to link various elements of a weapon system or groups of weapon systems to assess a force picture

Link together the various components of the threat weapon system



·         Rule-based fusion that uses the threat database to connect the various emitters detected that are working together to form a “weapon system”


Table 5.  Task, Applicable Fusion Level and Algorithm Considerations (Cont).



Fusion Level

Algorithm Considerations



Assess threat intent




·         Monitor threat RF emitter mode transitions

·         Monitor missile inertial LOS rate

·         Utilize offboard reports

·         Monitor optical and IR sensor use

·         Detect threat LRF ping(s)

·         Detect LSAH or LRR guidance signals



Assess lethality based on threat class/ID




·         Use table lookup for threat “effectiveness envelope” based on slant range estimate, vehicle heading, altitude and speed, and predetermined number of shots the threat can get off during the anticipated exposure time


Estimate TTI for missile threats



·         Utilize slant range, threat class/ID and velocity profile, and host vehicle speed, altitude and heading




Estimate CM effectiveness

(based on controlling/directing sensors)






·         Monitor LOS rate of inbound missiles

·         Monitor RF mode reversals of AI/SAM RF emitters due to RFCM

·         Monitor range rate “drop-off” for missile

·         Monitor EO/IR retroreflection after EOCM/IRCM application

·         Utilize offboard assessment reports

·         Utilize real-time FLIR imagery

·         Monitor elapsed time since CM applied


Assign priority value to each threat



·         Utilize a weighting function threat class/ID confidence value, intent, lethality, TTI (imminence) and CM effectiveness feedback

Apply CM to threat(s)


·         Based on a complex set of factors, assign CM assets [see Table 6]

Provide bomb damage indication (BDI) and offensive weapon (AAM) effectiveness



·         Control/analyze sensor data regarding threat emissions that have been removed and imagery to confirm bomb or weapon effectiveness


7.0  Issues Regarding CM Response


Table 6 captures some of the issues that the need to be handled by the countermeasure response management function.


Table 6.  Issues that affect Dynamic Optimization of CM Responses.





There are N threat classes (e.g., RF, IR,EO , MMW, Laser [AAA, AAM, SAMs])


There are M countermeasures (e.g., LO, IR/EO/RF CMs, weapons, maneuvers)


Each threat is assigned a priority between 0 and 1


For some threats, one (or more) CMs may be preferred over another (others)


Some threats can be countered by more than one CM


For some threats, one CMs preferred over another due to its capability of addressing the threat faster


Some CMs require time-to-intercept (TTI) of the threat to be greater than k1 seconds to be useful and require that it be invoked for at least k2 seconds to be effective


Some CMs can be reallocated if it is assessed to be effective


Some CMs, once invoked, are irreversible (e.g., flares, chaff, decoys)


Some CMs cannot be invoked if another CM has been deployed for more than k3 seconds


Some CMs can ONLY address one threat at a time


Some CMs can address multiple threats simultaneously


Each CM requires a minimum time to deploy it (i.e., to where it is addressing the threat)



2 or more threats of the same (or differing) class may be launched at the host vehicle that have the same (or differing) TTI values.  [Threats of the same class may be fired at different launch ranges which attributes to their different TTI values, or of differing classes fired at the same range]


For some threats, its guidance can be disrupted if the host vehicle takes offensive action, firing a weapon to the person guiding the weapon



In order for some CMs to be effective, a coordinated vehicle maneuver is required (e.g., in case of expendables where the CM is ejected from the vehicle or when the crew wants to run "silent”)


A CM is not required if the host vehicle can place itself behind an adequate “obstacle”


Some threats may have 2 or more CMs simultaneously applied against it


It is possible that an inappropriate CM has been applied against a threat (e.g., due to the fact that the threat was not classified properly)


Some threats may be avoided if one or more of their “targeting” sensors is detected prior to an actual weapon firing


When the specific ID (sub-class identification) of a threat can be discerned, then a more specific CM can be used that may be effective more quickly


There will be times when the crew has to focus in on executing the mission goal and time-on-target, in addition to the impending threat situation


There will be times when a wingman (wingmen) may provide CM coverage for the host vehicle


There are times when a simple vehicle maneuver will suffice, as to stay out of the threat’s weapon envelope


8.0  Viable Fusion Architectures


From a high functional view, the fighter closed-loop data processing architecture is as shown in Figure 2.  The data processing:


·         enhances information of threat/target kinematic and attribute information by fusing onboard and offboard multispectral data into a consolidated, unambiguous “picture” for use by the crew and situation assessment


·         supplies critical beyond-visual-range (BVR) targeting, threat class/ID and range parameters to the offensive function and route planner


·         prioritizes threats based on its class/ID, intent, lethality, time window of vulnerability, TTI, and CM effectiveness


·         schedules/requests onboard and offboard (e.g., UAV, unmanned fighters) assets to reduce threat priority (i.e., its “risk”) subject to real-time mission constraints


·         provides a “coasting” mechanism when GPS data is unavailable




Figure 2.  High-Level Closed-Loop Data Processing Architecture.


Expanding the three data processing functions of Fusion, Situation Assessment and Resource/Response Management, we see further into the details required as shown in Figures 3 through 5, respectively.

Figure 3.  Fusion provides Threat Kinematic & Attribute Data for Several Users.



Figure 4.  Situation Assessment Determines Valuable Pieces of Information.





Figure 5.  Resource/Response Management Schedules Sensors and Countermeasures to Support Offensive and Defensive Mission Requirements.

9.0  Algorithm Considerations


This section contains descriptions of some algorithms that may spark some interest and research.  The references are cited and provided in the Reference section.


IRW Signal Processing Improvements.  It is desired to detect IR SAM threats at their maximum launch ranges.  Typically the IRW is limited by the presence of heavy background clutter, solar glints, and sensor noise which lower the ability to detect these missiles.  The heavy background clutter may also cause non-missile objects such as flares, glints, and smokestacks to be incorrectly declared as missiles.  The longer detection range of missiles by these sensors is also limited by sensor noise, most noticeably in tropical weather conditions.  Atlantic Aerospace and USAF Wright Laboratory have demonstrated two robust algorithms: a Geometric Whitening Filter which enhances the signal-to-clutter ratio and a Morphological Track Before Detect algorithm which enhances signal-to-noise ratio.  Use of these two algorithms in tandem will extend current Advanced Development IRW prototype sensors to detect IR-guided SAMs in heavy urban clutter and tropical maritime weather conditions.  See
Reference 2.


Track Initiation and Data Association in Jamming and Low-RCS Target EnvironmentsConventional target association and tracking techniques such as PDA and JPDA have very fine performance when the measurement acquired from sensors are perfect.  However, when jamming and stealth techniques are widely used, it is very difficult for sensors to gain perfect measurements.  Though a single sensor in a distributed sensors system might fail to acquire continually perfect measurements of low-RCS (stealth) targets under jamming environments, the distributed sensors system might gain relatively perfect measurements by integrating measurement hits or fractional trajectories of targets from every sensor in the system.  See Reference 3.


Model for Integrated Sensor/Response Management.  The utility of information can be evaluated on the basis of its contribution to system mission goals.  Key factors in planning and executing any practical mission involve the unavoidable problems of situational uncertainty, contentions for finite system assets, and unexpected side effects of system actions.


Assuming perfect knowledge of current and future world states, a system could define a schedule of actions defined that would be optimal in terms of maximizing a mission objective function, give n the system’s available repertoire of actions.


Unfortunately, real-world systems must generate and maintain action plans based on the error-prone estimates provided by realistic sensors and associated processing and control, together with erroneous, incomplete, and uncertain a priori knowledge.  See Reference 4.


The goal of information acquisition in a system responding to its environment, then, is to provide resolution of that environment sufficient to support response decisions.  Moore and Whinston model the information acquisition problem as that of achieving a partition among possible world states such that the final partition corresponds to exactly one member of the system’s repertoire of responses (i.e., effecting the selection of a specific response action).  [See Reference 5]  Referring to Table 6, we can see that there is a challenge when it comes to allocating countermeasure resources due to the “interjection” of high-priority threats that can cause near-term “hind-sight regret” situations of resources committed.  Additional references can be found in Volume I of the “Proceedings of the 9th National Symposium on Sensor Fusion,” 12-14 March 1996, pp. 331-413.


Benchmark for Radar Allocation and Tracking in ECM.  A benchmark problem for tracking maneuvering targets is desired.  The benchmark problem involves beam pointing control of a phased array (i.e., agile beam) radar against highly maneuvering targets in the presence of false alarms and electronic countermeasures (ECM).  The testbed simulation described includes the effects of target amplitude fluctuations, beamshape, missed detections, false alarms, finite resolution, target maneuvers and track loss.  Multiple waveforms are included in the benchmark so that the radar energy can be coordinated with the tracking algorithm.  The ECM includes a standoff jammer (SOJ) broadcasting wideband noise and targets attempting range gate pull-off (RGPO).  The “best” tracking algorithm is the one that minimizes a weighted average of the radar energy and radar time, while satisfying a constraint of 4% on the maximum number of lost tracks.  See Reference 6.


Other Algorithm Ideas.  Some other ideas that should be kept in mind or consider are summarized in Table 7.


Table 7.  Other Algorithm Ideas to Keep in Mind or Consider.



Algorithm Idea



The RWR will detect AI RF emitters, but not every AI will necessarily radiate.  The IRST can provide raid count, and it might pay to consider the threat “cluster” rather than try to develop algorithms that struggle to match the RWR reports with the IRST reports





To passively estimate range to threat AI platforms, certain things are required:  (a) the threat aircraft is assumed to be traveling with constant velocity, at a constant heading course, (b) the host aircraft has to traverse a base leg with induced maneuvers to obtain observability from a state estimation viewpoint, (c) proper state vector initialization is required to maintain Kalman filter stability, and (d) this takes time (typically 30-60 seconds and depends on the scenario).  The use of offboard data can bypass the convergence-to-solution time and expedite threat avoidance, develop an offensive posture or select a countermeasure strategy



When one onboard sensor detects a threat, this knowledge can assist other sensors by possibly permitting the use of lowered threshold settings in the sensor’s signal processor.  This aids in threat/target confirmation and supports beyond-visual-range identification (BVRID)



UAVs will play a vital support role to the next-generation fighter.  UAVs can be equipped with RWRs to identify and localize hostile fire control radars.  This data can be down-linked to a mission control link and in turn to the fighter.  Furthermore, the UAV can be equipped with a towed decoy system and on-board jammers to enhance aircraft survivability



The LO features of the aircraft may need to be examined from the point of view that even though the aircraft may be within detection range of the radar(s), its LO cross-section may deny detection and the crew can exploit, or lean on, this fact to progress with the mission rather than abort or have to execute evasive actions


10.  References


1.        Real, E.C., Yannone, R.M. and Tufts, D.W., “Comparison of Two Methods for Multispectral 3D Detection of Single Pixel Features in Strongly Textured Clutter,” Image and Multidimensional Digital Signal Processing’98 Workshop, Austria, July 1998.

2.        Peli, T., Pauli, Myron., Monsen, P., McCamey, K. and Stahl, R., “Signal Processing Improvements for Infrared Missile Warning Receivers,” IEEE 1997, pp. 1052-1062.

3.        Hongwei, C., Longbin, M., Zhongkang, Z., “A Joint Probabilistic Data Association Algorithm for Distributed Multisensor System in Jamming and Low-RCS Target Environments,” IEEE 1997, pp. 1002-1008.

4.        Robinson, S., “The Infrared & Electro-Optical Systems Handbook,” Volume 8-Emerging Systems and Technology, Section 3.8.1, pp. 314-316.

5.        Moore, J., Whinston, A., “A Model of Decision-Making with Sequential Information Acquisition.” Decision Support Systems Volume 2, pp. 285-307, (1986); and Decision Support Systems Volume 3, pp. 47-72, (1987).

6.      Blair, W., Kirubarajan, T., Bar-Shalom, Y., “Benchmark for Radar Allocation and Tracking in ECM,” IEEE Transactions on Aerospace and Electronic Systems, pp. 1097-1114, October 1998.  A companion paper is in the same reference, pp. 1115-1131.