Revisiting the JDL Data Fusion Model II

1. Revisiting the JDL Data Fusion Model II James Llinasa, Christopher Bowmanb, Galina Rogovac, Alan Steinbergd, Ed Waltze , and Frank Whitef a: Research Professor, University at Buffalo. Buffalo, NY, USA, llinas@eng.buffalo.edu b: Consultant, Data Fusion & Neural Networks, Colorado, USA, cbowman@indra.com c: Encompass Consulting, Honeoye Falls, NY, USA, rogova@rochester.rr.com d: Technical Director, Utah State University Space Dynamics Lab, Utah, USA, Alan.Steinberg@sdl.usu.edu e: Technical Director, Intelligence Programs, General Dynamics - Advanced Information Systems, Ann Arbor, MI, USA, ed.waltz@gd-ais.com f: Director, Program Development, US Navy SPAWAR Systems Center, San Diego, CA, USA, frank.white@navy.mil

2. Some History of Fusion Models • JDL—Original, circa 1987 • Dasarathy, Data-Feature-Decision Layered Model—1997 • Steinberg, Bowman, and White, Revision I to JDL, 1999 • Bedworth and O’Brien, Omnibus Model, 1999-2000 • Salerno, Situation Awareness Model, 2002 • Blasch and Plano, Level 5, 2003

3. The Reference JDL Model* * Steinberg, A.N., Bowman, C.L., and White, F.E., “Revisions to the JDL Data Fusion Model”, in Sensor Fusion: Architectures, Algorithms, and Applications, Proceedings of the SPIE, Vol. 3719, 1999

4. Motivations for Revisiting the JDL Model(s) “External” Factors (Driven by Opnl Needs) “Internal” Factors (Driven by Need for Deeper Understanding) • “Common (or Consistent, or Relevant or Single Integrated or User Defined) Operational Picture” • “Network-Centric Warfare” • “Dominant Battlespace Knowledge” • “Operations Other Than War” • “Asymmetric Warfare” • “Information Warfare” • “FORCEnet” Distributed, Service-Based Information Architectures Better understanding of the “Levels” Dynamically-Composable Data and Information Fusion Services Insight into Inter-Level Processing --Information operations --Adjudication and conflict resolution --Output management --Effects of Input Reliability Pedigree, Metadata, Context Services Conventions and Standards Ontologies Integrating Inductive/Abductive Inferencing Underlying implications for the primary conceptual and semantic DF model: The JDL Model

5. Reexamining our understanding • of the “Levels” 4) Aspects of Distributed Fusion 2) Insight into Inter-Level Processing --Information operations --Adjudication and conflict resolution --Output management --Effects of Input Reliability 5) Pedigree 6) Ontologically-based Data Fusion Processes 3) Integrating Inductive/Abductive Inferencing Discussion Topics

6. 1) Revisiting the “Levels” • For Alan to do—some bullets on new perspectives re Levels

7. 1) Revisiting the “Levels” cont’d

8. 2) Insight into Inter-Level Processing(a) Information operations • The idea of inter-Level information and control flow is not very explicit in the traditional JDL Model • Need to specify inter-Level “informing”, controlling, and exploitation • Trades off added value/utility vs cost of additional processing; raises need for consistency

9. Data Fusion Tree Node

10. Level 0 Level 1 Level 2 Level 3 Operations Within/Across Levels State Estimation Bias Inter-Level Considerations Level 4

11. 2) Insight into Inter-Level Processing(b) Adjudication and conflict resolution • Both Atomic Level and Meta-Level Adjudication

12. 2) Insight into Inter-Level Processing(c) Output management • JDL Model not specific in how output Quality & Consistency are controlled • Expect hierarchical Value system; within-process and system-level New State Estimate Quality and Consistency achieved as per operations in 2(a),2(b) Output Inferencing Quality Control via Addtl info using L4 Output Inferencing Consistency via Belief Change

13. Possible situations (Dubois and Prade, 1992) • It is possible to assign a numerical degree of reliability to each source. • A subset of sources is reliable but we do not know which one. • Reliabilities of the sources can be ordered but no precise reliability values are known. • Strategies to be considered: • Strategies for identifying the quality of data input to fusion processes and elimination of data of poor reliability. • Strategies for modifying the data and information by considering their reliability before fusion. • Strategies for modifying the fusion process to account for the reliability of the input. • Combination of strategies mentioned above. - is a context dependent operator, which depends on the strategy selected and is defined within the framework used for uncertainty representation 2) Insight into Inter-Level Processing(d) Effects of Input Reliability • Reliability akin to second-order Uncertainty (in source inputs) • Typically not accounted for in fusion algorithms • Even if Source Reliability specified, how to compute Fused-Estimate Reliability?

14. Discovery • Data mining tools to locate • patterns of meaningful relationships • Correlated patterns are examined • for relevance • Abductive Phase • Generalization & Validation • Applies inductive generalization • Model parameters are estimated • Detection • Validated model provides a target • detection “template’ 3) Integrating Inductive, Abductive Inferencing • Asymmetric adversaries are quite unpredictable in their behavior, tactics, weapons, and choice of targets. • Induction usually a precursor to Deduction but requires knowledge of relationship between observable signatures and Truth states • Abduction forms best plausible explanation for the observables and observable patterns • A Hybrid Inferencing Process follows the typical sequence of scientific discovery and proof, using a sequence of steps to conjecture, hypothesize, generalize and validate.

15. 3) Integrating Inductive, Abductive InferencingIntegrated Data Mining and Data Fusion Processes

16. 3) Integrating Inductive/Abductive Inferencing * * Waltz, Edward L., “Information Understanding: Integrating Data Fusion and Data Mining Processes”, Proc. of IEEE International Symposium on Circuits and Systems, Monterey CA, May 31-June 4, 1997. For a more detailed description of the integration, see, Waltz, Edward, Knowledge Management in the Intelligence Enterprise, Norwood MA: Artech, 2003, Chapter 8.

17. 4) Aspects of Distributed Fusion • Requirement, framework for sensibly all modern, future military, homeland security IT environments • Architectural issues—need for empirical studies, architectural analysis tools • Need for local and network fusion algorithms • Specification of Information-Sharing Strategies • Design of adaptive network topologies • Need for a “Distributed Fusion JDL Model”

18. Extensions to the Distributed Case

19. 5) Pedigree • We define Pedigree as “an attachment to a massage or communication between nodes that includes any information necessary to the receiving node(s) such that the receiving node fusion processing maintains it’s formal and mathematical processing integrity”.

20. 6) Ontologically-based Data Fusion Processes • One important foundation toward achieving Interoperability and Shared Understanding, especially for Higher-Level Fusion states • Ontological relationships as a basis for development of new Theoretical constructs for the “True” world • Theory as a basis for Algorithm development in the observed world

21. Model (Theory) of Thing-component In the Real World Model (Theory) of Thing-component In the Real World Model (Theory) of Thing-component In the Real World What about a L2 Ontology? • What is a “Situation”? --Not adequately specific --No common understanding • How is it “Refined” --No metrics/quality measures • What kind of algorithmic process yields a “Situation Estimate”? • Llinas assertion: “Situation” is too coarse/abstract to engineer to—MUST get more specific Observability What IS a “Convoy”? Algorithm re Thing-component In the Real World Some set of Components, in some relationship = “Situation” Algorithm re Thing-component In the Real World Partitioning And Labeling Nature of, models of Observational Processes Algorithm re Thing-component In the Real World Situations in the Real World Algorithm re Thing-component In the Observed World Aggregated-object (“Convoy”) Tracking Algorithm (Analysis;Ontology— Sufficient Specificity To develop Theories) • Nature of : • Aggregated Objects • Behaviors, • Events Assumptions, Approximations, Application-needs Informs, Bounds Real World User World Task Reqmts Observed World in the Application (Task) Context Situations as inherent: an attack (situation) may be occurring even if the user’s task-at-hand has no interest in an attack state

22. Summary • There is a clear need for expanding and enhancing the JDL Model to deal with and incorporate the effects of the various issues raised herein • The Model has been an anchor-point for communication and understanding in the Fusion Community and has served us well but it needs contemplative review and a consensus-based modernization