Developing OC Testbeds and Concepts

1. Developing OC Testbeds and Concepts Kirstie L. Bellman, Ph.D. Aerospace Integration Science Center (AISC) The Aerospace Corporation January 18, 2006 • IR&D Aerospace Leads: Dr. Kirstie Bellman, Dr. Chris • Quick remarks (I hope) on OC and Biology • iroom as testbed for OC studies • Emotional reasoning and ‘self’ (if time) Kirstie Bellman The Aerospace Corporation

2. OC and Natural Systems • The OC community should use biological ideas to draw inspiration from and re-implement in technical systems. Hence, • Be impressed (very) with phenomena and capabilities • Be unimpressed with the explanations and theories • Current technology often becomes the explanation of the moment,e.g.’automatons’ & hydraulic pumps, electrical circuits, computers, holograms, Kirstie Bellman The Aerospace Corporation

3. Some of the Noteworthy Phenomena • Self-organization and emergence • Over different timescales: growth, maturation, memory • Of structure, patterns, behavior, symbols, processes, ‘cultures’ • Reflection - even of cognitive processes • Scalable Integration • of ‘top-down’ and ‘bottom up’ processing • of diverse processing styles • of perception and movement with decision processes • Limited components and superb overall performance • Relatively fast, precise with superb discrimination and generalization • YET slow and inefficient components, sloppy levels, leaky components • Emotional reasoning –situated, embodied, widespread and integrative, compelling, immediate, subjective Kirstie Bellman The Aerospace Corporation

4. The iroom(s)Integrating Biologically-Realistic Controllers of Sensors with Traditional and New Computational Capabilities Funded by NSF and The Aerospace Corporation In collaboration with research teams of Michael Arbib, Laurent Itti, USC Original ones built at USC and Aerospace; Others being built at CalPoly Pomona and Univ. Maryland Kirstie Bellman The Aerospace Corporation

5. Motivation for iroom studies: Too Much Data and Too Little Time • Ability to process data from sensors and other information sources is lagging behind the ability to create data. • New types of rapid military operations require even more information quickly gathered from a large number of information sources. • In order to meet these time requirements, • Need to quickly identify the most “relevant” data • Manage sensor and processing resources in such a way as to maximize the collection of the most relevant or salient data • Adapt resources as needed to current mission and available data and processing resources Kirstie Bellman The Aerospace Corporation

6. Goals of Aerospace Research • Transition NSF work developing adaptive controllers based on realistic brain models to Aerospace networks of sensors and processors • Extend techniques to space sensors and data • Powerful bottom-up processing (“noticing”) uses saliency work by Laurent Itti and C. Koch • Add “top-down” targeting information, classic image processing algorithms • Develop systems better than biology by exploring suitable scanning and coverage strategies and policies. • New control issues with semi-autonomous sensors • Without a priori knowledge of sensor locations or geometry of locale Kirstie Bellman The Aerospace Corporation

7. Schematic of Brain Areas and Models Kirstie Bellman The Aerospace Corporation

8. Slide Courtesy of Laurent Itti Kirstie Bellman The Aerospace Corporation

9. IR&D Slide Courtesy of Laurent Itti Kirstie Bellman The Aerospace Corporation

10. Kirstie Bellman The Aerospace Corporation

11. Kirstie Bellman The Aerospace Corporation

12. Kirstie Bellman The Aerospace Corporation

13. GOAL Slide by Nathan Mundhenk Kirstie Bellman The Aerospace Corporation

14. Sensor (Camera) Controller Slide by Jacob Everist Kirstie Bellman The Aerospace Corporation

15. Scenario Overview Kirstie Bellman The Aerospace Corporation

16. iroom Tasks • Without a priori knowledge of locale’s geometry, sensor positions and capabilities: • Scanning • Calculate and maintain minimum coverage and revisit time requirements of viewable area • Tracking • Maintain positioning and trajectory of all moving objects; prediction for likely sensor region • Initiative • “Notice” all new objects within viewable area; determine if moving object • Interrupts • Only in ‘emergencies’ else Sensornet manager tasks & sensor controller decides how to manage its task list. Kirstie Bellman The Aerospace Corporation

17. System Overview Role is to “notice”, create signature on moving objects, track as directed Role is to task, monitor global goals, hand-off, analysis Kirstie Bellman The Aerospace Corporation

18. Algorithm Development and Testing in the iroom Many other computational components than saliency: Object clustering, sensor controllers, performance feedback and metrics, trackers, “gist” and signatures, and so forth Kirstie Bellman The Aerospace Corporation

19. N. Mundhenk’s Work on Object Clustering and “Gist” Kirstie Bellman The Aerospace Corporation

20. J. Everist: Task Map Merging Kirstie Bellman The Aerospace Corporation

21. System Overview: Associated SW at Sensor Saliency Clustering Objects Task merging Metrics Task list DBs Sensor manager Policies/goals Policy interpreter ‘world’ state (built) Analyses Hand off algorithms Historical archives List of moving objects (position, loss) Programs associated with environment Probes to measure performance, state Wrappings KB, Wrappings algorithms VW/UI reflecting iroom Kirstie Bellman The Aerospace Corporation

22. Wrappings provide in iroom (1) • First Flexibility: Ability to drop in new algorithms, probes, models as developed for experiments, comparisons on scenarios • No fixed configuration of resources in iroom • The wrappings gives the type of resource and ‘problems’ and distinctions, constraints e.g. kmeans or slinc because cameras are looking at big objects or looking for local connectivity • The study manager pulls resources in as needed by context and other processing elements dynamically Kirstie Bellman The Aerospace Corporation

23. Wrappings provide in iroom (2) • Second, enables reflective processes and analyses (doesn’t build them automatically) • Allows questions, such as, • How successful is this policy for monitoring and control by cameras given the number of people tracked in iroom? • How well was the iroom able to use this task merging strategy? • Allows diversity of reflective processes at different levels, e.g. different probes, different metrics and different analyses used to determine self-organization (here self-configuration); approaches to problem-solving Kirstie Bellman The Aerospace Corporation

24. What Do We Add to the iroom to Make It Serve As a Testbed for OC Studies? • The iroom is a Testbed for “Integration Science” • Allows explicit testing of different computational components with metrics, e.g. Jacob Everist’s experiments on camera controller; Nathan Mundhenk’s experiments on cluster algorithms • Allows explicit research on top-down and bottom-up integration strategies • Compares “biologically-inspired” approaches with other computational approaches • Would like suggestions and guidance on the types of experiments/components that would facilitate OC studies Kirstie Bellman The Aerospace Corporation

25. Possible Roles of Emotion in OC • Roles of emotion in biological systems • Arousal and motivation (Lindsay& Norman) • Damasio “They occur as a dynamic newly instantiated, “on-line” representation of what is happening in the body now. … The process of continuous monitoring, that experience of what your body is doing.”(1994, pp.144-45) • Prioritization, Choice, and Discernment • Damasio from motivation and self-awareness to “Cognitive guidance”; others quick choice strategies • Associative, widespread integration • Emotional characteristics: • Situated,embodied, associative, integrative, widespread, compelling, subjective, Kirstie Bellman The Aerospace Corporation

26. My View • Being in the world is not dispassionate, neutral, or unbiased. • We have critical need to understand one’s world in relationship to oneself. • A self helps maintain a coherent viewpoint and a set of values about the world relative to that system’s needs and experience. • Emphasized self as an integration concept – that “global construct”, helps integrate/summarize over the set of self-monitoring and self-perception mechanisms, and self-reflection capabilities. • Sacks,Bruner believes some autistic patients may lack the ability to integrate their perceptual experiences • “with higher integrative ones and with concepts of self, so that relatively unprocessed, uninterpreted, unrevised images persist.” (1995, p. 282). • Emotion is “processing with an attitude” biased, private, importantly self-oriented and supports a self Kirstie Bellman The Aerospace Corporation

27. Back up slides Kirstie Bellman The Aerospace Corporation

28. Saliency Slide Courtesy of L.Itti Kirstie Bellman The Aerospace Corporation

29. Naïve Tracker Naïve and non-naïve tracker What is interesting Feature tracker Kirstie Bellman The Aerospace Corporation

30. Classes Kirstie Bellman The Aerospace Corporation

31. Targets Kirstie Bellman The Aerospace Corporation

32. Kirstie Bellman The Aerospace Corporation

33. Nathan Mundhenk experiments Kirstie Bellman The Aerospace Corporation

34. Camera Controller and Sensornet Manager Current Purpose of Camera Controller: • Directly control camera resources • Schedule and merge assigned tasks: surveillance, tracking assigned objects, “noticing” new objects • Send feedback on task and data to Sensornet Manager Kirstie Bellman The Aerospace Corporation

35. Sensornet Manager Current Purpose: • Assign general tasks to camera controllers • Synthesize data sent back by controllers – develop global model of combined sensor coverage • Ensure target attendance and minimum room surveillance • Coordinate hand-off of target tracking to different camera Kirstie Bellman The Aerospace Corporation