Computational Intelligence Research in ECE

1. Linguistic Scene Description Target detection and Recognition Sketch Understanding Scene Matching What’s Coming? Robot Spatial Reasoning Computational Intelligence Research in ECE Jim Keller, Marge Skubic, Dominic Ho and others

2. Computational Intelligence Technologies • Fuzzy Set Theory and Fuzzy Logic • Neural Networks • Probabilistic Reasoning • Including Evolutionary Computation and Genetic Algorithms • To Build Intelligent Systems for: • Image/Signal Processing • Pattern Recognition • Robotics • Strong Applications Orientation

3. Forward-looking GPR Seismic/Acoustic Sensing Object Detection and Recognition Large long-term effort in Landmine Detection

4. GSTAMIDS HSTAMIDS Technology Transfer Army Research Office MURI Basic Research Technology Transfer from Basic Research to Fielded Systems

5. Little Bob Robotic Tripwire Detection

6. Morphological Shared Weight Neural Networks for Object Recognition

7. Output Plane Original Frame Final Output Target Aim Point Selection First Developed to Find Object (Blazer) in Visible Imagery

8. All Twelve Targets Detected with no False Alarms Target Detection in SAR Imagery

9. Trained on 2 frames from one sequence (8 instances) Tested on Different Flight Sequence Application to Tank Detection in (Processed) LADAR Range Images

10. Typical Training Image of Bob Face Recognition (Homeland Security)

11. Even with Glasses on Examples of “Bob” Detection

12. 1. Input Image (from video sequence) 2. Locate Iris Pair 3. Locate Eyelids 4. Locate Pupil Pair 5. Locate Hirschberg Points / Estimate Fixation 6. Extracting Features (e.g., Crescents) Automated Amblyopia Screening Assistant

13. Screenshot of Program User Interface

14. Motion capture Data analysis Classification dysfunctional normal Pre-process and store Database Animation for visualization Equine Gait Analysis

15. v A B q Histogram of forces Scene Description • Natural scene understanding is an important aspect of computer vision • Spatial relations among image objects play a vital role in the description of a scene

16. 1 constant p p - 2 gravitational 1 • Each histogram gives its opinion about the relative position • between the objects that are considered. p p - 3 2. The two opinions are combined. Four numeric and two symbolic features result from this combination. 3. A system of 27 fuzzy rules and meta-rules allows meaningful linguistic descriptions to be produced. Linguistic Scene Description

17. Linguistic Scene Description There are 5 missile launchers (1, 2, 3, 6, 8) They surround a center vehicle (4) The image includes a SAM site A convoy of vehicles (5, 7, 9, 10) is BelowRight of the SAM site

18. Loosely ABOVE-LEFT. Scene Description and Recognition The system here describes the relative position of the red object(s) with respect to the group of buildings (in blue).

19. Image 1 Image 2 Do These Images Contain The Same Power Plant ? Another Question : If they are indeed the same power plant, which building(s) that appear on Scene 1, also appear on Scene 2 ? (labeling problem)

20. overhead view ? DIRECT MATCHING IS DIFFICULT THIS WOULD BE EASIER overhead view ? Scene Matching Using Fuzzy Regions

21. Scene Matching and Recovery of View Parameters 3,628,800 ways to match the two scenes. The only true matching got the highest matching degree

22. Human/Robot Dialog • Spatial Reasoning incorporated into NRL’s Natural Language Understanding System for mobile robots • Sensed data results in a “grid map” that displays occupancy of cells (doesn’t need to be binary) Grid map after component labeling – robot heading towards Object 5

23. DETAILED SPATIAL DESCRIPTIONS for 6 OBJECTS: • Object 1 is mostly behind me but somewhat to the right (the description is satisfactory). The object is very close. • Object 2 is behind me (the description is satisfactory) The object is very close. • Object 3 is to the left of me but extends to the rear relative to me (the description is satisfactory). The object is very close. • Object 4 is mostly to the right of me but somewhat forward (the description is satisfactory). The object is very close. • Object 5 is in front of me (the description is satisfactory). The object is very close. • Object 6 is to the left-front of me (the description is satisfactory). The object is close. Scene 1

24. There are objects in front of me and behind me. Object number 3 is to the left of me. Object number 4 is mostly to the right of me. What do you see, Roby? High-Level Description

25. Understanding Sketched Route Maps PATH DESCRIPTION GENERATED FROM THE SKETCHED ROUTE MAP 1. When table is mostly on the right and door is mostly to the rear (and close) Then Move forward 2. When chair is in front or mostly in front Then Turn right 3. When table is mostly on the right and chair is to the left rear Then Move forward 4. When cabinet is mostly in front Then Turn left 5. When ATM is in front or mostly in front Then Move forward 6. When cabinet is mostly to the rear and tree is mostly on the left and ATM is mostly in front Then Stop

26. Sketch-Based Navigation The robot traversing the sketched route The sketched route map

27. The digitized sketched route map The robot traversing the sketched route Sketch-Based Navigation

28. Centered here “GORT, go behind object #3” Identification of Spatial Regions Region shown in green Based on Histograms of Forces Future work: combine ATR with spatial language

29. Cognitive Robotics: Collaborative Research with Vanderbilt Univ. Funded by NSF What’s Coming? (Here, Actually)

30. Robot Skill Acquisition: Teaching Robonaut New Skills What’s Coming? (Proposed)

31. Mobile Robot Teams Research Laboratory Mini-, Micro-, Nano- and Bio-Scales What’s Coming? (We Hope) Sensor-loaded (Sensor nets) Intelligent Action (Reasoning) Cooperative Behavior Power Issues Communication Hardware and Systems Concerns Potential Cast of Characters: Jim, Marge, Henry, Lex, Shubhra, Randy, Mike, Rusty, Dominic, Yi (CS), Sheila (BAE), …