1 / 65

Carnegie Mellon University

Carnegie Mellon University. Imaging & Information Research. Ellen K. Hughes Program Manager (412) 256-3630. Leveraging Synergistic Technologies Ellen Hughes Dave Simon Brian Colonna Ben Levine. Northrop Grumman Technology Exchange Alliance with Carnegie Mellon University. Objective:

abla
Download Presentation

Carnegie Mellon University

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Carnegie Mellon University Imaging & Information Research Ellen K. Hughes Program Manager (412) 256-3630 Leveraging Synergistic Technologies Ellen Hughes Dave Simon Brian Colonna Ben Levine Northrop Grumman Imaging & Information Research

  2. Northrop Grumman Technology Exchange Alliance with Carnegie Mellon University Objective: • Create new vehicles to promote joint research endeavors and exchanges • Pursue joint research that brings together unique resources and technologies from both sides • Create new technologies that would not have existed without this alliance Northrop Grumman Imaging & Information Research

  3. NORTHROP GRUMMAN Science & Technology Center NORTHROP GRUMMAN Corporation Kent Kresa, Chairman, President, & CEO Electronic Sensors & Systems Sector James Roche, Vice President & General Manager Systems Development & Technology Kelly Overman, Vice President & Center Manager Products & Technology Taylor Lawrence, Vice President Science & Technology Center Phil Schweizer, Director Advanced Materials & Electronic Devices Research Imaging & Information Research Robert Mitchell, Manager Compound Semiconductor Research STC Operations & UREI Security.......Baltimore Northrop Grumman Imaging & Information Research

  4. From Undersea to Outer Space Battle-space Awareness Sensor Fusion Unmanned Vehicles Target Tracking Compression Space-Time Adaptive Processing Target Detection, Classification & ID Algorithms Sonar Laser Line Scan Synthetic Aperture Radar Electro-Optical Hyperspectral Video Signals Intelligence SW Agents Compression Robotics Pattern Recognition Autonomous Control Signal Processing Northrop Grumman Imaging & Information Research Embedded Programming Data Mining Path Planning HCI

  5. Imaging & Information Research Programs • Sonar Computer Aided Detection & Classification Dave Simon • Man-made Object Detection Brian Colonna • Reconfigurable Computing w/ Virtual Hardware Ben Levine • SAR Automatic Target Recognition Ellen Hughes • Synthetic Aperture Radar • Underwater Robot Swarms Ellen Hughes Northrop Grumman Imaging & Information Research

  6. Unmanned Underseas Vehicles (UUVs) for surveillance Obstacle Avoidance Sonar Side LookSonar Processors Sonar Computer Aided DetectionComputer Aided Classification (CAD/CAC) Dave Simon Northrop Grumman Imaging & Information Research

  7. Sonar CAD/CAC Team • Paul Haley – NG/Pgh - Algorithms and Verification • Jim Koos – NG/Pgh - Classification and Configuration • Gary Sherwin – NG/Pgh - Detection and Display • Bill Solominsky – NG/CMU - Image Correction and Data Management • Dave Simon – NG/CMU - Image Normalization and Experiments • Walt Petlevich – NG/Pgh - Mercury Computer and Legacy Code • Frank Snyder – NG/Pgh - System Engr & Program Management • Gene Cumm – NG/Annapolis - Support and Customer Interface • Frank Tighe – NG/Baltimore - Support and Tech Transfer Northrop Grumman Imaging & Information Research

  8. Northrop Grumman CAD/CAC Systems are in the Field • The AN/AQS-14 Side-Look-Sonar (SLS) imaging system is a 6” real-aperture-sonar pulled by a helicopter for mine detection • Northrop Grumman developed CAD/CAC algorithms that analyzes Q14 images 4X faster than real-time rate. • 10 Navy PMA CAD/CACs are in the field with the NG CAD/CAC algorithms • PMA has achieved detection and false alarm performance as good as a human image analysis – and sometimes better • Northrop Grumman is developing additional CAD/CAC technology to support: • Real-time CAD/CAC from the helo • CAD/CAC/CAI on Synthetic-Aperture-Sonar (SAS) images World’s first automatic mine detection system #PMA http://sensor.northgrum.com/essd/oceanic/q-14.html#pma Northrop Grumman Imaging & Information Research

  9. Normal Resolution Q14A SLS Image tx070404.0001.gws tx070404.0001.exp • 56 data sets • 5243 images • 512 ground truthed mines Detections Detections Classify Northrop Grumman Imaging & Information Research

  10. Two mines are present in this screen - a small MK 36 and a MK 55 SLS Classifier Processes the Image Below in 1.6 Seconds MK36 MK55 Northrop Grumman Imaging & Information Research

  11. Automatic Cueing Improves Exploitation Process 75 % confidence 90 % confidence Northrop Grumman Imaging & Information Research

  12. Beam Defect Detection & Correction Before • Beam Row • 0 pad row • row gain 0 Row Correction After Northrop Grumman Imaging & Information Research

  13. Beam Defect Detection & Correction - continued Row Gain Correction Normalize on the sum of the rows across one side of the image: port, starboard Before After Northrop Grumman Imaging & Information Research

  14. Image Normalization – Brings Objects out of the Shadows Northrop Grumman Imaging & Information Research

  15. CAD/CAC CY2000 Improvements: AutoPd, DeNoise Northrop Grumman Imaging & Information Research

  16. Underwater Man-made Object Detection Brian Colonna Northrop Grumman Imaging & Information Research

  17. Man-Made-Object Team • Jean Seidel – NG/Pgh - Ground Truth, Performance • Cathy Dietz – NG/Boston - Lines, Texture, Feature Integrator • Brian Colonna – NG/CMU - Fourier Energy, Bayesian Classifier • Jeff Ethridge - NG/Annapolis - Camera Data, Support • Frank Snyder – NG/Pgh - Systems Engr, Program Mgmt • Frank Tighe – NG/Baltimore - Support Northrop Grumman Imaging & Information Research

  18. Man Made Object Detection of Underwater Optical Images • Analysis of straight line segments. • characterize perpendicular and parallel edges • Other features: • low order wavelet analysis • texture analysis (Laws filters) • co-concurrence (gray scale) Northrop Grumman Imaging & Information Research

  19. Software Architecture for MMO Detection Video Camera real-time image 2k x 1k x 12bit Image Store UW scene Pixel File ImageManager Pixel Stream Image Corrections • Line List: • xa(1),ya(1);xb(1)yb(1) • xa(2),ya(2);xb(2)yb(2) • xa(i),ya(i);xb(i)yb(i) • ... Pixel Structure FindLines FourierEnergy 15 Longest lines LineFeatures TextureFeatures StatScore, LineLen, NumRight, AvgRight, NumVertex, AvgVertex, NumPll, AvgPll, NumOrd, AvgOrd Wavelet, Laws, Co-occur, Fft_DC, Fft_AC, Fft_Corr FourierEnergy FeatureIntegrator Operation Mode Tuning Mode Bayesian Matrix Ground Truth TuneClassifier Classifier Natural Image ManMade Image external October Status: done in-progress Northrop Grumman Imaging & Information Research

  20. Quadrant Image Gain Correction and Edge Smoothing Assume the pixel brightness is continuous across the edge in value and slope. 1. Sum the pixels across/down the row/column near the edge. 2. Adjust the quadrant brightness with the ratio of the constant at the edge (b and c). 3. Replace edge rows with average of nearest neighbors PCM-102-22HRCg2\Worms\ CH1S_102_234513.tif B45%, C90% yi=axi+b yi=axi+c Northrop Grumman Imaging & Information Research

  21. Quadrant Image Gain Correction and Edge Smoothing Before After Northrop Grumman Imaging & Information Research

  22. Starfish • The human eye can only discern 4 bits of gray scale differences – and 6 bits of object acuity • Analyst sets the brightness and contrast for best scene acuity then readjusts for a region of interest • Computer algorithms use the full dynamic range of the image data (12 bits) without clipping 2. Balanced Image 1. Original Image 4. Segments on Image 3. Line Segments Northrop Grumman Imaging & Information Research

  23. Washing Machine at the Bottom of the Ocean • Preprocess image • Brighten & Smooth • Edge detect • Sobel • Canny, Marr-Hildreth, Shen-Castan • Postprocess edge image • Threshold (85/15) & Thin • Depends on edge detector • Find lines • Segment merging • Hough HEADING=-64.594 DEPxxxxxxxxx CRABANGLE=-2.597 ROLL=-1.274 PITCH=0.167 PITCHRATE=-0.93933 YAWRATE=-2.72461 FRWDSTAT=0 CENTSTAT=0 AFTSTAT=0 ALTITUDE=24.25 SENSOR=xxxxxxxxxx Cam1MIN=194 Cam1MAX=389 Cam1MEAN=281 Cam1LowPop=14.00 Cam1UprPop=20.00 Cam2MIN=237 Cam2MAX=524 Cam2MEAN=402 Cam2LowPop=20.00 Cam2UprPop=30.00 Bright 45%, Contrast 90% Northrop Grumman Imaging & Information Research

  24. Fourier Energy for MMO Detection Region1 Region2 Region3 1024 2048 • Fourier Energy is calculated in 3 overlapping regions. • Energy weighted by the inverse radius from the center of each region • Feature value comes from the region with the maximum Energy Northrop Grumman Imaging & Information Research

  25. Object Classification with CMU – Prof Henry Schneiderman H. Schneiderman, T. Kanade. "A Statistical Method for 3D Object Detection Applied to Faces and Cars". To appear in IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2000) http://www.cs.cmu.edu/afs/cs.cmu.edu/user/hws/www/ Northrop Grumman Imaging & Information Research

  26. PipeRench Implementation of Northrop Grumman SAR ATR Ben Levine Northrop Grumman Imaging & Information Research

  27. Automatic Target Recognition (ATR) for Synthetic Aperture Radar (SAR) Imagery Ellen Hughes Northrop Grumman Imaging & Information Research

  28. SAR ATR Team • Prof. Kumar - CMU ECE • Danny Carlson - CMU - former graduate student • Bob Mitchell - NG - Manager, Algorithm Design • Mike Hoffelder - NG - Algorithm Design • Bill Zachar - NG - Experimentation & Test Northrop Grumman Imaging & Information Research

  29. Screener Discriminator Classifier Identifier Architecture for Automatic Target Recognition • Designates Regions of Interest (ROI) • Processes entire image • Detect areas of high contrast • Pixel Clustering • Eliminates Natural Clutter • Processes only ROI’s • Computes several features including pose estimate • Uses linear classifier • Finds target class • Processes only ROI outputs from discriminator • Generate correlation planes using composite filters • Computes scalar features from correlation plane • Identifies specific target type from target class • Processes with refined filters • Computes detail features from correlation planes Northrop Grumman Imaging & Information Research

  30. Screener Module: Find any potential target with local contrast detection and mark as a Region of Interest (ROI) ROI’s Input Image Contrast Thresholding Clustering Screener Discriminator Classifier Identification Northrop Grumman Imaging & Information Research

  31. Discriminator Module: Pass only man-made ROI’s and reject natural clutter • Fractal Dimension • Clustering Measure • Connectivity Measure • Neighborhood Quota • Axes Moment Ratio • Standard Deviation • Mean Pose Estimate Binary Man-Made Objects Linear Classifier Gray Scale Input ROI Natural Clutter Target Pixel Identification Discriminator Features Screener Discriminator Classifier Identification Northrop Grumman Imaging & Information Research

  32. Off-lineTraining Off-lineTraining On-line ATR On-line ATR Input ROI score=.98 score=.44 Target Type ID match reject score=.26 score=.30 Refined Pose reject reject Classifier and Identifier Modules:Recognition Workhorse OTSDF Composite Filter Design Target Chip Training Data ID Filters Noise, Clutter & Target Variability Parameters Aspect class1 class2 . . . Class Filters Classifier Pose Estimate Depression Angle Squint Angle Identifier Class Filters ID Filters score=.41 score=.95 score=.53 Class Type Refined Pose reject reject Class found Correlations Correlations Screener Discriminator Classifier Identification Northrop Grumman Imaging & Information Research

  33. Automatic Target Cueing Results Royal Dragon 96 TESAR Imagery TANK unknown TANK Northrop Grumman Imaging & Information Research

  34. Automatic Target Cueing Results Royal Dragon 96 TESAR Imagery TANK TANK TANK Northrop Grumman Imaging & Information Research

  35. Automatic Vehicle Detection Royal Dragon 96 TESAR Imagery unknown unkno unknown unknown unknown Northrop Grumman Imaging & Information Research

  36. ARL SAR ATR Blind Test No False Alarms Below ~40% PID Northrop Grumman Imaging & Information Research

  37. Maritime Robotics - Robot Swarms NORTHROP GRUMMAN Information & Imaging Research Path Planning & Obstacle Avoidance Underwater Domain Expertise Sensor Expertise Carnegie Mellon University Robotics Institute Robot Team Simulation & Implementation including - Complete Coverage - 2D to 3D Simulation - Communication Strategies - Team Robot Capability Mix Prof. Tucker Balch Prof. Howie Choset Prof. Manuela Veloso Dr. Frank Snyder Ellen Hughes Northrop Grumman Imaging & Information Research

  38. Vehicles for Technology Exchange • Graduate Research Fellowship • Funding for graduate research in areas of interest to NG • Beginning Fall 2000: Rajan Singh performing Image Understanding Research under Prof. Kumar • University Research Exchange Initiative • NG engineers perform sabbatical-type research at CMU • Fall 2000: Dr. Andy Miklich visiting scientist with Prof. Katia Sycara in Intelligent SW Agents Northrop Grumman Imaging & Information Research

  39. Vehicles for Technology Exchange • Summer Intern Program • Summer 2000: 4 interns • Part-time Intern Program with Tuition Assistance • Beginning Fall 2000: 2 summer intern students, Brian Colonna and Dave Simon, continue to work part-time through the academic year and receive 50-75% tuition assistance • Student Projects for joint NG - CMU Research • Beginning Fall 2000: ECE student, Brian Colonna, pursuing research in reconfigurable computing on application of NG interest. Student combines CIT honors research with NG project to attain course and job requirements. Benefits: business driven research performed for professor; advanced study, salary and tuition assistance earned by student; new academic research performed for NG. Northrop Grumman Imaging & Information Research

  40. Many more Joint Opportunities Ahead • Joint programs currently underway represent the beginning of this Technology Exchange Alliance • Many other opportunities exist that exploit NG and CMU’s joint synergies • We’re working toward growing this relationship and building many more programs to support CMU, NG and the government’s missions Northrop Grumman Imaging & Information Research

  41. Imaging & Information Research Organization & Staff Carnegie Mellon U. NG-STC IIR Pittsburgh IMAGING & INFORMATION RESEARCH DEPT. Bob Mitchell, Manager Karen Blotzer, Secy. NG-STC IIR STAFF Dietz, Cathy Forrester, Martin Haley, PH Paul Hoffelder, Mike Hughes, Ellen Kline, Larry Koos, Jim Miklich, Andy Oblak, Tod Oravec, Jim Petlevich, Walt Seidel, Jean Sherwin, Gary Snyder, Frank Zachar, William CMU FACULTY teamed w/ NG Robotics Institute Balch, Tucker Choset, Howie Schneiderman, Henry Thorpe, Chuck Veloso, Manuela Electrical & Computer Engineering Khosla, Pradeep Kumar, BVK Schmit, Herman Computer Science Sycara, Katia JOINT NG-CMU RESEARCHERS Student Interns Colonna, Brian Levine, Ben Simon, Dave Solominsky, Bill Research Exchange Scientist Miklich, Andy (SW Agents) Research Fellow Rajan Singh (Image Understanding) NG-RICE UNIVERSITY ALLIANCE Research Fellow Vidya Venkatachalam (Wavelets) Northrop Grumman Imaging & Information Research

  42. PipeRench Implementation of Northrop Grumman SAR ATR October 26, 2000 Ben Levine NORTHROP GRUMMAN

  43. The Team • Prof Herman Schmit – CMU ECE • Ben Levine – CMU graduate student & NG Intern • Brian Colonna – CMU/NG – student intern • Ellen Hughes – NG – Program Design • Tod Oblak – NG – Embedded Systems • Mike Hoffelder – NG – Algorithm Design NORTHROP GRUMMAN

  44. PipeRench • PipeRench is a reconfigurable computing chip developed at CMU. • The PipeRench project involves faculty and students from CEDA (Center for Electronic Design Automation) in the ECE Department and the School of Computer Science. • The PipeRench project has encompassed everything from physical design to microarchitecture to compiler design. • More information can be found at: http://www.ece.cmu.edu/research/piperench NORTHROP GRUMMAN

  45. Reconfigurable Computing • Design “hardware” for each application. • Custom logic design gives high performance, like an ASIC. • Map it onto programmable hardware. • We can change the hardware, like software on a CPU. • Advantages: • Performance approaching that of an ASIC. • Flexibility approaching a general purpose processor. • Quick design cycles and low design costs for each application, like software design. NORTHROP GRUMMAN

  46. PCI- PipeRench CPU RAM PCI- PipeRench PCI Bus NORTHROP GRUMMAN

  47. PCI-PipeRench Architecture PCI BUS Fabric Input Controller Output Controller Configuration Cache Config Data NORTHROP GRUMMAN

  48. PipeRench Data Flow PCI BUS Fabric Input Controller Output Controller Configuration Cache Config Data NORTHROP GRUMMAN

  49. Pipeline Stages PipeRench Fabric • Example: FIR Filter x y W1x + y x y W2x + y NORTHROP GRUMMAN

  50. Stripe Architecture ... Stripe 1 Interconnect Network Stripe 2 Stripe 3 ... PE1 PE2 PEN Global Bus ... Stripe 4 Stripe N Interconnect Network NORTHROP GRUMMAN

More Related