Research on Vision Systems for Small Unmanned VTOL Vehicles

1. Research on Vision Systems for Small Unmanned VTOL Vehicles K. P. Valavanis, M.Kontitsis, R.Garcia

2. A UAV Vision System for Airborne Surveillance M.Kontitsis, K. Valavanis N. Tsourveloudis University of South Florida Technical University of Crete

3. Objectives • Present a methodology for the design of a machine vision system for aerial surveillance by Unmanned Aerial Vehicles (UAVs) • Identify specified thermal source • Perform these functions on board the UAV in Real time • Flexible enough to be used in a variety of applications

4. Machine Vision System Noise reduction Feature extraction (Size, Mean intensity) Feature vectors classification Alarm on/off Persistence IR/NIR image

5. Input Images • IR (3μm~ 14μm) • Near IR camera (1μm ~ 3μm) • 8bit grayscale

6. Noise Reduction • 5x5 spatial Gaussian filter + Smoothes noise while preserving most of the features on the image

7. Feature Extraction This module attempts to extract information about the regions on the image • Size of region using a region growing algorithm • Mean intensity of region defined as

8. Feature Vector Classification Subsystem Mean Intensity of Region Fuzzy Classifier Target Identification Possibility Size of Region

9. Mean Intensity Membership Functions Low Mid High Grayscale values

10. Region Size Membership Functions Small Medium Large # of Pixels

11. Objective ID Possibility Membership Functions

12. Output of the Fuzzy Classifier Possibility Mean Intensity Size (pixels)

13. Classification Example

14. Classification Result p>0.8 0.5<p<0.8 p<0.5

15. Alarm raising • Persistent classification of a certain region as of High Possibility raises the alarm • The region that raised the alarm is pin-pointed by a red cross • The alarm stays on even if the thermal source is temporarily occluded by surroundings or lost due to violent camera vibration

16. Alarm raising • Mechanism used : Alarm Registry • Ifpi > Ton => Activate alarm • Ifpi <Toff => Deactivate alarm

17. Complexity • Noise Reduction O(n2) for (nxn) image • Region Growing O(n2) for (nxn) image • Fuzzy Logic Classifier*  O(nxm) *in its current implementation n inputs, m rules

18. Case Study: Forest fires Adjusting membership functions manually

19. Classification Example objective present Thermal source (fire)

20. Classification Result objective present possibility>0.7 possibility <0.5 0.5< possibility <0.7

21. Classification Example objective absent

22. Classification Result objective absent possibility>0.7 0.5< possibility <0.7 possibility <0.5

23. Classification Result (Video) (objective present)

24. Classification Result (Video) (objective absent)

25. Automatic Parameter Selection aij bij cij dij • aijbijcijdijfori=1 andj =1,2,3 which define the form of the membership functions of Mean Intensity

26. Basic Elements of the Genetic Algorithm • Chromosome => parameters x=(aijbijcijdij) • Fitness function correct activation of the alarm fitness(x)=1correct deactivation of the alarm fitness(x)=0 in any other case

27. Basic Elements of the Genetic Algorithm • Selectionoperatorselects individualsfor matingas many times as the ratio of their fitness to the total fitness of the population • Crossover operatorcrossover probabilitypc=0.7 • Mutation operator mutation probabilitypm=0.001

28. Mean Intensity M.F. as evolved by GA

29. Result (using GA for parameter selection)

30. Remarks • Adjustable for a variety of applications • Real time execution • Correct identification rate of about 90% • False alarms not entirely avoided (especially in the system evolved by the GA)

31. Design, Implementation and Testing of a Vision System for Small Unmanned VTOL Vehicles K. P. Valavanis, M.Kontitsis, R.Garcia

32. Aim of the work • To explore the design alternatives in the attempt to implement a functional vision system for a small Unmanned VTOL. • Two different approaches examined: • On board processing • On the ground processing

33. Limitations • Weight Limitations • Power Supply Limitations • Processing power issues • Communications

34. Centralized approach (processing wise) UAV / VTOL • Sensing (camera) • Transmitting data to GCS Commands Data link (telemetry + video) • Map Building • Target Identification • Command Issuing Processing is left to the PC on the Ground Control Station Ground Control Station (GCS )

35. De-centralized approach (processing wise) Processing is carried out locally on the PC onboard the UAV / VTOL UAV / VTOL • Sensing (camera) • Map Building • Target Identification • Transmitting data and alarm signal to GCS Commands Data link (telemetry + video) Ground Control Station (GCS ) • Command Issuing

36. General trends in the area

37. Functionality and characteristics

38. Processing on the Ground

39. Hardware configuration (on the ground processing)

40. Vision algorithm overview

41. Experimental Results

42. “Mine” detection results

43. “Mine” detection results 2

44. “Mine” detection results 3

45. “Mine” detection results 4

46. Processing on board the VTOL

47. Raptor 90 with on board vision system.

48. On board processing Used for processing Firewire camera Wireless 802.11b Onboard PC Wireless 802.11b Ground Computer Used for monitoring

49. On board system

50. Onboard system architecture (hardware)