Detecting and Tracking Tractor-Trailers Using View-Based Templates

1. Detecting and Tracking Tractor-Trailers Using View-Based Templates Masters Thesis Defense by Vinay Gidla Apr 19,2010

2. Introduction • Object tracking: • Sports analysis • Games and gesture recognition • Retail video mining • Automobile driver assistance • Traffic surveillance • Volume, individual speeds, classification • Lane changes, speed violations, congestions

3. Feature-based vehicle tracking • Beymer et al. 1997 use feature point approach with motion cues to segment vehicles using homography • Kanhere et al. 2008 use features with 3D estimation using multi-level homographyFeature_based.avi • Drawback: These approaches track features on the vehicle, not vehicle as a whole

4. Template-based tracking • Model the object by 2D template of image intensities • Compare search image with template image • Comparison usually by discrete cross-correlation • Good: Both spatial and appearance information Able to retrieve shape of the object • Bad: Encode vehicle appearance from single viewpoint Do not adapt to changes in appearance of object

5. Proposal • Overcome the limitations of a single template by using a template sequence instead of a single template • The template sequence encapsulates all of the vehicle’s perspective deformations • As a starting step, aim to detect and track contours of tractor-trailers in multi-lane traffic

6. Video Sequences

7. Template creation Training sequence: • A portion of traffic video containing a tractor-trailer • Process the video frames to create a template sequence

8. Training Sequence

9. Training frame

10. Manual contour selection

11. Template creation

12. Template sequence

13. Algorithm Overview

14. Step 1: Background subtraction

15. Reference background

16. Input Video Frame

17. Background subtracted frame

18. Step 2: Blob-Template match

19. Blob-Template match

20. Plot of Blob-Template match

21. Step 3: Trace contour

22. Results based on template-blob correlation

23. Plot of misalignment

24. Gradient magnitude match • Reduce the misalignment by including salient features such as points of high gradient magnitude • These points are located at identical spatial locations in every tractor-trailer

25. Training frame

26. Gradient Magnitude

27. Template Gradient Magnitude

28. Template Gradient Sequence

30. Results based on template-frame gradient match

31. Plot of misalignment

32. Test sequences

33. Results(Lane 3)

34. Results(Lane 2)

35. Level set-based trackingfor automatic template generation

36. Conclusion • The new approach accurately traces the contours of all the tractor-trailers in the traffic video • Works for multi-lane traffic • Minor misalignment

37. Future extensions • Tracking other classes of vehicles such as passenger cars, buses etc • Compact template sequence with minimal template redundancy • Implement matching using level set techniques

38. Thank You

39. Questions & Discussion