1 / 49

Consistent Visual Information Processing

Consistent Visual Information Processing. Axel Pinz EMT – Institute of Electrical Measurement and Measurement Signal Processing TU Graz – Graz University of Technology pinz@emt.tu-graz.ac.at http://www.emt.tu-graz.ac.at/~pinz. “Consistency”. Active vision systems / 4D data streams.

owen-clark
Download Presentation

Consistent Visual Information Processing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Consistent Visual Information Processing Axel Pinz EMT – Institute of Electrical Measurement and Measurement Signal Processing TU Graz – Graz University of Technology pinz@emt.tu-graz.ac.at http://www.emt.tu-graz.ac.at/~pinz

  2. “Consistency” • Active vision systems / 4D data streams • Multiple visual information • Imprecision • Ambiguity • Contradiction

  3. This Talk: Consistency in • Active vision systems: • Active fusion • Active object recognition • Immersive 3D HCI: • Augmented reality • Tracking in VR/AR

  4. AR as Testbed Consistent perception in 4D: • Space • Registration • Tracking • Time • Lag-free • Prediction

  5. Agenda • Active fusion • Consistency • Applications • Active object recognition • Tracking in VR/AR • Conclusions

  6. Active Fusion Simple top level decision-action-fusion loop:

  7. Active Fusion (2) • Fusion schemes • Probabilistic • Possibilistic (fuzzy) • Evidence theoretic (Dempster & Shafer)

  8. Probabilistic Active Fusion Nmeasurements, sensor inputs: mi Mhypotheses: oj , O = {o1, …, oM } Bayes formula: Use entropy H(O)to measure the quality ofP(O)

  9. Probabilistic Active Fusion (2) Flat distribution: P(oj )=const. Hmax Pronounced distribution: P(oc ) = 1; P(oj ) = 0, j  c H = 0 • Measurements can be: • difficult, • expensive, • N can be prohibitively large, … •  Find iterative strategy to minimizeH(O)

  10. Probabilistic Active Fusion (3) Start with A  1 measurements: P(oj|m1, … ,mA), HA Iteratively take more measurements: mA+1, … ,mB Until: P(oj|m1, … ,mB), HB  Threshold

  11. Summary: Active Fusion • Multiple (visual) information, many sensors, measurements,… • Selection of information sources • Maximize information content / quality • Optimize effort (number / cost of measurements, …) Information gain by entropy reduction

  12. Summary: Active Fusion (2) • Active systems (robots, mobile cameras) • Sensor planning • Control • Interaction with the scene • “Passive” systems (video, wearable,…) • Filtering • Selection of sensors / measurements

  13. Consistency • Consistency vs. Ambiguity • Unimodal subsets Ok • Representations • Distance measures

  14. Consistent Subsets Hypotheses O = {o1 ,…, oM } Ambiguity: P(O) is multimodal Consistent unimodal subsets Ok  O Benefits: • Application domains • Support of hypotheses • Outlier rejection

  15. Distance Measures Depend on representations, e.g.: • Pixel-level SSD, correlation, rank • Eigenspace Euclidean • 3D models Euclidean • Feature-based Mahalanobis, … • Symbolic Mutual information • Graphs Subgraph isomorphism

  16. Mutual Information Shannon´s measure of mutual information: O = {o1 ,…, oM } A  O, B  O I(A,B) = H(A) + H(B) – H(A,B)

  17. Applications • Active object recognition • Videos • Details • Tracking in VR / AR • Landmark definition / acquisition • Real-time tracking

  18. Active vision laboratory

  19. Active Object Recognition

  20. Active Object Recognitionin Parametric Eigenspace • Classifier for a single view • Pose estimation per view • Fusion formalism • View planning formalism • Estimation of object appearance at unexplored viewing positions

  21. Applications  Active object recognition • Videos • Details  Control of active vision systems • Tracking in VR / AR • Landmark definition / acquisition • Real-time tracking Selection, combination, evaluation  Constraining of huge spaces

  22. Landmark Definition / Acquisition What is a “landmark” ? corners blobs natural landmarks

  23. Automatic Landmark Acquisition • Capture a dataset of the scene: • calibrated stereo rig • trajectory (by magnetic tracking) • n stereo pairs • Process this dataset • visually salient landmarks for tracking

  24. Automatic Landmark Acquisition visually salient landmarks for tracking • salient points in 2D image • 3D reconstruction • clusters in 3D: • compact, many points • consistent feature descriptions • cluster centers  landmarks

  25. Processing Scheme

  26. Office Scene

  27. Office Scene - Reconstruction

  28. Office Scene - Reconstruction

  29. Unknown Scene Landmark Acquisition Real-Time Tracking

  30. Real-Time Tracking • Measure position and orientation of object(s) • Obtain trajectories of object(s) • Stationary observer – “outside-in” • Vision-based • Moving observer, egomotion – “inside-out” • Hybrid • Degrees of Freedom – DoF • 3 DoF (mobile robot) • 6 DoF (head and device tracking in AR)

  31. Outside-in Tracking (1) stereo-rig IR-illumination • wireless • 1 marker/device: • 3 DoF • 2 markers: 5 DoF • 3 markers: 6 DoF devices

  32. Outside-inTracking (2)

  33. Consistent Tracking (1) • Complexity • Many targets • Exhaustive search vs. Real-time • Occlusion • Redundancy (targets | cameras) • Ambiguity in 3D • Constraints

  34. Consistent Tracking (2) • Dynamic interpretation tree • Geometric / spatial consistency • Local constraints • Multiple interpretations can happen • Global consistency is impossible • Temporal consistency • Filtering, prediction

  35. Consistent Tracking (3)

  36. Hybrid Inside-Out Tracking (1) Inertial Tracker • 3 accelerometers • 3 gyroscopes • signal processing • interface

  37. Hybrid Inside-Out Tracking (2) • complementary sensors • fusion

  38. Summary: Consistency in • Active vision systems: • Active fusion • Active object recognition • Immersive 3D HCI: • Augmented reality • Tracking in VR/AR

  39. Conclusion Consistent processing of visual information can significantly improve the performance of active and real-time vision systems

  40. Acknowledgement Thomas Auer, Hermann Borotschnig, Markus Brandner, Harald Ganster, Peter Lang, Lucas Paletta, Manfred Prantl, Miguel Ribo, David Sinclair Christian Doppler Gesellschaft, FFF, FWF, Kplus VRVis, EU TMR Virgo

More Related