1 / 27

Interactive Virtual Relighting and Remodelling of Real Scenes

Interactive Virtual Relighting and Remodelling of Real Scenes. C. Loscos 1 , MC. Frasson 1,2 ,G. Drettakis 1 , B. Walter 1 , X. Granier 1 , P. Poulin 2 (1) iMAGIS* - GRAVIR/IMAG - INRIA Rhône-Alpes * iMAGIS is a joint project of CNRS/INRIA/UJF/INPG

abreen
Download Presentation

Interactive Virtual Relighting and Remodelling of Real Scenes

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Interactive Virtual Relighting and Remodelling of Real Scenes C. Loscos1, MC. Frasson1,2,G. Drettakis1, B. Walter1, X. Granier1, P. Poulin2 (1) iMAGIS* - GRAVIR/IMAG - INRIA Rhône-Alpes * iMAGIS is a joint project of CNRS/INRIA/UJF/INPG (2) Département d ’informatique et de recherche opérationnelle, Université de Montréal

  2. Motivation Interior design Changes in lighting Geometric modification

  3. Motivation • Goal: interactive system • simple capture process • interactive ( ~ 1 sec. per frame) • modification of lighting • modification of geometry

  4. Motivation • We have to: • create a simple model of the real scene • geometry • approximate reflectance • represent real global illumination • develop interactive methods for modifications • Goal is to be convincing, not highly accurate

  5. Previous Work • Geometric reconstruction • vision methods [Faugeras et al. 97, ...] (Realise) • constraint-based systems [Debevec et al. 96, Poulin et al. 98] • software: Photomodeler, etc. • Reflectance recovery • e.g., [Sato et al. 97, Ward92, Debevec98, Yu et al. 98, etc].

  6. Previous Work • Real-time direct shadows • real point light source [State et al. 96] • Common global illumination • non-interactive • [Nakamae et al. 86, Fournier et al. 93, Jancène et al. 95, Debevec 98, Yu et al. 98, Yu et al. 99] • interactive • [Drettakis et al. 97, Loscos et al. 98]

  7. Algorithm Overview • Input • Pre-process • Interactive modification

  8. Algorithm Overview - Assumptions • Single viewpoint • Diffuse assumption • Lighting: • direct lighting: ray casting • indirect lighting: hierarchical radiosity radiosity = reflectance x ( direct light + indirect light )

  9. Simple Input Process • Geometric reconstruction • several (4-5) images from different viewpoints • geometric modelling using “Rekon” [Poulin et al. 98] • Reflectance reconstruction • several (5-7) images from a single viewpoint • different lighting conditions: single light source at different positions • “radiance images”

  10. Input • Radiance images from single viewpoint • combining multiple images reduces artefacts of estimation different lighting conditions

  11. Pre-process • Computation of approximate diffuse reflectance pixel by pixel • compute individual reflectance images • merge reflectance images using confidence values • Initialise lighting system • data structure • hierarchical radiosity system

  12. Reflectance Computation • For each radiance image reflectance=radiosity/(directlight+indirectlight) photograph reflectance

  13. Confidence Images • Estimate confidence • confidence ~ quality of reflectance estimate • create a confidence image per light source position • Begin with confidence = Visibility • low in shadow regions • Filtering process to remove unwanted effects • low for outliers (specular effects, light tripod)

  14. x avg. x Merged Reflectance Computation reflectance confidence merged reflectance

  15. Reflectance Direct lighting Indirect lighting pixel Interactive Modification: Shadow Reprojection • Direct illumination: pixel by pixel • Indirect illumination: optimised radiosity solution a

  16. Shadow Re-projection simulated photograph

  17. original object insertion Add/move/remove object (virtual or real) • Visible surface changes: pixel by pixel local update • project bounding box of dynamic object • localise directly affected pixels

  18. original object insertion Add/move/remove object (virtual or real) • Direct lighting updates: shaft structure • localisation of visibility changes (shadows) • accelerate visibility computation (blocker lists)

  19. Add/move/remove object (virtual or real) • Indirect illumination computed by a radiosity solution (optimised by the shaft structure) • Example: moving object Position 1 Position 2

  20. Real Object Removal

  21. Removing Real Objects • Use of the reflectance image (lighting effects removed) to generate new textures reflectance images

  22. Light Source Modification • Insertion of a virtual light source • computation for every pixel • new form-factors • new visibility • Indirect illumination: radiosity solution

  23. Lighting Modification Original virtual lighting Insertion of virtual light

  24. Video

  25. Conclusion • Input • data simple to acquire • Pre-process • data structures optimised for fast updates • Interactive modification • add and move virtual objects • remove real objects • relighting

  26. Future Work • Improve reflectance computation • use of high dynamic range images (instead of RGB) • better control of indirect illumination • Allow motion of real objects • Faster: parallel computation

  27. Future Work • Remove restrictions • diffuse reflectance [Yu et al. 99] • fixed view-point

More Related