1 / 52

Exploiting Temporal Coherence for Incremental All-Frequency Relighting

Exploiting Temporal Coherence for Incremental All-Frequency Relighting. Our Method. Ng et al. 2003. 30 Wavelet lights per frame. Columbia University. CG Lighting Design. Movies. Video Games. Unreal Championship 2 2004 ( www.unrealchampionship2.com ).

nairi
Download Presentation

Exploiting Temporal Coherence for Incremental All-Frequency Relighting

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Exploiting Temporal Coherence for Incremental All-Frequency Relighting Our Method Ng et al. 2003 30 Wavelet lights per frame Columbia University

  2. CG Lighting Design Movies Video Games Unreal Championship 2 2004 (www.unrealchampionship2.com) The Lord of the Rings: The Two Towers 2002 (www.lordoftherings.net) Star Wars Episode I 1999 (thecia.com.au/reviews/s/star-wars-1.shtml) Doom 3 2004 (www.doom3.com)

  3. CG Lighting Design: Why is it hard? Specularities / Reflections Hard Shadows • Complex Lighting • Complex Materials • Takes Hours to Render • Need Interactivity • PRT • Sloan et al. 2002 • Ng et al. 2003 Soft Shadows Caustics

  4. PRT Relighting: Matrix-Vector Multiply Slides from Ng et al. SIGGRAPH 2003

  5. PRT Relighting: Matrix-Vector Multiply • Input Lighting(Cubemap Vector) • Output Image(Pixel Vector) • TransportMatrix Slides from Ng et al. SIGGRAPH 2003

  6. Light-Transport Matrix Columns Slides from Ng et al. SIGGRAPH 2003

  7. Light-Transport Matrix Columns Slides from Ng et al. SIGGRAPH 2003

  8. Matrix Multiplication is Enormous • Dimension • 512 x 512 pixel images ( ) • 6 x 64 x 64 cubemap ( ) • Full matrix-vector multiplication is intractable • On the order of 1010 operations per frame • PRT exploits coherence to enable real-time rendering Slides from Ng et al. SIGGRAPH 2003

  9. PRT: Exploiting Coherence Temporal Coherence [Our Contribution] Image / Vertex Colors Lighting Vector Transport Matrix Signal / Spatial Coherence [Sloan et al. 2003] [Liu et al. 2004] Angular Coherence [Ng et al. 2003] 30 – 100 Wavelet Lights

  10. Previous Work: PRT • Dorsey, J., Arvo, J., and Greenberg, D. 1995.Interactive Design of Complex Time-Dependent Lighting. In IEEE Computer Graphics and Applications, 15(2): 26-36. • Ng R., Ramamoorthi R., Hanrahan P.All-frequency shadows using non-linear wavelet lighting approximation.ACM TOG(SIGGRAPH 03) 22, 3 (2003), 376-381. • Sloan, P., Kautz, J., Snyder, J.Precomputed Radiance Transfer for Real-Time Rendering in Dynamic, Low-Frequency Environments. In Proceedings of SIGGRAPH 2002 • Ng R., Ramamoorthi R., Hanrahan P.Triple product wavelet integrals for all-frequency relighting.ACM TOG (SIGGRAPH 04) 23, 3 (2004), 475-485. • Sloan P., Hall J., Hart. J, Snyder J.Clustered principal components for precomputed radiance transfer.ACM TOG (SIGGRAPH 03) 22, 3 (2003), 382-391. • Sloan P., Luna B., Snyder J.Local, deformable precomputed radiance transfer. ACM TOG (SIGGRAPH 05) 24, 4 (2005), 1216-1224. • Wang R., Tran J., Luebke D.All-frequency relighting of non-diffuse objects using separable BRDF approximation.In EGSR (2004), pp.345-354. • Wang R., Tran J., Luebke D.All-frequency interactive relighting of translucent objects with single and multiple scattering.ACM TOG (SIGGRAPH 05) 24, 3 (2005), 1202-1207. • Zhou K., Hu Y., Lin S., Guo B., Shum H.Precomputed shadow fields for dynamic scenes.ACM TOG (SIGGRAPH 05) 25, 3 (2005). • Ben-Artzi A., Overbeck R., Ramamoorthi R.Real-time BRDF editing in complex lighting. ACM TOG (SIGGRAPH 06) (2006). This Session Wang R., Luebke D., Humphreys G., Ng R.Efficient wavelet rotation for environment map rendering. In EGSR (2006). Kontkanen J., Turquin E., Holzschuch N., Sillion F.Wavelet radiance transport for real-time indirect lighting. In EGSR (2006) Einarsson P., Chabert C., Jones A., Lamond B., Ma A., Hawkins T., Sylwan S., Debevec P.Relighting human locomotion with flowed reflectance fields.In EGSR (2006)

  11. Previous Work: PRT • Dorsey, J., Sillion, F., and Greenberg, D. 1991.Design and simulation of opera lighting and projection effects. In Computer Graphics (Proceedings of SIGGRAPH 91), vol. 25, 41-50. • Ng R., Ramamoorthi R., Hanrahan P.All-frequency shadows using non-linear wavelet lighting approximation.ACM TOG(SIGGRAPH 03) 22, 3 (2003), 376-381. • Sloan, P., Kautz, J., Snyder, J.Precomputed Radiance Transfer for Real-Time Rendering in Dynamic, Low-Frequency Environments. In Proceedings of SIGGRAPH 2002 • Ng R., Ramamoorthi R., Hanrahan P.Triple product wavelet integrals for all-frequency relighting.ACM TOG (SIGGRAPH 04) 23, 3 (2004), 475-485. • Sloan P., Hall J., Hart. J, Snyder J.Clustered principal components for precomputed radiance transfer.ACM TOG (SIGGRAPH 03) 22, 3 (2003), 382-391. • Sloan P., Luna B., Snyder J.Local, deformable precomputed radiance transfer. ACM TOG (SIGGRAPH 05) 24, 4 (2005), 1216-1224. • Wang R., Tran J., Luebke D.All-frequency relighting of non-diffuse objects using separable BRDF approximation.In EGSR (2004), pp.345-354. • Wang R., Tran J., Luebke D.All-frequency interactive relighting of translucent objects with single and multiple scattering.ACM TOG (SIGGRAPH 05) 24, 3 (2005), 1202-1207. • Zhou K., Hu Y., Lin S., Guo B., Shum H.Precomputed shadow fields for dynamic scenes.ACM TOG (SIGGRAPH 05) 25, 3 (2005). • Ben-Artzi A., Overbeck R., Ramamoorthi R.Real-time BRDF editing in complex lighting. ACM TOG (SIGGRAPH 06) (2006). Temporal Coherence [Our Contribution] This Session Wang R., Luebke D., Humphreys G., Ng R.Efficient wavelet rotation for environment map rendering. In EGSR (2006). Kontkanen J., Turquin E., Holzschuch N., Sillion F.Wavelet radiance transport for real-time indirect lighting. In EGSR (2006) Einarsson P., Chabert C., Jones A., Lamond B., Ma A., Hawkins T., Sylwan S., Debevec P.Relighting human locomotion with flowed reflectance fields.In EGSR (2006)

  12. Our Method

  13. Outline • Motivation / Previous Work • Basic Approach to Incremental Relighting • Analysis of Temporal Coherence in Lighting • Per-Band Incremental • Results

  14. Basic Incremental Algorithm

  15. Basic Incremental Algorithm

  16. Basic Incremental Algorithm More Compressible

  17. Basic Incremental Algorithm

  18. Basic Incremental

  19. Basic Incremental: Problems Frame 0 Incremental Reference

  20. Basic Incremental: Problems Frame 30 Incremental Reference

  21. Basic Incremental: Problems Frame 75 Incremental Reference Ghost Shadows

  22. Basic Incremental: Problems Frame 125 Incremental Reference

  23. Basic Incremental: Problems Frame 400 Incremental Reference

  24. Outline • Motivation / Previous Work • Basic Approach for Incremental Relighting • Analysis of Temporal Coherence in Lighting • Per-Band Incremental • Results

  25. Frequency Analysis of Temporal Coherence Medium Frequency High Frequency Low Frequency

  26. Frequency Analysis of Temporal Coherence Medium Frequency High Frequency Low Frequency

  27. Frequency Analysis of Temporal Coherence Medium Frequency High Frequency Low Frequency

  28. Frequency Analysis of Temporal Coherence Medium Frequency High Frequency Low Frequency

  29. Frequency Analysis of Temporal Coherence Medium Frequency High Frequency Low Frequency

  30. Frequency Analysis of Temporal Coherence Medium Frequency High Frequency Low Frequency Temporal Wavelet Transform

  31. Frequency Analysis of Temporal Coherence Medium Frequency High Frequency Low Frequency Temporal Wavelet Transform

  32. Frequency Analysis of Temporal Coherence 100 % ENERGY Temporal Frequency 0 % Angular Frequency

  33. Outline • Motivation / Previous Work • Basic Approach for Incremental Relighting • Analysis of Temporal Coherence in Lighting • Per-Band Incremental • Results

  34. Per-Band Incremental (PBI) +  B = T L 1 1 B = T L +  2 2 B = T L 3 3 B B B 1 2 3 B = T L Incremental Incremental Non-Incremental B = + +

  35. Oracle (Incremental or Not) • Exhaustive • Try all combinations over all wavelet bands. • Very Slow. • Simple • Compare L1 Error in each band individually.

  36. Oracle (Incremental or Not) • Exhaustive • Try all combinations over all wavelet bands. • Very Slow. • Simple • Compare L1 Error in each band individually. L1 Distance L1 Distance Non-Incremental Incremental

  37. Oracle (Incremental or Not) • Exhaustive • Try all combinations over all wavelet bands. • Very Slow. • Simple • Compare L1 Error in each band individually. • Almost zero overhead. • Comparable results to Exhaustive. <or> L1 Distance L1 Distance Non-Incremental Incremental

  38. PBI vs. Basic Incremental

  39. Outline • Motivation / Previous Work • Basic Approach for Incremental Relighting • Analysis of Temporal Coherence in Lighting • Per-Band Incremental • Results

  40. Results

  41. Results

  42. Results 16 15 14 13 12 70 85 55 Per-Band Incremental 30 Wavelets Simple Exhaustive Percentage L1 Error Frame

  43. Results

  44. Results

  45. Results

  46. Results

  47. Per-Band Incremental • 3x – 4x Speed / Quality Improvement • Progressively Convergent • Minimal Overhead

  48. Minimal Overhead • ~ 100 Lines of Code (Pseudo-code in paper). • < 10 % Memory Overheads • Speed: • Average case (30 wavelets): 5 % Overhead

  49. Applies to All (?) Wavelet PRT Frameworks • Old PRT • Standard All-Frequency PRT [Ng et al. 2003] • Current PRT • Clustered PCA [Liu et al. 2004] • Changing View with Separable BRDF Approximation [Wang et al. 2004] • Future PRT • Real-time BRDF Editing [Ben-Artzi et al. SIGGRAPH 2006]

  50. PBI: CPCA and Complex BRDFs

More Related