1 / 36

Diffuse Reflection Imaging: Earthshine and other Faint Signals

Diffuse Reflection Imaging: Earthshine and other Faint Signals. Sam Hasinoff MIT CSAIL, TTIC, Google[x]. Samuel W. Hasinoff, Anat Levin, Philip R. Goode, and William T. Freeman, Diffuse Reflectance Imaging with Astronomical Applications,

zelda
Download Presentation

Diffuse Reflection Imaging: Earthshine and other Faint Signals

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Diffuse Reflection Imaging: Earthshine and other Faint Signals • Sam Hasinoff • MIT CSAIL, TTIC, Google[x] Samuel W. Hasinoff, AnatLevin, Philip R. Goode, and William T. Freeman, Diffuse Reflectance Imaging with Astronomical Applications, Proc. 13thIEEE International Conference on Computer Vision, ICCV 2011 http://people.csail.mit.edu/hasinoff/diffuse/

  2. Joint work with: • Bill Freeman • MIT CSAIL • AnatLevin • Weizmann Institute • Philip Goode • Big Bear Solar • Observatory Special thanks to: Bernhard Schölkopf, Frédo Durand, LiviaIllie, David Chen

  3. Indirect Imaging with Reflective Objects • specular • reflectance • virtual camera • glossy • reflectance • bad camera? • diffuse • reflectance • really bad camera?

  4. Resolution of a Diffuse Reflector 1 Diffuse surfaces blur together incident lighting from many directions 0 -180° 0° +180° Lambertian reflectance • How much detail can we resolve?

  5. Single-Bounce Light Transport 1 0 -180° 0° +180° surface albedo incoming radiance visibility BRDF clipped cosine reflected radiance . • reflectance = (albedo lighting) BRDF • for convex objects, distant illumination *

  6. Linear Light Transport • Usual matrix formulation observed pixels transfer matrix distant lighting noise transfer matrix encodes geometry, BRDF, surface albedo

  7. Resolution of a Diffuse Reflector • about 9 pixels? • images of a convex Lambertian objects with distant lighting lie in a 9D subspace (<2% average error) • [Basri and Jacobs, 2001] • [Ramamoorthi and Hanrahan, 2001] … e.g. lighting-insensitive recognition with 9 basis images

  8. Key: Occlusion Geometry Codes Illumination convex about 9 pixels? self-occlusion potentially much better

  9. Key: Occlusion Geometry Codes Illumination • What 3D shape lets us best resolve the lighting? convex about 9 pixels? self-occlusion potentially much better

  10. Sculpture Design for Reflectance Imaging • self-occlusion preserves high frequencies • extreme: isolate rays by building pinhole camerasusing the scene • arbitrary fidelity (in theory), limited only by diffraction and geometric calibration double-pinholes • natural complexity • coded reflectance

  11. Bayesian Reconstruction Method Standard MAP estimation • Gaussian prior [Wiener 1949] • Sparse derivative prior [Levin et al. 2007]

  12. Reconstructing a Changing Target For time-varying lighting, reconstruct stable part • marginalize out temporal variations temporal binning with Gaussian prior temporal variations as correlated noise

  13. Approach #1: Rim Reflectance Imaging • Exploit visibility changes near occlusion boundary • works for compact lighting, e.g. object lit head on • multiple orientations = tomography • +single shot enough • - need high resolution • - need accurate geometry not previously used in astronomy or computer vision

  14. Approach #2: Time Varying Imaging • Exploit variation over time in the occlusion geometry • classical astronomy (“light curves”) • +single pixel enough • - need natural variation • - need many shots • - target might change e.g. surface of Pluto from Charon’s 1985-1990transits [Young et al. 1999]

  15. Earth from Space from Earth - Feasibility Study Earth from Apollo 17, 1972 (NASA)

  16. Low-Resolution Expectations NASA’s “Blue Planet” reconstruction we’d be happy with

  17. Moon as Earth Reflector (“Earthshine”) Leonardo Da Vinci’s Codex Leicester (ca. 1510)

  18. (Exaggerated) Geometry of Earthshine Earth north pole sunlight observer Moon from observer Moon north pole

  19. (Exaggerated) Geometry of Earthshine Earth north pole Earthshine sunlight observer Moonshine Moon from observer Moon north pole

  20. Integrating over Earth from the Moon albedo of “unknown” Earth patch cos(incoming) Earth phase (≈ 180° - Moon phase) cos(outgoing) Earth integrand, from point on the Moon (simulation) Aug 8, 2010, 5am Feb 24, 2010, 1am

  21. Earthshine as Linear Transfer Matrix y = T ∙ x + n noise observations of earthshine Earth-Moon transfer matrix Earth image

  22. Recipe for Earthshine Imaging • feasible observing times: • moon >2° above horizon (air mass) • moon <85% full (moonshine) • sun >8° below horizon (twilight) • clear skies • track moon - multi-minute exposures • > -2.5 app. mag. (V) • block moonshine glare with occluder BBSO [Qiuet al. 2003]

  23. Approach #1: Visibility at Moon Rim • From rim of Moon, different Earth regions visible • Special challenges: • need very high resolution, SNR • need detailed moon topography • BRDF at grazing angles? Earthrise, as seen from Apollo 8 (NASA)

  24. Best Case Earth-Bound Moon Imaging 8m telescope ESO Very Large Telescope (Chile) 0.07” resolution 26,000 pixels across lunar disk by comparison, HST is 0.05” (i.e. near diffraction limit for it 2.4m mirror) http://optics.org/article/9654/

  25. Sampling the Earth and Moon Surface moon simulation, 2475 samples (55 per radial line) ground truth earth model (75x30 pixels) relative contribution, Lambertian Earth

  26. Simulated Reconstruction for Single Shot 80 dB 60 dB 40 dB sparse prior reconstruction std. optics (0.4”) adaptive optics (0.07”)

  27. Approach #2: Scanning over Time • Over time, Earth region visible to Moon & Sun changes • Single pixel observations is fine • Special challenges: • 1D set of viewpoints per day • time varying cloud cover, snow, etc. 1am, Feb 24, 2010 5am, Feb 24, 2010 2am, Aug 14, 2010 region of Earth visible from Moon’s disk, i.e. contributing to earthshine

  28. Simulated Reconstruction over Time grd. truth contrib. recon. 1 night Aug 12-13, 2010 107 obs. (@3 min) 1 month Aug, 2010 412 obs. (@10 min) 1 year Jan-Dec, 2010 963 obs. (@60 min)

  29. Simulated Reconstruction over Time grd. truth contrib. recon. 1 night Aug 12-13, 2010 107 obs. (@3 min) 1 month Aug, 2010 412 obs. (@10 min) 1 year Jan-Dec, 2010 963 obs. (@60 min)

  30. Time-Varying Clouds • 50-75% of Earth covered by clouds • large-scale systems last about 3 days earth with ISCCP clouds next day clouds mean clouds const. mean cloud reconstruction naïve time-varying reconstruction covariant reconstruction simulated reconstruction (1 year @3min, 60 dB)

  31. ReconstructingMars from Light Curves • direct observation • outer planet, so less phase change (>85% disk lit) • much less active weather than earth • axial tilt of 25° (2D directions) • more Lambertian? Martian surface

  32. Martian Reconstruction from Historical Spectra relative contribution ground truth + visibility recon. from simulation (scaled) recon. from real data (hVB), 47 dB 234 spectra (single-pixel images) observed at 2 sites, from 1963-1965 [Irivine et al. 1968ab]

  33. Simulation: Limits of Single-Pixel Mars Reconstruction 200 samples 400 samples 800 samples 1600 samples 3200 samples 23731 samples 1 Martian year (= 2 years) single-pixel images, 60 dB, no temporal variation 0.3% noise 0.1% noise 0.03% noise

  34. 10 Years of Moon Photos moonshine ∙ filter • 10 years • 40k+ photos • photos every 5 min • Big Bear Solar Observatory • (BBSO), near LA • single CCD (grayscale) Ib Ia earthshine Moon from BBSO [Qiuet al. 2003]

  35. The Earthshine-Moonshine Ratio irradiance from “unknown” Earth moonshine ∙ filter < 1° Ib lunar phase 0° = full, 180° = new Ia atmosphere earthshine Earth moonshine reference: factors out viewing effects

  36. Coda: Optimizing Capture Strategies • imaging tradeoffs, e.g. blur vs. noise • sensor characteristics vs. 2.8 dB 14.6 dB http://people.csail.mit.edu/hasinoff/

More Related