1 / 27

Computational and statistical problems for the Virtual Observatory

Computational and statistical problems for the Virtual Observatory. With contributions from/thanks to: GAVO team: Wolfgang Voges, Matthias Steinmetz, Harry Enke, Hans-Martin Adorf Joerg Colberg (NVO@UPitt), Pat Dowler (CVO), Tony Banday (MPA), Class X team. Overview. Intro to VO

mahdis
Download Presentation

Computational and statistical problems for the Virtual Observatory

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Computational and statistical problems for the Virtual Observatory With contributions from/thanks to: GAVO team: Wolfgang Voges, Matthias Steinmetz, Harry Enke, Hans-Martin Adorf Joerg Colberg (NVO@UPitt), Pat Dowler (CVO), Tony Banday (MPA), Class X team CMU-CS lunch talk, Gerard Lemson

  2. Overview • Intro to VO • IVOA standards process • Some concrete examples, demos • Scenarios, science cases • Interesting problems CMU-CS lunch talk, Gerard Lemson

  3. Intro to VO • Very large data sets • Multi-wavelength astronomy made easy • Federation of distributed archives. • Publication of expert services. • New software developments. • Why contribute ? • Too easy to do bad science ? CMU-CS lunch talk, Gerard Lemson

  4. CMU-CS lunch talk, Gerard Lemson

  5. IVOA standards and specifications • Collaboration of national VOs • Develop standards for interoperability • publication (registry) • description (dm, ucd) • query (dal, voql) • data transfer (votable) • services (grid/web services) • Interest groups: • architecture • applications • theory CMU-CS lunch talk, Gerard Lemson

  6. Babylonian confusion CMU-CS lunch talk, Gerard Lemson

  7. VO domain model as Esperanto CMU-CS lunch talk, Gerard Lemson

  8. CMU-CS lunch talk, Gerard Lemson

  9. CMU-CS lunch talk, Gerard Lemson

  10. Protocols • VOTable + UCD  DM based XML + XSLT • SCS/SIAP/SSAP  ADQL  VOQL • SkyNode • Registry resource model and harvesting interface CMU-CS lunch talk, Gerard Lemson

  11. Data models • Targeted “small” data models • Quantity • Observation • Simulation • Domain model as ontology • Meta-data repository • Bindings • Representations, views, transformations CMU-CS lunch talk, Gerard Lemson

  12. CMU-CS lunch talk, Gerard Lemson

  13. CMU-CS lunch talk, Gerard Lemson

  14. Theory in the VOWith Joerg Colberghttp://ivoa.net/pub/papers/TheoryInTheVO.pdf • Spatial query protocols irrelevant • No object-based federation • New phenomena/observables. • Different kind of provenance. • Model dependency. • Theoretical archives rather unstructured. • Theory/observational interface. CMU-CS lunch talk, Gerard Lemson

  15. Observed Simulated Thanks to Volker Springel Thanks to Alexis Finoguenov, Ulrich Briel, Peter Schuecker, MPE) CMU-CS lunch talk, Gerard Lemson

  16. Some concrete efforts • NVO (USA): Registry (DIS), ADQL, SkyNode, data mining (UPitt+CMU) • AstroGrid (UK): grid/web services, work flows • AVO (ESO, CDS, AstroGrid): Aladin visualization tool, science demos • CVO (Canada): archive federation • France VO: GalICS • GAVO (Germany): data publication (RASS photons), application prototypes, data mining, theory CMU-CS lunch talk, Gerard Lemson

  17. Scenarios, use cases, results • Registry based data discovery and retrieval (GAVO, DIS) • Class X classifier and generalizations • X-Ray cluster analysis using simulations • Cluster detection by combining SDSS and RASS catalogues (Schuecker et al, astro-ph/0403116) • Discovery of obscured quasars using VO tools (Padovani et al, astro-ph/0406056) CMU-CS lunch talk, Gerard Lemson

  18. Typical workflow CMU-CS lunch talk, Gerard Lemson

  19. Download manager CMU-CS lunch talk, Gerard Lemson

  20. ClassX@GAVO CMU-CS lunch talk, Gerard Lemson

  21. Theory/observational interface: X-Ray clusters Goal: interpret observations of X-Ray cluster using results of hydro simulations: • Extract parameters from the observation (services) that can be queried directly (dm, ucd). • Find simulations that may be relevant, that are “similar” to observation by searching registry for hydro simulations of clusters (registry, voql). Requires simulation results to be published and described in sufficient detail (dm, ucd). • Observe simulations using “virtual telescope” (application, grid/webservices) configured according to telescope configuration extracted from observation (dm). • Compare real with virtual observation (services). • For interesting simulation, extract full simulation result (dal) for further analysis, • or analyse the simulation using services (grid-services) provided by the archive or some other service provider CMU-CS lunch talk, Gerard Lemson

  22. CMU-CS lunch talk, Gerard Lemson

  23. Computational, statistical and astronomical challenges I Data models • Data modeling • Data model transformations, views • Archive structure • Database tuning Querying, matching • Distributed query algorithms • Probabilistic matchers, systematic errors, identification of moving sources • Improve identification using full point process information • Add physical properties, not just position, to identification • Complex, frequency dependent source definition • Characterization of complex results in "few" parameters for discovery (PCA (after transformation)? 3D->2D ?) • Comparison of real and virtual observations CMU-CS lunch talk, Gerard Lemson

  24. Usage • Complex model • Simplify using view concept • Example from RDB • XSLT for translation between domain XSD and application-specific derived schemas. CMU-CS lunch talk, Gerard Lemson

  25. CREATE VIEW SEXTRACTOR_GALAXIES AS SELECT S.RA AS _RAJ2000, S.DEC AS _DECJ2000, -2.5 * LOG(S.FLUX) AS M_APP, S.CLASSIFICATION, I.STORAGE_URL AS IMAGE FROM SOURCE S, SOURCE_CATALOGUE SC, IMAGE I, SOURCE_EXTRACTOR AS SE WHERE S.CLASS = ‘GALAXY’ AND S.FLUX < 15 AND S.CATALOGUE_ID = SC.ID AND IMAGE.ID = SC.IMAGE_ID AND SC.EXTRACTED_WITH = SOURCE_EXTRACTOR.ID AND SE.IDENTIFIER = ‘SExtractor’ CMU-CS lunch talk, Gerard Lemson

  26. Probabilistic cross matching CMU-CS lunch talk, Gerard Lemson

  27. Computational, statistical and astronomical challenges II Data mining • Algorithms for analyzing generic SEDs (classifiers ? visualization ? incorrect identification ?) • Source extraction using multiple images, at very different wavelengths, how to take into account different physics/images of same source at different wavelengths ? • Cluster finders using multiple catalogues • Publish sophisticated statistical analysis algorithms Implementation • Efficient implementation virtual telescopes (parallel, distributed, grid based, data structures) CMU-CS lunch talk, Gerard Lemson

More Related