1 / 27

Pilot presentation RICH

Pilot presentation RICH. Introduction. RICH = Reading Images for the Cultural Heritage Two pilot projects Application for semi-automatic dendrochronology Content-based image retrieval for historical glass Today, focus on the latter project. Introduction.

bambi
Download Presentation

Pilot presentation RICH

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Pilot presentation RICH Nederlandse Organisatie voor Wetenschappelijk Onderzoek

  2. Introduction • RICH = Reading Images for the Cultural Heritage • Two pilot projects • Application for semi-automatic dendrochronology • Content-based image retrieval for historical glass • Today, focus on the latter project Nederlandse Organisatie voor Wetenschappelijk Onderzoek

  3. Introduction • Classification of archaeological artefacts • Now performed manually by experts • Expert compares artefact with objects from reference collection • Reference collections consist of drawings in books • Thus: slow, subjective, and error-prone process Nederlandse Organisatie voor Wetenschappelijk Onderzoek

  4. Example • An archaeological artefact: Nederlandse Organisatie voor Wetenschappelijk Onderzoek

  5. Example • And its corresponding drawing:  Nederlandse Organisatie voor Wetenschappelijk Onderzoek

  6. The task • Given an artefact photograph • Find the most ‘similar’ drawings • Speeds up the classification process • Can give archaeological experts new insights Nederlandse Organisatie voor Wetenschappelijk Onderzoek

  7. The problem • Drawings contain no color information • Drawings contain only very abstract texture representation • Texture hard to extract from glass photographs • Thus: only outer shape information is accurate • Shape matching Nederlandse Organisatie voor Wetenschappelijk Onderzoek

  8. Shape matching • Several approaches in literature: • Shape contexts (Belongie, 2000) • Curvature scale spaces (Mokhtarian, 1996) • Turning functions (Tanase, 2003) • Dynamic programming (Petrakis, 2002) • Moment invariants (Hu, 1962) • Hausdorff, Procrustes, etc. • MPEG-7 standard: Curvature scale spaces Nederlandse Organisatie voor Wetenschappelijk Onderzoek

  9. Shape matching • For today, focus on: • Shape contexts • Curvature scale space • Desired properties of approach: • Invariant to scale, translation, and rotation • Robust to distortions due to • Broken artefacts • 3D rotations • Drawing interpretations Nederlandse Organisatie voor Wetenschappelijk Onderzoek

  10. Broken artefacts  Nederlandse Organisatie voor Wetenschappelijk Onderzoek

  11. Or worse… Nederlandse Organisatie voor Wetenschappelijk Onderzoek

  12. 3D rotations  Nederlandse Organisatie voor Wetenschappelijk Onderzoek

  13. Shape contexts • Sample points from outer contour • For all points: • Compute angle (relative to baseline) and distance to all other points in coarsely discretized log-polar space • All resulting histograms form the shape representation Nederlandse Organisatie voor Wetenschappelijk Onderzoek

  14. Shape contexts • Matching using startpoint invariant k-NN classifier (using χ2-distance) • Startpoint invariance obtained by circular shifting one of the histograms Nederlandse Organisatie voor Wetenschappelijk Onderzoek

  15. Curvature scale space • Determine positions of zero-crossings of curvature for an ‘evolving’ shape contour • Curvature is a function that is 0 for a straight line, and 1 / r for a circle with radius r • Shape evolution: convolve coordinates with 1D Gaussian kernel with increasing variance Nederlandse Organisatie voor Wetenschappelijk Onderzoek

  16. Shape evolution • Evolving shape with curvature zero-crossings: Nederlandse Organisatie voor Wetenschappelijk Onderzoek

  17. Curvature scale space • CSS image: • Align CSS by aligning global maximum • Sum distances between main peaks Nederlandse Organisatie voor Wetenschappelijk Onderzoek

  18. Results • Low identification performance (as expected) due to difficult dataset Nederlandse Organisatie voor Wetenschappelijk Onderzoek

  19. Results • We examined various variations, such as quantization in shape context space, etc. • Best performance: 33% for hitlist size 10 Nederlandse Organisatie voor Wetenschappelijk Onderzoek

  20. Results • However: • For ‘good’ artefacts results are encouraging • Shape analysis on reference collection allows for making ‘shape maps’ (using MDS) • This allows for archaeologists to create new typologies (since typologies are not ‘fixed’) • Allows for new methods of presenting collections • Good results expected for flint data Nederlandse Organisatie voor Wetenschappelijk Onderzoek

  21. Example query  Nederlandse Organisatie voor Wetenschappelijk Onderzoek

  22. Applications • Matlab application (local) Nederlandse Organisatie voor Wetenschappelijk Onderzoek

  23. Applications • Navigation structure for collection presentation • Precalculated and stored in database http://www.referentiecollectie.nl/richglas Nederlandse Organisatie voor Wetenschappelijk Onderzoek

  24. Applications • Web-based CBIR tool • Servlet running on webserver (using Tomcat) • User uploads photograph to webserver • Sends query image to UM calculation server (using RMI) • RMI server executes original Matlab-scripts (using JMatLink) • Results are sent back to servlet • Servlet generates result pages • Advantages: no local calculations, no porting of code Nederlandse Organisatie voor Wetenschappelijk Onderzoek

  25. Conclusions • Matching glass artefacts with drawings is a difficult problem • Shape context matching outperforms (MPEG-7 standard) CSS matching • Allows for shape analysis of reference collection • Number of (preliminary) applications delivered • We expect shape matching to be usefull for flint data • Possible improvements: • Incorporating texture features • Design of shape matching methods for partial shape matching using closed contours Nederlandse Organisatie voor Wetenschappelijk Onderzoek

  26. Questions ? Nederlandse Organisatie voor Wetenschappelijk Onderzoek

More Related