1 / 39

CASPAR Framework and Lessons Learned

CASPAR Framework and Lessons Learned. David Giaretta. Overview. CASPAR OAIS Threats and Solutions Validation. CASPAR Project. EU FP6 Integrated Project Total spend approx. 16MEuro (8.8 MEuro from EU). http://www.casparpreserves.eu. Digital Preservation.

vevay
Download Presentation

CASPAR Framework and Lessons Learned

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CASPAR Framework and Lessons Learned David Giaretta

  2. Overview • CASPAR • OAIS • Threats and Solutions • Validation

  3. CASPAR Project EU FP6 Integrated Project Total spend approx. 16MEuro (8.8 MEuro from EU) http://www.casparpreserves.eu

  4. Digital Preservation • Ensure that digitally encoded information are understandable and usable over the long term • Long term could start at just a few years • Easy to make claims • Difficult to provide proof • Reference Model for Open Archival Information System (ISO 14721) • The basic standard for work in digital pres. • Defines terminology and compliance criteria

  5. Information Object 1+ interpreted interpreted using Data Representation 1+ using Object Information Physical Digital Object Object 1+ Bit Sequence Information Model & Representation Information The Information Model is key Recursion ends at KNOWLEDGEBASE of the DESIGNATED COMMUNITY (this knowledge will change over time and region)

  6. Basic concept of CASPAR • Digital preservation had been dominated by libraries and (state) archives • However there was a focus there on “rendered objects” and • Tendency to think data is an “easy” add-on HOWEVER • Need to deal with DATA – processed to new things, not just rendered • Need to follow OAIS – finer grained view • Need to test and prove that things work “metadata”

  7. Preservation Strategies • Emulation • Access software • Migration • Transformation • Description techniques

  8. Data… Level 2 GOME Satellite instrument data

  9. Contains numbers – need meaning

  10. ...to process to this

  11. ...or this

  12. ...through complex processing schemes

  13. Just Format? sfqsftfoubujpo jogpsnbujpo svmft You have a file JHOVE tells you it is WORD version 7

  14. ..with some extra information.. representation information rules Format Registries – useful but not enough: formats can be used for multiple purposes e.g. audio files used to store configuration parameters

  15. Examples (cont) • “504b0304140000000800f696….” • “This is a ZIP file which contains Word files, each of which contains an encoded message which needs the key ‘!D$G^AJU*KI’ to decode it using encryption method SHA7”

  16. Examples (cont) • LaTex file containing an EPS (Encapulated Postscript) version of an image • Web page containing Java Applet generating random numbers • SWISS-PROT data • Foreign Language emails

  17. XML enough? – can stare at this and probably understand it <family> <father>John</father> <mother>Mary</mother> <son>Paul</son> </family>

  18. ..but what about this? <VOTABLE version="1.1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.ivoa.net/xml/VOTable/v1.1 http://www.ivoa.net/xml/VOTable/v1.1" xmlns="http://www.ivoa.net/xml/VOTable/v1.1"> <RESOURCE> <TABLE name="6dfgs_E7_subset" nrows="875"> <PARAM arraysize="*" datatype="char" name="Original Source" value="http://www-wfau.roe.ac.uk/6dFGS/6dfgs_E7.fld.gz"> <DESCRIPTION>URL of data file used to create this table.</DESCRIPTION> </PARAM> <PARAM arraysize="*" datatype="char" name="Comment" value="Cut down 6dfGS dataset for TOPCAT demo usage."/> <FIELD arraysize="15" datatype="char" name="TARGET"> <DESCRIPTION>Target name</DESCRIPTION> </FIELD> <FIELD arraysize="11" datatype="char" name="DEC" unit="DMS"> <DATA> <FITS> <STREAM encoding='base64'> U0lNUExFICA9ICAgICAgICAgICAgICAgICAgICBUIC8gU3RhbmRhcmQgRklUUyBm b3JtYXQgICAgICAgICAgICAgICAgICAgICAgICAgICBCSVRQSVggID0gICAgICAg ICAgICAgICAgICAgIDggLyBDaGFyYWN0ZXIgZGF0YSAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgIE5BWElTICAgPSAgICAgICAgICAgICAgICAgICAgMCAv IE5vIGltYWdlLCBqdXN0IGV4dGVuc2lvbnMgICAgICAgICAgICAgICAgICAgICAg

  19. Representation Information The Information Model is key Recursion ends at KNOWLEDGEBASE of the DESIGNATED COMMUNITY (this knowledge will change over time and region)

  20. Representation Information Network

  21. Rep • Info • Virtualisation /DISCIPLINE

  22. FITS FILE MULTIMEDIA PERFORMANCE DATA FITS DICTIONARY FITS STANDARD DICTIONARY SPECIFICATION C3D DirectX MAX/MSP FITS JAVA s/w PDF STANDARD 3D scene data files 3D motion data files motion to music mapping strategy XML SPECIFICATION PDF s/w JAVA VM UNICODE SPECIFICATION Modules and Dependencies:defining the Designated Community README.txt ENGLISH LANGUAGE TEXT EDITOR WINDOWS XP

  23. Cost sharing DRM • USE DATA • Use application to find data in Repository • Create DIP with enough RepInfo for the user (via DC profile) • Obtain more RepInfo from Registry if necessary Preservable infrastructure

  24. RepInfo toolkit, Packager and Registry – to create and store Representation Information. In addition the Orchestration Manager and Knowledge Gap Manager help to ensure that the RepInfo is adequate. Registry and Orchestration Manager to exchange information about the obsolescence of hardware and software, amongst other changes. The Representation Information will include such things as software source code and emulators. Authenticity toolkit will allow one to capture evidence from many sources which may be used to judge Authenticity. Digital Rights and Access Rights tools allow one to virtualise and preserve the DRM and Access Rights information which exist at the time the Content Information is submitted for preservation. Persistent Identifier system: such a system will allow objects to be located over time. Orchestration Manager will, amongst other things, allow the exchange of information about datasets which need to be passed from one curator to another. The Audit and Certification standard to which CASPAR has contributed will allow a certification process to be set up.

  25. As part of the validation the CASPAR tested simulated the following: hardware changes software changes changes in the environment (including legal framework) changes to the knowledge bases of the Designated Communities Accelerated Lifetime tests

  26. Test scenarios vs Threats to digital preservation

  27. STFC Testbed – various STP data

  28. ESA testbed

  29. UNESCO testbed The Villa Livia dataset is a collection of files used within the "virtual museum of the ancient Via Flaminia" project: a 3D reconstruction of several archaeological sites along the ancient Via Flaminia, the largest of them being Villa Livia

  30. This is an elevation grid (height map) of the area where Villa Liva is located. It is an ASCII file in the ESRI GRID file format

  31. Contemporary Art Testbed

  32. Performance Viewer: side-by-side comparison and validation of the transformation. From left to right: 3D visualization in Ogre3D, 3D model of the stage including the virtual dancer in VRML.

  33. Figure 8 Some aspects of acousmatic production

  34. CASPAR Validation • In all cases members of the Designated Community, with appropriate changes to mimic changes over time, verified that the metadata was adequate for the use despite simulated changes of hardware, software, environment and Designated Community over time. • Full details are available in the validation report (CASPAR Validation report, 2009)

  35. Links • CASPAR – http://www.casparpreserves.eu • CASPAR Source code - http://sourceforge.net/projects/digitalpreserve/ • OAIS Reference Model -http://public.ccsds.org/publications/archive/650x0b1.pdf • and the updated draft is available from http://public.ccsds.org/sites/cwe/rids/Lists/CCSDS%206500P11/Overview.aspx • CASPAR Validation report http://www.casparpreserves.eu/Members/cclrc/Deliverables/caspar-validation-evaluation-report/at_download/file • PARSE.Insight: • www.parse-insight.eu • Alliance for Permanent Access: • www.alliancepermanentaccess.eu • Digital Curation Centre: • www.dcc.ac.uk

  36. Persistent ID resolver RepInfo Registry Authenticity tools Processing Context Certification Orchestration/Brokering Knowledge Gap Manager Persistent ID resolver RepInfo Registry Authenticity tools Processing Context Certification Orchestration/Brokering Knowledge Gap Manager Storage Compute Resource Local Authentication Local Authorisation WAN LAN Router Switch Cable Interconnects Gateways Management WAN LAN Router Switch Cable Translators Thesauri Cross-references Discipline repositories Storage Compute Resource Local Authentication Local Authorisation Resource Registries Process ID Scheduler Shibboleth Repositories Users Automated systems Repositories Users Automated systems Discipline repositories Translators Thesauri Cross-references FUTURE • Users may be unable to understand or use the data e.g. the semantics, format, processes or algorithms involved • Non-maintainability of essential hardware, software or support environment may make the information inaccessible • The chain of evidence may be lost and there may be lack of certainty of provenance or authenticity • Access and use restrictions may not be respected in the future • Loss of ability to identify the location of data • The current custodian of the data, whether an organisation or project, may cease to exist at some point in the future • The ones we trust to look after the digital holdings may let us down

  37. END

More Related