lis512 lecture 4

1. lis512 lecture 4 XML: documents and records

2. up until now • Relational databases can store information that is internal to an organization. • But a lot of information has to related to the outside world. • This is when other considerations come in.

3. two basic types • There are two basic types of outside communication tools • records sets • documents • It's difficult to separate them precisely, but let's say that records are much more precisely defined.

4. general outside communication • Traditional communication has mainly been achieved through issuing of documents. • Example: a court issues a judgment on a case. • Most documents contain character data. • But they also contains something else. • That's where markup comes in.

5. special outside communication • In special cases, organizations make records available to other. • These records have a format that allows others to process them to • That format is quite rigid and usually purpose-built.

6. metadata • Metadata is another form of records. • The term metadata is usually defined as “data about data”. As such it is controversial what is metadata and what is data. • As far as we are concerned metadata are records that are attached to documents.

7. metadata example mail • If you send and receive email, you will sometimes see what is knows an email headers. • These collection of fields are of the form attribute: value. • Example on next slide

8. From krichel@openlib.org Sun Jul 12 14:55:16 2009 Date: Sun, 12 Jul 2009 14:55:16 +0700 From: Thomas Krichel <krichel@openlib.org> To: krichel@lilrc.org Message-ID: <20090712075516.GA25777@trabbi.openlib.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Envelope-to: Thomas Krichel <krichel@openlib.org> Return-Path: Thomas Krichel <krichel@openlib.org> User-Agent: Mutt/1.5.18 (2008-05-17) Status: RO Content-Length: 5 Lines: 1

9. metadata example: http headers • HTTP/1.1 200 OK • Date: Wed, 24 Feb 2010 17:34:33 GMT • Server: Apache/2.2.14 (Debian) • Last-Modified: Sun, 13 Dec 2009 08:03:42 GMT • ETag: "5f8271-f76-47a9798613380" • Accept-Ranges: bytes • Content-Length: 3958 • Connection: close • Content-Type: text/html

10. example id3v1 • A fixed 128 byte format. • header 3 bytes "TAG" • title 30 bytes of the title • artist 30 bytes of the artist name • album 30 bytes of the album name • year 4 byte year • comment 30 bytes • zero-byte 1 If a track number is stored, this byte contains a binary 0. • track 1 The number of the track on the album, or 0. • genre 1 Index in a list of genres, or 255

11. MARC • MARC is as important example of a record format used in by the library community • Integrated Library Systems (ILSs) all • import MARC records into relational database system • export MARC records from relational database systems • MARC records describe records from library catalogs.

12. MARC format • The MARC format is very complicated. • The basic structure is • Leader • Directory • Variable Control Fields • Variable Data Fields

13. MARC leader • Described in http://www.loc.gov/marc/bibliographic/bdleader.html • When they talk about character, they mean a byte.

14. MARC directory • The MARC directory follows the leader. • I am not sure what it’s purpose is. • The general record structure is at http://www.loc.gov/marc/specifications/specrecstruc.html

15. MARC variable fields • In MARC all field names are numbers. There are three digits to each fields. • Numbers that start with 00 are for fields that are called control fields. • Fields that start with 0 are numbers and control fields. • Fields that do not start with 0 are the main field we study in cataloging.

16. field indicators • Each field other than those starting with 00 can have zero, one or two field indicators. • The field indicator says something additional about the field.

17. subfields • Fields other than the one’s starting with 00 admit subfields. • A subfield is identified by a letter a to z.

18. markup • Markup is the information contained in a document that is not its contents. • Markup mainly comes with two types of information • information related to the structure • information related to the appearance • In good documents, structure and appearance are related.

19. if there where no markup • If markup would not exist, it would be quite trivial to represent every document with a relational database structure. • You simply have a table with character positions (first character position to last character position) and the character found there. • But this would hardly correspond to our idea of a document.

20. structure • The structure of a document is a bit difficult to define, but easy to understood by example. • In a printed document, the sequence of pages defines one structure. • But if the book has chapters and sections, they to define structures, and so do index page, title pages etc. • A database tableu representation of this becomes messy.

21. structure • The structure of a document is a bit difficult to define, but easy to understood by example. • In a printed document, the sequence of pages defines one structure. • But if the book has chapters and sections, they to define structures, and so do index page, title pages etc. • A database table representation of this becomes messy.

22. appearance • Appearance is usually used to communicate the structure of document in a way that aids a human to understand the structure. • For example, look at this slide. We can conside that it is a document. Find way in which the appearance communicates the structure.

23. appearance • Appearance covers things such as • fonts used • background and foreground colors • positioning of structural elements • If a document has some appearance and structure, it is tough to adapt it to a relational database structure.

24. XML • XML is a syntax to encode information as documents. • XML is not really a language since it has no vocabulary. • You can use any vocabulary you like.

25. XML nodes • XML is written in the form of nodes. I will only discuss three types of nodes here • character data • XML elements • attributes to elements • Character data as just that: characters.

26. XML elements • If you write an element, write something of the form. • <name>contents</name> • here name is the name of the element and contents is the contents of the element. • The contents can be character data and or other elements.

27. XML tags • <name> is the start tag of an element that is called name. • </name> is the end tag of an element that is called name. • XML tags a syntactic feature of XML. They are not nodes.

28. empty elements • If an element has no contents whatsoever, it can be written as <foo></foo> • or <foo/> • in the latter case it is an empty element

29. element examples • <name>Thomas Krichel</name> • <name>Mr. <first>Thomas</first> <last>Krichel</last></name> • <thomaskrichel/> • <foo><bar>hello world</bar></foo>

30. child elements • If an element is in the contents of another element, it is called a child element. • When you write an XML document all elements much be children of one single element. That single element is the called the root element.The root element is the only element without a parent element.

31. attributes • Attributes attach name=value pairs to element. • These attribute value pairs appears written at the start stage

32. attribute examples • <name type="full">Thomas Krichel</name> • <name type="reverse">Krichel, Thomas</name> • <name string="Thomas Krichel"/>

33. more on attributes • Attributes names and values are strings. • Attribute values are surrounded by single or double quotes. • Attributes names are separated from values by the = sign.

34. XML application examples • HTML is the language used to encode a specific type of documents known as a web page. • It has a vocabulary on element names and attribute names. • HTML is written in XML syntax or a syntax that is close to it.

35. example HTML element <a> • The <a> element creates an anchor. This is a part of the document that leads to another. • Where it leads to is given by an attribute called href. Example • <a href="http://openlib.org/home/krichel"> Thomas Krichel</a>

36. example HTML element <img/> • The HTML element <img/> requests an image to be included in the web page • <imgsrc="http://openlib.org/home/krichel/ToK.gif" alt="picture of Thomas Krichel"/> • Note that this element is empty.

37. MARC XML • In order to increase the interoperability of MARC defined a mapping of the MARC format into the XML syntax. • Not everybody thinks it is a good idea. http://serials.infomotions.com/ngc4lib/archive/2009/200909/1450.html • A shamelessly copied example is at http://wotan.liu.edu/home/krichel/courses/lis512/external_doc/sandburg.xml

38. start of the example <collection> <record> <leader>01142cam 2200301 a 4500</leader> <controlfield tag="001"> 92005291 </controlfield> <controlfield tag="003">DLC</controlfield> <controlfield tag="005">19930521155141.9</controlfield> <controlfield tag="008">920219s1993 caua j 000 0 eng </controlfield> <datafield tag="010" ind1=" " ind2=" "> <subfield code="a"> 92005291 </subfield> </datafield>

39. end of the example <datafield tag="650" ind1=" " ind2="1"> <subfield code="a">Visual perception.</subfield> </datafield> <datafield tag="700" ind1="1" ind2=" "> <subfield code="a">Rand, Ted,</subfield> <subfield code="e">ill.</subfield> </datafield> </record> </collection>

40. comments on example • In an XML document, there must be one element that all other elements are children of. • In this case this is the <collection> element. • The <collection> can contain many <record> elements. In the example, there is just one. • Find the features of MARC as set out in the description of MARC.