Component Software Architecture

1. Professional Content Management Systems7th Lecture: CMS Software Modules Dr. Andreas MautheSCC – Lancaster University

2. Application Plane Session Layer Session Manager Job Mgmt SystemAdministration Application Layer System Mgmt Plan&Draft Logging Browsing Export Cataloging Import Admini-stration Configuration Service Remote Installation Service User Management Accounting&Licensing Service Messaging Service Resource Broker Ingest Retrieval Editing Clearance Naming Service Event Log Service Process Monitor Commis-sioning Workflow Engine Task Management Service Transaction Service Services Plane Ingest Service IndexingService ImportService Water- marking Service Web Service Analysis Service Export Service Editing Service Conversion Service Ordering Service Core Plane Distribution Layer Essence Broker DeviceBroker Data Broker Abstraction Layer EssenceManager EssenceManager EssenceManager DeviceManager DataManager DataManager DataManager Foundation Layer Archive Servers Streaming Servers DeviceServer Legacy System DBMS Full Text Search ImageSimilaritySearch AudioSimilaritySearch Cache Servers Transfer Servers Component Software Architecture

3. Device Manager • Tasks • Control of external devices • IT based access to broadcast devices • E.g. to devices that are only controllable via RS-422

4. Device Servers • Tasks • Unified control interface to various external devices • Functionalities • File system interface • Device control interface, enabling remote control of the device via network • Inventory management, storing and providing metadata and status information • Event handling (e.g. such as appearance, deletion or status changes of content) • Cache management

5. Data Manager (DM) • Tasks • Access to all information systems and databases • Support for different • IS and DB • Search engines

6. Data Management Elements • Information Systems • Legacy Databases • Existing IS and data storage systems • News room systems, keeping rundowns, scripts, etc., • Systems supporting planning and drafting (synopses, stories, etc.), • The management systems of various production systems (EDLs, titles, descriptions, etc.), • Studio automation systems (play lists, as–run–logs, etc.) • Integration possibilities • Mapping of legacy data model into CMS data model • Problem: overhead • Standard Data Manager enables a set of defined queries and reports  dedicated Data Manager Required • CMS Databases • Hosts the CMS data model • E.g. SMEF or equivalent according to organisational roles • Organisation dependent • Full Text Search • Via full text search engine • Operating on DB or indexed files • Requirements onto the search engine • Configurable parser • Support for attribute based retrieval and sorting • Support for limiting searches to ranges of numbers • API for access to retrieval functionality and highlighting information.

7. Data Management Elements II • Information Systems (cont.) • Image Similarity Search • Based on keyframe sets • Comparing image features • Low-level features, e.g. colour, texture, shapes, etc. • Issues • User expectation •  Image identity • Scalability: 100,000h x 1,000 = 100,000,000 in 10 sec!! • Audio Similarity Search • Using audio samples in audio and video • Similar problem space • Data Managers • Integration hook for IS • Have to fully abstract from the IS specific • Query language • Data model • Structure • Generic part • Interface to other CMS components • Specific part • Wrapper around IS • Using standard exchange format • E.g. XML

8. Data Management Elements III • Data Manager Resolver • Task • To map IS system proprietary IDs to CMS system-wide ID • To identify content objects in the system • Mapping table that contains a • Line that contains all known system IDs for each content object • Column that stores the CMS-wide ID for each content object • Column for each Data Manager that stores the the system ID of the respective object the Data Manager interfaces to •  CMS ID from DM & local ID; local ID from CMS ID & DM • Data Brokers • Task • Distribution of requests to multiple Data Managers • Selects relevant DM for a specific query • Merge responses into unified response set • Order and enrich responses if necessary • Based on Broker/ Manager concept • Provides a unique update, query and delete interface • Controls DM that have registered with a broker • Broker can be distributed

9. CMS Services

10. Services: Ingest & Playout • Ingest Service • Task • Controlling Ingest of audiovisual signals • Satellite & cable • Studio devices (tape DAT, CD payer, VCR) • Digitised into multiple formats • E.g. MPEG-1, MPEG-2, Web Format • Capabilities • Identifying the object that the essence should be associated to • Generating unique identifiers • Enforcing entry of minimum metadata; introducing in DM • Monitoring and supervising the encoding process • Identifying and handling files resulting from the encoding • Registering essence with EM while recording • Playout Service • Task • Controlling playout of audiovisual signals at signal level • By selecting suitable decoders • Streaming requests that are not inherent part of the content handling process

11. Import Service • Tasks • Support import of • Single objects • Batch (multiple objects) • Workflow processor to support various requirements • Pre-processor decides if pre-conditions are met • E.g. enough resources available • Processor controls import task • Including movement of files, metadata creation, update, etc. • Conversion into additional formats • Post-processor to complete the task

12. Export Service • Tasks • Support Export of • Single objects • Batch (multiple objects) • Workflow processor to support various requirements • Pre-processor decides if pre-conditions are met • E.g. export area ready • Processor controls import task • Including retrieval or metadata • Conversion into target system format & to support partial export • Post-processor to complete the task

13. Services: Editing & Conversion • Editing Service • Task • Split essence into elementary pieces • Video & audio clips • Stills, graphics, text • Assembling pieces • In conjunction with production system • Editing functions • Simple cuts, • trimming, multiplexing, de-multiplexing • Problems • Inter-frame coding • MPEG base coding • Cut at GOP start • Conversion • Task • Controlling transcoding between different media and file formats • Framework to integrate third-party tools • Relevant conversions • MPEG-2 4:2:2P@ML to DV, MP1 to other formats • MXF to AAF, WAVE to BWF

14. Analysis Service • Video Analysis • Task • Analysing video essence • Shot detection, keyframe selection • Keyframe selection • Temporal location in video stream • Significant change in image content • Reasons • First, last frame in shot • Camera operation or transition • User selected • Transition classification • Cut, Fade-in, Fade-out, Cross-fade,etc. • Shot clustering for indexing (e.g. dialogue scene) • Framework for existing technologies • Audio Analysis • Task • Analysing audio essence and producing additional metadata • Functionality • Classification of segments containing music, speech or other sounds, • Speaker recognition • Speech recognition to derive transcripts • Keyword spotting

15. Services: Watermarking & Web Retrieval • Watermarking Service • Task • Authentication of essence objects • Actual watermarks • Additional technologies • E.g. fingerprinting • Assembling pieces • In conjunction with production system • Wrapper around existing and emerging technologies • Providing of unified interface • For accessing essence • Providing metadata & processed essence • Web Retrieval • Task • Web-base access to content • Provides via standard Web-browser • Simple & advanced query interfaces • Displaying hit-lists, metadata, key-frames • Requirements • Each view provided should be available via URL • Accessing content via URL • User management support • Easily configurable user interface • Scalable and distributable over multiple platforms

16. Services: Order Management & Watch Folder • Order Management • Task • Support mediated research by • Archive staff • Researchers • Orders for the delivery of essence • Based on • Messaging service • Has to associate related messages • Priorities messages • Watch Folder Service • Task • Trigger background processes upon appearance of • File • Event

17. Workspace Management • Task • To organise the user’s work environment • Content objects, queries • Browsing file systems • Structure • Folder structure • Folders belong to folder classes • User interaction triggers processes • E.g. format conversions, video analysis, etc. • Using XML and SOAP for communication

18. CMS Applications • Sessions Management • Provides Application Framework • Verifies user ID and Password • Configuration according to user roles and access rights • Application Integration via ActiveX

19. CMS Supporting Services

20. Job Management: Workflow Management • Task • Construction of complex tasks from basic jobs • E.g. ingest • Specify user workflows • Monitor workflows • Functionality • Workflow tracing production progress • Planned • Approved essence • Available • Rough cut completed • Video edit completed • Colour correction completed • Audio edit completed • Formal cataloguing completed • Full cataloguing completed • Administrators need to • Modify, monitor, remove jobs

21. Job Management: Task Management & Transaction Control • Task Management • Scheduling • Of one-time jobs • Periodically recurring jobs • Job processing • Queuing of ready jobs • Stills, graphics, text • Provides information of running jobs • Distribution of jobs • Core components • Essence Management, Device Management, Data Management • Assembling pieces • In conjunction with production system • Administration interface • Transaction Control • Task • Transaction security of distributed processes • Using existing mechanisms • Distributed Transaction Processing (DTP) • X/Open Standard

22. CMS System Management

23. System Management: Naming & Event Logging • Naming Service • Task • Naming authority in the CMS • Local and remote services • In conjunction with middleware technology • Functionality • Registering of all components with the naming service, • Providing information about the validity a registration • Updating or renewing registration information • Un-registering from the Naming Service. • Event Control • Task • Logging system events • Notifications, warning, errors • Can be centralised or distributed • Functionality • Registering with the event log service • Checking the validity registration • Opening a log channel to write messages to the event log service • Writing messages to this channel • Closing open log channels • Un-registering from the event log service

24. System Management: Process Monitoring & Resource Management • Process Monitoring • Task • Controlling system state • Restarting failing systems • Local and remote services • Backend component need to be able to • Register with the process monitor • Check the validity of its registration at any point in time • Update or renew its registration information • Provide the process monitor with information how to start, pause, continue shutdown and restart itself • Provide the process monitor with status information about its health state • Un-register from the process monitor • Resource Management • Task • Resource reservation, allocation and monitoring • Information provided by services & clients • Service and core components need to register and provide information about resources • Required resources, time span, priority, confidence level

25. CMS System Administration

26. System Administration: User Management & Accounting & Licensing • User Management • Task • Management of • Users • User groups • Defines access rights to • Content • Applications & functionality • Functionality • Controlling access via Access Control Lists (ACL) • ID of owner, ID groups that have specific rights, ID of groups with generic rights • Rights • See, read/access/replay, write/edit/modify, create, delete, copy, change ACL & owner • Accounting & Licensing • Task • Registering access to various part of the system • Control licenses • Compliance with license agreement • Registering access to essence

27. System Management: Messaging, Configuration & Remote Installation • Messaging • Task • To facilitate client-to-client communication • Considering CMS context • Configuration • Task • Providing configuration of all Core and Services components • Also some client components • Central registry • Facilitating reconfiguration • Remote Installation • Task • Remote installation of • Client applications • Services, servers & core components • Download components from central repository • Automatic installation