2002: The Year of Archiving

1. 2002: The Year of Archiving Robert Parkinson PCS Limited “Creating Solutions Through Knowledge”

2. What We’ll Cover • Archiving benefits • Definitions of terms • Archiving features • Archiving steps • Storage and management • Archiving by module • What next?

3. What is Archiving? • Move data no longer required in database to archive files R/3 Database Archiving Program Archive Files Archiving Session

4. Why Archive? • Volume of data in the database increases • Disk space • Performance • Size of index trees, full table scans, cache hit ratios, ... • Effort and resources required for database administration • Backup/restore, reorganization, create index, statistics, ... • Business and legal requirements • Auditing • Record-keeping requirements • Potential re-useability of data

5. Data Continues to Grow • Yearly data growth 400 GB, retention period 6 months • Data not needed online after 5 years: 1,800 GB • Volume without data archiving in 5 years: 1,000,000 GB • Volume with data archiving in 5 years: 225,000 GB Backup, restore and database administration time grows proportional to database size without data archiving

6. Archiving Requirements • Independence • of database version • of version and customizing of application software • of hardware (codepage / numeric format) • Auditability • Revenue authorities – IRS, auditors • Data security officials • Integration in applications • Direct access • Reporting functions • Performance and robustness

7. What We’ll Cover • Archiving benefits • Definition of terms • Archiving features • Archiving steps • Storage and management • Archiving by module • What next?

8. Optical Archiving • ArchiveLink can be used to archive documents • Original documents or print lists can be stored in optical archive systems and displayed, but cannot be reloaded or analyzed • Archive files which were created by R/3 data archiving can be stored, read, and reloaded using ArchiveLink

9. Database Reorganization • Should be performed after archive • Compelling reasons: • Tables keep free space allocated after archive • Gives free space back to the database • Improves performance • Reduces database maintenance times

10. Retention vs. Residence Period • The retention period is the entire time that data spends in the database before its archived • The residence period is the minimum length of time that data must spend in the database before its eligible to be archived

11. Backup & Restore • Contains a copy of the database contents for security • Backups are usually made at regular intervals, according to a standard procedure • Reloading the backup into the database is referred to as restore • In earlier SAP releases, the archiving procedure required a database backup. SAP states this is no longer necessary in Release 3.X and 4.X

12. What We’ll Cover • Archiving benefits • Definition of terms • Archiving features • Archiving steps • Storage and management • Archiving by module • What next?

13. Archiving Features • High data security during archiving • Optical archive connection via ArchiveLink • Data compression • Online data archiving • Archiving without database backup • Random and sequential access to archived data objects, reload to the database possible • Automatic conversion of old archive files • The ADK (Archive Developer’s Toolkit)

14. Security Provided in Archiving • Archiving is a two step process: • Create archive files • Read data from file and execute delete program • Storage of the archive files on other media • Provides data security in case of problems during the archiving process

15. Archive file Step 1 Create the Archive File Archiving program R/3 Data- base

16. Archiving program R/3 Data- base R/3 Data- base Archive file Delete program Archive file Archive file Delete program Archive file Delete program Step 2 Execute Delete Program

17. R/3 Data- base Archive file Archive file Delete program Step 3 Store the Archive Files • ArchiveLink • HSM-Systems

18. R/3 System ArchiveLink Archive system Optical Archive Link ArchiveLink • The archive files can be stored on tertiary storage media • Files can be stored manually or automatically Hierarchical storage management systems (HSM systems) • Archive files can also be passed to an HSM system • HSM systems can use optical and optomagnetic storage media • The HSM system manages the archive files automatically

19. Compression Data is automatically compressed (Up to a factor of 5 or greater) Online data archiving • Data archiving may run in parallel to dialog and batch processing Data archiving without database backup • Data archiving is fully recoverable

20. Access to Archived Data • There are two ways to access archived data: • Sequential access to a dataset (reporting) • Random access to individual data objects • Archive files can be • Read • Analyzed (for reporting purposes) • Reloaded (Not Recommended – only right after archiving!) • Actions depend on archiving object • Access for reporting and reloading requires that the file is in the file system • Access to individual objects depends on an index

21. Conversion of Old Archive Files • Necessary conversions are performed automatically • Temporary conversion during data access • In rare cases, conversion programs are provided

22. The ADK Database R/3 System L D B Archiving Class ABAP/4 Program Codepage, Number Format, Structure Conversion Direct Access, Compression, Batch-Handling, . . . . ADK: Archive File ArchiveLink manually HSM Archive System with Tertiary Storage Media

23. Archive Database R/3 System What is an Archiving Object? • Key to archiving concept • Description of database objects • Collection of programs

24. Customization During Data Archiving • Creating an archiving object • Define archive object using transaction AOBJ • Create programs for archiving • Archive program • Delete program • Reload program • Index program • Create programs for accessing archived data • Reports • Direct access via online transaction

25. What We’ll Cover • Archiving benefits • Definition of terms • Archiving features • Archiving steps • Storage and management • Archiving by module • What next?

26. Archiving Personnel • Archiving can be performed • by the Basis Administrator • by the application department • Motivation (and point of entry) for both are different • Basis Administrator • The Basis Administrator checks the database/table size • Archiving object relationship has to be determined • Application Dept. • Application Dept. performs archiving for various reasons (periodically – month end, quarter end, year end) • Application Dept. uses direct point of entry

27. Archiving Flowchart

28. Identifying Critical Tables • Database Performance Screen (DB02) • Overall database growth analysis • Determine size of tablespaces and tables • Free storage space – allocated vs used space • Table Call Statistics Screen (ST10) • Locate the most active tables • Which tables are growing • Which tables have slow response

29. Identifying Critical Archiving Objects • Use archive administration (SARA) • Browse through the archiving objects • Determine which tables are in the archiving objects • Schedule all required archive programs • Use CCMS database administration (DB15) • Determine which archiving objects use a table • Check online space information for all tables of an archiving object

30. Business Requirements for Archiving Objects (SD_VBAK) • Document has status "Complete" • All documents in document flow have status "Complete" • Financial document is balanced • Additional customized checks possible

31. What We’ll Cover • Archiving benefits • Definition of terms • Archiving features • Archiving steps • Storage and management • Archiving by module • What next?

32. Storage & Management • Automated storage • Low management effort • Manual storage • Data medium maintenance • Data medium management • Archive production

33. Storage of Archive Files • Choice of medium is critical for: • Utilization of the functionality offered by SAP • Effort required for maintenance • Effort required to access data • Durability of archive • Protection of investment by means of combined methodology • Data archive files • Workflow for incoming and outgoing records • DMS (Document Mgmt. System) technical documents • Printed reports • Automation of database backup and restore

34. Archive Files on Hard Disk • Advantages • Single document access < 5 seconds • Quick access for reporting • Disadvantages • File system limitations • Considerable manual management effort required • Replicate the archive files • Storage of archive files • Data security

35. Archive Files on HSM System • Advantages • Single document access < 30 seconds (with block access) • Quick access for reporting (with block access) • Scalable file system • Long data medium retention period • Data medium management support • Disadvantages • Can be a costly solution

36. HSM System Requirements • "Write through" should be possible • Increased data security, as the archive files migrate before the delete program runs • Data retained in "magnetic cache" as copy • "Random access" to files must be possible • "Hardware sharing" is useful • Use of a jukebox via both HSM and ArchiveLink • "Media management" should be supported • Replicate archive files • Store archive files

37. Archive Files on Jukebox via ArchiveLink • Advantages • Scalable system • Long data medium retention period • Data medium management support • Jukebox can also be used for other R/3 tasks • Disadvantages • Can be a costly solution • Single document access only feasible for small archive files • Access time heavily depends on archive file size

38. ArchiveLink Capabilities • With ArchiveLinkyou can: • Archive incoming R/3 documents • Archive information created in R/3 • Directly access the optically archived documents from the R/3 applications • Integrate an optical archive system with R/3 • Replace your existing paper archive with electronic archive systems R/3 System ArchiveLink Archive system

39. Some Considerations for Media • Jukebox via ArchiveLink • Archived data seldom accessed individually • Access time is not critical, random access slow in Rel. < 3.1G • HSM system with block access • Archived data are often accessed individually • Access time can be on the order of 30 seconds • Hard disk • Archived data are regularly accessed individually • Access time must be on the order of 5 seconds • Magnetic tape • Archived data are only accessed in exceptional cases

40. Sample Access Times for Archived Data Sec.

41. What We’ll Cover • Archiving benefits • Definition of terms • Archiving features • Archiving steps • Storage and management • Archiving by module • What next?

42. FI Documents • 600,000 FI documents per month • Retention period up to 18 months • 120 GB used space in DB for FI data • 100,000 documents archived in 3 hours, 80 MB archive file • Reduce retention period to 3 months • Data archiving by IT organization

43. MM Documents • 450,000 MM documents and 2,500,000 line items per month • Line-item table bigger than 10 GB • 100,000 line items being archived in 3 hours • Direct access implemented by customer • Reduce retention period • Random access and reporting on data archive files shows good performance

44. Access Times … Example • Optimal situation, single job, elapsed time

45. What We’ll Cover • Archiving benefits • Definition of terms • Archiving features • Archiving steps • Storage and management • Archiving by module • What Next?

46. What next? • Increase in third party solutions • Better reporting for archiving objects • More robust document imaging solutions • Enhanced performance of ADK • Further integration of archiving and database administration

47. Sources of Information • Check OSS for your platform • SAPNET good source of information • SAPLabs, SAP Search, mySAP.com • SAP’s Early Watch for data archiving https://www014.sap-ag.de/dma • SAP-R3-L@LISTSERV.SAPFAQ.COM

48. Your Turn! (to launch Q&A) PCS Limited, A Professional IT Consulting Services Company “Creating Solutions Through Knowledge”  PCS Limited 5425 Parkford Circle Granite Bay, CA 95746 USA  Tel 916.773.9991 Fax 916.773.9998 www.pcs-limited.net rparkinson@pcs-limited.net