SharePoint Content Lifecycle Management

1. SharePoint Content LifecycleManagement Presented by: Mary Leigh Mackie

2. Content Lifecycle Management Design Produce Consume Sites Work-flow Office Doc Library Version Publish-ing Site Record Center SharePoint

3. Agenda Content Organization & Storage Storage Optimization Content Access Archiving

4. Content Organization & Storage

5. Accountability of published content using workflows or approvals Managing search scopes, security trimming, federation Isolate intranet content from extranets Testing for consistency and performance Training your site/content owners and end users http://technet.microsoft.com/en-us/library/cc262873.aspx#section2 Determine the business goals What will your site structure and taxonomy look like? Standardize branding with templates and master pages Other considerations Information Architecture Source: Governance Resource Centre on Microsoft TechNet

6. Storage in a Content Repository Data in SQL Time in years Increase in % of inactive data over time

7. Planning for SharePoint Storage • Recycle bin • Versioning • Search and index information • Growth • Good rule of thumb for initial planning is: 3.5 x file system

8. Basic Storage Management Methods • Set site quotas and alerts! • 10 GB quota, 8 GB alert is my favorite • Monitor growth trends • Sites: slow over time or large jump in size? • Overall content DB size • Split Content DBs if they get “too big”

9. How SharePoint “chooses” a Content DB for a site • Highest remaining allotment rule • Content DB 1: 100 sites max • Content DB 2: 100 sites max • Content DB 1: 100 sites max • Content DB 2: 200 sites max SharePoint Site Content BD selection process: http://blog.jesskim.com/kb/293

10. Optimal Content DB Sizing • Backup & Recovery operations(<50-100 GB) • Performance (<500 GB… nervous at 300 GB) • # of objects • size of objects • Hardware (servers and storage) • Storage Cost (as small as possible!) So what is too big?

11. BLOBs-- What’s the Issue? • BLOBs = Binary Large Objects • SharePoint Content = BLOB + Metadata • Content DB = database of … BLOBs + Metadata • SQL DB storage needs high IOPS (input/output operations per second) and low latency • High IOPS + low latency storage = $$$$ • BLOBs do not participate in query operations, so no real reason to have BLOBs in a DB • DB full of BLOBs = wasted $$$

12. Default SharePoint Storage SharePoint WFE SharePoint Object Model BLOBs & Metadata SQL Server Content DB Config DB

13. Database Size Implications BLOBs increase DB size, creating issues with: Backup & Recovery operations Performance Storage Costs

14. Inactive sites, documents, list, libraries take up SQL storage, hindering performance Issues with BLOBs Get much worse over time… Data in SQL Time in years Increase in % of inactive data over time

15. Storage Optimization

16. SharePoint Storage Optimization Methods • Move the BLOBs out of the database • Archive content

17. No native archiving tools EBS extended to include RBS Available only in SQL Server 2008 SP2 Only accessible via API BCS (BDC in 2007) extended to allow for easier connectivity with legacy data systems Planning for Data Use & Growth What does SharePoint 2010 offer OOTB?

18. Storage Optimization Extending BLOBs out of the database

19. Available APIs for Extending SharePoint External BLOB Service (EBS) SQL Remote BLOB Service (RBS)

20. EBS/RBS Overview Blob Services to change BLOB storage locations EBS = External BLOB Service • SharePoint 2007 SP1+ API RBS = Remote BLOB Service • SQL Server 2008R2 Feature Pack API, with SharePoint 2010 support Both are interface specifications • Need a provider to actually work Cannot have both providers

21. EBS • EBS provider can take ownership of the BLOB • Provider gives SharePoint a token or a stub so SharePoint knows how to retrieve the object  (context) • Transparent to the end-user  SharePoint WFE SharePoint Object Model SQL Server BLOB BLOB Store Metadata Content DB Config DB EBS Provider

22. EBS • Implemented by SharePoint • Only 1 EBS Provider per SharePoint farm • Orphaned BLOBs- no direct method to compare BLOB store and Content DB • Compliance- what if I don’t want to allow SharePoint to delete the object?

23. RBS SharePoint WFE • Not unique to SharePoint, available to any application • A Provider Library can be associated with each database SharePoint Object Model BLOB & Metadata SQL Server Relational Access RBS Client Library Metadata BLOB Provider Library X Provider Library Y Content DB X Content DB Y BLOB Store BLOB Store

24. RBS • Implemented by SQL • Only 1 RBS Provider per Content DB • Orphaned BLOBs much less of an issue • Can lock down operations, from a unified storage perspective • Can be managed via Powershell

25. RBS: SQL Server 2008 Feature Pack API Handled natively by database Default Provider: FILESTREAM Enable FILESTREAM provider on SQL Provision data store and set storage location Install RBS on all SP Web and App servers Enable RBS

26. RBS versus SQL Filestream • Filestream storage must be file system locally attached to the SQL server • RBS is an API set that allows storage on external stores - physically separate machines that may be running custom storage code, for instance EMC Centera

27. EBS versus RBS, which is better? EBS Tighter integration with application, allows for more rules and settings

28. EBS versus RBS, which is better? RBS EBS Simpler, allows unified storage architecture across applications Tighter integration with application, allows for more rules and settings http://www.codeplex.com/sqlrbs

29. It looks like RBS has won… SharePoint 2007 SharePoint 2010 Future SharePoint Release (SPS 5?) SharePoint External BLOB Service (EBS) SQL Remote BLOB Service (RBS) SQL Server 2005 SQL Server 2008 Future SQL Releases Microsoft will provide a powershell solution to migrate from EBS to RBS

30. Benefits of Extending BLOBs • Backup & Recovery operations • Databases are 60-80% smaller • Need a method to backup BLOBs synchronously • Performance • Databases are 60-80% smaller • Performance improvement increase as the file/BLOB size increases. Microsoft research indicates: • <256kb, SQL better • 256kb to 1mb, SQL and file system comparable • >1mb, file system better • Storage Cost • “Not as expensive” storage • Archiving still needed for true savings

31. RBS is Completely Seamless for Users • Users can access contents by: • Clicking and downloading directly through SharePoint • Opening the file using their Office client • Referencing the URL • Searching for contents natively in SharePoint • Users can interact with contents by: • Modifying metadata and content types • Modifying permissions • Applying alerts • Using workflows or publishing templates • Using site Quotas and Locks

32. Cloud Storage Use Case SharePoint “Overdraft Protection” DB alert set at 80 GB, limit at 100 GB 0 80 100 • Could be any storage • Cloud is ideal “insurance”--cheap to setup, expensive to use Alert sent to admin No action taken Cloud Storage

33. Content Access

34. Where is it in it’s lifecycle? Do you want to expose it in SharePoint? BCS is intended for connecting LOB’s (Databases, Windows Communication Foundation (WCF) or Web services, .NET connectivity assemblies, Custom data sources) into SharePoint, without migrating the data No OOTB solutions for getting content out of users desktops, file shares, or other ECM systems SharePoint 2010 Support Connecting Legacy Data

35. Options for Exposing Legacy Data EBS/RBS API’s preferred (File Shares, Notes, Exchange Public Folders, eRoomDocumentum, LiveLink… etc?) • Migrate • Manually download/upload, losing author, time, security, history, other metadata • 3rd Party Tool • Connect • BCS Mechanisms • Most major ECM Vendors • AvePoint’s DocAve Connector

36. Which option is better? Connecting vs. Migrating • Value add of legacy system • Maintenance costs • Hardware • Licensing and support • Knowledge • Migration costs • Migration process • Tools • Training

37. Migrating vs. Connecting Migrating Connecting Data is available through SharePoint Data is left in source (legacy) system Give legacy system second life by increasing its value Burden of storage is on legacy system Changes propagate to source Connect and forget • Data is available in SharePoint • Data is moved into SharePoint • SharePoint replaced legacy system • Burden of storage is on SharePoint • Changes saved in SharePoint • Migrate and decommission

38. Connect to SharePoint: BCS Mechanisms .NET Assembly Connector • Provided with Microsoft Business Connectivity Services (BCS) • Each .NET connectivity assembly is specific to an external content type • Provides no Administration interface integration Custom Connector • Connect to external systems not directly supported by Business Connectivity Services • Agnostic of external content types that connect to a kind of external system (all databases or all Web services) • Provides an Administration UI integration http://msdn.microsoft.com/en-us/library/ee554911.aspx

39. Which BCS Mechanism Should I Use? The .NET Assembly Connector approach is recommended if the external system is static. Otherwise, for every change in the back end, you must make changes to the .NET connectivity assembly DLL. This, in turn, requires recompilation and redeployment of the assembly and the models. Custom connector approach is recommended if the back-end interfaces frequently change. By using this approach, only changes to the model are required. http://msdn.microsoft.com/en-us/library/ee554911.aspx

40. Connecting: 3rd Parties EBS/RBS API’s preferred (File Shares, Notes, Exchange Public Folders, eRoomDocumentum, LiveLink… etc?) • Most major ECM Vendors • Other 3rd Parties

41. How much content needs to be migrated? How long will this take? How much downtime can you tolerate? How much customization do you have? Is this a “big bang” migration or can you migrate in a scaled/phased approach? Can you accept loss of metadata and securities? Can you engage other members to assist in the process and arrange for proper training? What minimal requirements do you have for this migration? Can you properly map non-SharePoint related assets into SharePoint? Questions to ask yourself… etc… Options for Exposing Legacy Data: Migration

42. SharePoint Migration Strategies Cons Pros • Environments retaining ample amounts of outdated information • Moving to new hardware or new architecture • Puts Power Users in charge to recreate and manage sites • Migrate relevant content to avoid import of old data • Completely retains old environment • Virtually no downtime – requires user switch to new environment • Manual process, very resource intensive • Requires willing participants and intensive training • Requires additional steps to retain original URLs • Requires new server farm and additional SQL Server storage space for new content Best For User-Powered Manual Migration SharePoint Administrator installs the new version on separate hardware or a separate farm and allows Power Users to manually recreate content

43. SharePoint Migration Strategies Cons Pros • Any size environment, from single server environments to large, distributed farms • Granular migration • Retains all metadata • Virtually no downtime • Applicable to non-SharePoint repositories • Costs associated with purchasing of additional software • Requires new server farm Best For Migration via 3rd Party Tool SharePoint Administrator installs the new version on separate hardware or a separate farm, and migrates content and users using 3rd Party Tool

44. What About Access for Geo-Dispersed Users? • Centralized environment, accessed globally • Centralized environment plus local content (sites, etc) • Fully distributed, replicated architecture accessed locally • Centralized or cloud storage backup for high redundancy

45. Single Centralized Environment Global Architectures • Out of the box SharePoint • Lowest complexity, least costly • Varied User Experience • Evaluate bandwidth and usage patterns

46. Centralized plus local content Global Architectures • Local services and sites, in addition to main farm • Increased infrastructure complexity • Governance can be an issue • Relocating teams/users is a pain

47. Fullydistributed Global Architectures • Fast local access to SharePoint content • Replicate only what is relevant • Ability to handle remote locations

48. Distributed w/Centralized Backup Global Architectures • Backup locally or to alternative sites • Consider cloud storage • Can be used for high redundancy Cloud Storage

49. Archiving Adding Lifecycle Management to the picture

50. Lifecycle of a Typical Item Initial content creation High Moderate content retrieval Access / SLA Requirements Low Time