1 / 77

SharePoint 2010 Search First Migration Strategy & Implementation Tips

SharePoint 2010 Search First Migration Strategy & Implementation Tips. Guy Mounier, Technical Services Sang Shin, Business Development. Presentation to Harvard University. Agenda – 12pm to 1:30pm. 15 min: Introductions & Grab Lunch 20 min: What’s New in SharePoint 2010 Search

duane
Download Presentation

SharePoint 2010 Search First Migration Strategy & Implementation Tips

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. SharePoint 2010 Search First Migration Strategy & Implementation Tips Guy Mounier, Technical Services Sang Shin, Business Development

  2. Presentation to Harvard University Agenda – 12pm to 1:30pm • 15 min: Introductions & Grab Lunch • 20 min: What’s New in SharePoint 2010 Search What’s New in MS Enterprise Search 2010 offering for Users, Admins, Developers. Differences between FAST Search 2010 for SharePoint and SharePoint 2010 Search out-of-the-box. • 20 min: Search First Migration Strategy How to take advantage of SharePoint 2010 Search first in a phased migration strategy from MOSS. • 20 min: Best Practices for SharePoint 2010 Search Implementations Capacity Planning and Architecture Design Recommendations from the Trenches. • 10 min: About BA Insight Integrated Search Technologies for SharePoint & FAST Quick Overview & Demo • 5 min: Next Steps

  3. Introductions Tell us a bit more about your environment: • Current use of MOSS? • Current plans to upgrade to SharePoint 2010? • Any plans to leverage FAST for SPS? • Any plans to integrate external applications?

  4. Introductions Exclusively Supporting Microsoft Enterprise Search Since Day 1 • Microsoft Managed Partner ISV focused on Unified Enterprise Search Since 2004 • Headquartered in NYC, with Offices & 2m+ users worldwide • Google Compete & Autonomy Compete Team Members • TAP Member for FAST, SharePoint, Office365, and Azure • MTC Global Alliance Partner • MCS Certified Sub-Contractor • Technical Team Led by Jeff Fried, former Senior Product Manager for Microsoft Enterprise Search products • Key Competitive Wins include ExxonMobil, Accenture, Siemens, eBay, ADP, DIA, US Army, US Navy

  5. A Track Record of Success

  6. BA Insight for Education Sector • Harvard JFK School of Government • 3,000 User Licenses of Longitude Search for SharePoint • Deployed in 2010 • Contact: Don Oppenheimer

  7. What’s New in SharePoint 2010

  8. Microsoft Enterprise Search SKUs

  9. Flavors of FAST Bringing high-end search to the mainstream market

  10. FAST Search Extends SharePoint Superset of capabilities; common base functionality and platform Best High-End Content Search based on FAST ESP Advanced Content Processing Content SSA FAST Search for SharePoint Common query and federation APIs Common connectors and content APIs Content search Query SSA SharePoint Server 2010 People search End Users Content Unique Social Search from SharePoint Server 2010 + Profiles Common IT Pro and developer experience

  11. SharePoint 2010 and FAST search General Productivity search Light customization Social, Conversational Connectivity, Security, Relevance, Performance, Administration, Scale Customized productivity search and search driven applications Social, Conversational++, Visual, Contextual Connectivity+, Security, Relevance+, Performance, Administration+, Scale++ SharePoint Standard CAL + SharePoint server license SharePoint Enterprise CAL + FAST Search for SharePoint server license

  12. SharePoint 2010 & FAST End User Perspective

  13. FAST Search Interface Sorting on any property Query completion Relatedsearches & people Scrolling PPT Document thumbnails Read in Office Web Apps Federated results

  14. Sorting and Deep Refinement Enables precise control of results • Enables conversational experience across all of the results • Better findability and exploration • Discover non-obvious relationships across the entire result set • Exact counts shows relative weight • Provides analytic view of your results • Indicates priority and importance • The right lever to slice and dice your content • Sort on any field • Rearrange the result set to meet specific criteria • Alphabetical, numeric, and date Out of the Box Sorting Options Exact Counts

  15. SharePoint 2010 & FAST IT Pro Perspective

  16. Introducing the Processing Pipeline A systematic approach to interpreting your content • Sequential stages perform specific tasks while ingesting content • Breaks down content to the smallest addressable chunks to build meaning • Understands file encoding, data formats, and written languages • Supports 400+ file formats, 80+ languages • Process your content to make it searchable • Normalizes content so that a consistent relevancy model can be applied • Identifies structured and unstructured metadata in your content • Maps document metadata to SharePoint Crawled Properties

  17. Extending Pipeline capabilities Straightforward way to add custom text analysis functionality Configure Optional Processing Steps • XML Properties mapper • Offensive Content Filter • Verbatim (whole word) extractor • Use a dictionary for custom extraction • Pipeline Extensibility • Calls external applications for custom item processing • Field Collapsing • Entity Extraction Add Custom Processing Pipeline Extensibility is a specially defined stage that takes a set of crawled properties, as flat text as input and maps output to another crawled property Sandboxed execution Executable arguments and temporary files are automatically handled with timeouts. Runs just before the Crawled Property Mapper, providing accessibility within SharePoint

  18. Tune relevancy Improve accuracy and control with Rank Profiles Rank Profiles are made by combining multiple ranking elements Create custom ranking algorithms to combine multiple ranking properties Rank Profiles are exposed by modifying the sorting web part. Rank Profiles created in PowerShell

  19. SharePoint 2010 & FAST Developers Perspective

  20. Customizing the Search Experience IP Portfolio Mgt. Intel/Surveillance Drug Discovery …. Intranet Search People Search Site Search Research Portal Case Management Save Results to Excel …..

  21. Configuring Web Parts to extend UI Example: Add new Refiner to Results page • Expose custom collections (verbatim extractors) in refinement panel • Modify shared refinement web part on results page • Add new Managed Properties to existing XML file Custom Collections

  22. Query Language Expressiveness Use FAST Query Language (FQL) for precise query development • FQL provides a robust and expressive query language • Wildcard support - *, ? • Numeric Data types (Integer, Float, Decimal, Datetime) • Operators • Direct field access (e.g., title:othello, author:shakespeare) • Numeric (COUNT, RANGE, <, <=, >, >=) • Boolean (AND, OR, ANY, NOT) • Rank (RANK, XRANK) • Proximity (NEAR, ONEAR) • String (operator support for strings) • Boundary (starts-with, ends-with, equals) • Filter

  23. User Interface Extensibility Extend the interface by modifying web parts or creating new ones • Create custom interactive Web Parts • Bar Charts, Pie Charts, Maps, Tag Clouds • All Web Parts communicate through common interface • Create new Web Parts that can interact with others • All Web Parts are public • Extend Web Parts to change default behavior • Expand Queries and Results with SharePoint data • Use Federation Object Model to query: • SharePoint Search • FAST Search for SharePoint • Open Search (both synchronous and asynchronous) • Federation Object Model • Extend/Add Web Parts on the search page • Connect to new data sources (e.g.. Exchange) • Combine results from multiple locations.

  24. Search First Migration

  25. Search First Migration Project • Facilitate SP2010 migration and adoption: «Search First» • Form • Documentation and guidance (on TechNet) • Migration scripts and Web Parts (free and supported on partner site) • BAinsight.com/Resources –Free Downloads

  26. Migration Scenarios covered 1) New search instance with search center in SP 2010 2) New search instance with sites/search box in MOSS 2007 3) Existing search in MOSS 2007, migrating to search in SP2010

  27. Required Steps • Implement a new SP2010 Farm • Crawl SP2007 content • Set up search center • Migrate settings • Scopes • Best bets • Synonyms • Modify existing SP2007 Farm • Redirect query requests to SP2010

  28. Search First Migration SP 2007 SP 2010 Web Application A Site Collection A-1 Search Center Search Center A Site Collection A-N Search Center Web Application B Site Collection B-1 Search Center Search Center B Site Collection B-N Search Center

  29. Tools and Web Part provided • Migration tools for • Scopes • Best Bets • Site Collection Search Settings • Updating Web Parts for redirection to SP2010 • Profile Replication • SP 2007 Search Box Control • Displays Query Completion • Supports SP 2010 Scopes

  30. Master page webparts Specific page webparts

  31. Need to replicatescopes in SP2010

  32. Need to update all links to advancedsearch

  33. This Site and This List This Site and This List searches search from top of site

  34. SharePoint Search 2010Deployment Best Practices

  35. SP2010 Search Architecture • Multiple Query Servers • Partition index across multiple query servers • Mirror index for high availability • 100 mio. items with sub-second query latency • Multiple Crawlers (aka Index Servers) • Better crawl performance • Stateless, enabling failover • No copy of index on server, propagates indexed data to query servers • Multiple Property Databases • Minimize DB bottleneck known from SP2007

  36. SP2010 Search Architecture Partioned and Mirrored Index with 4xQuery + 2xProp. DB A A B B 2x2 Distributed Stateless Crawlers + 2xCrawl DB Admin Component + Admin DB

  37. Search Application Boundaries • Max. 20 index partitions • Max. 10 million items in each index partition • Max. 100 million items across all partitions • Max. 16 crawl components • Max. 10 crawl databases • Max. 10 property databases • Max. 64 query components • Max. 50 content sources • Max. 100 crawl rules

  38. Scale-Out Decision Points

  39. Scale-Out Decision Points

  40. Storage Architecture Search Database File Spindle Priority

  41. Hardware Requirements

  42. Search System Testing • Create a test plan • Create a test environment from the architecture you have designed • Migrate your SharePoint content to the test environment for realistic testing of indexing performance. Also, setup indexing of external content if applicable • Use load test tools to simulate expected demand, e.g. Visual Studio 2010 Ultimate • Analyze results, identify bottlenecks and optimize the architecture. Retest if needed

  43. Search System Monitoring • Monitor the live search system to • Identify and address performance bottlenecks • Address changes in demand like increasing size of search index and increasing query latency • Use the crawl log to verify that all content is successfully added to the index • Use performance counters to monitor CPU, Memory, Network and Disk usage. • Use search administration reports to monitor crawl rate and query latency

  44. Monitoring the Crawl Rate

  45. FAST Search 2010 for SPSDeployment Best Practices

  46. FAST Search – Main Components SharePoint Crawler Capacity: ~30 mill items per crawler node, SQL server needs to be scaled for high IO Web Analyzer CPU/disk footprint can vary by a factor of 10 depending on the content: - number of links - length of links - internal cross link ratio Average capacity: ~30 mill items per web analyzer node Can be deployed with the Indexer in normal scenarios Indexer/search node Two supported models: - Normal mode: ~15 mill items per node ~25 QPS - High Density Mode: ~ 40 mill. items per node ~ 7 QPS

  47. FAST Search HW – Best Practices CPU: 2 x 2GHz+ (Quad/six core) Memory: 24-48 GB Disk: 2 x 300 GB, SAS, 10K RPM (RAID 1) CPU: 2 x 2GHz+ (Quad/six core) Memory: 24-48 GB Disk alternatives: 1.0 TB: 8 x 300 GB, SAS, 10K RPM (RAID10) 1.8 TB: 8 x 300 GB, SAS, 10K RPM (RAID 5) 3.6 TB: 16 x 300 GB, SAS, 10K RPM (RAID 5+0) New: 7.2 TB: 16 x 600 GB, SAS, 10K RPM (RAID 5+0) SAN: Configured for “database performance”

  48. FAST Search – Server Calculation Assumptions • Query rate: < 25 QPS • Intranet search with 70% items and 30% documents, source doc between 10-100Kb. • Standard deployment and configuration Disclaimer: The numbers might not be representative for the customer environment and data. Please use caution when using these numbers for sizing.

More Related