1 / 19

Huawei Big Data Storage N9000

Huawei Big Data Storage N9000. Jan Stibor Storage & Server Product Manager. 1. Definition of Big Data. 2. HUAWEI Big Data Storage Solution. 3. Application Scenarios of HUAWEI Solution. Era of Big Data Is Coming. 250 million pictures per day. Videos of 28,800 hours per day.

eros
Download Presentation

Huawei Big Data Storage N9000

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Huawei Big Data Storage N9000 Jan Stibor Storage & Server Product Manager

  2. 1 Definition of Big Data 2 HUAWEI Big Data Storage Solution 3 Application Scenarios of HUAWEI Solution

  3. Era of Big Data Is Coming 250 million pictures per day Videos of 28,800 hours per day 2.9 million emails per second globally 24 PB data processed per day 200 PB per safe city 6.3 millionorders per day 50 million messages per day More than 1 PB per 3D movie By 2015, data amount will reach 8 ZB with the valued of $16.9 billion.

  4. What Is Big Data? Volume Mass data Exponential growth of unstructured data Velocity Quick data flow and dynamic data system Real-time analysis, not post analysis Variety Veracity Various data types Unstructured data such as files, emails, and videos accounting for 90% of the data amount generated in the next 10 years Veracity of Great data Prediction and analysis of the future and the behavior mode IDC: four characteristics of big data

  5. Driving Force of Big Data 1 • Storage — open architecture, gradual scale out, and integration of near-line systems and offline systems Collection 2 Big Data • Analysis — compound data, unified storage of multiple data sources, unified query, and sharing of storage resources Storage 3 • Archiving — lifecycle management of mass data at a low cost and high energy efficiency Management 4 • Maintenance — easy to use Analysis

  6. Contents 1 Definition of Big Data 2 HUAWEI Big Data Storage Solution 3 Application Scenarios of HUAWEI Solution

  7. High-performance solution Large-capacity solution Big Data Mass data query and analysis solution CSS/CSE HUAWEI Big Data Storage Solution Analysis High-performance solution Large-capacity solution Storage Archiving Mass data query solution

  8. HUAWEI OceanStor N9000—X in 1 • Linear expansion from 3 to 288 nodes • More than 3,000,000 OPS • Capacity up to 100 PB • NFS/CIFS interfaces • HDFS interface • Quick data import • Support for JDBCand ODBC • SQL 92/2003 • Data compression Scale-out NAS Scale-out DB • Global online deduplication • Support for LTFS Scale-out Backup

  9. Apple HUAWEI OceanStor N9000 Architecture Application layer … Internet HPC Media asset Behavior prediction, real-time analysis, … …… Big data analysis Big data sharing BI Unstructured data, object data, structured data Map-reduce Object interface Archiving interface Database interface HDFS NAS interface Unified management and namespace Periodic data inspection Lifecycle policy Index of quick file search Remote replication and snapshot Global deduplication Storage Analysis Archiving Storage layer 10GE/Infiniband Node Node Node Node Node Node

  10. N9000 Hardware 2 U high 25 disk slots 2-channel hexa-core CPU 48 GB memory at least SSDs for storing metadata High-performance node： The scene of frequent readand writesmall files Front view Rear view 4 U high 36 disk slots 2-channel hexa-core CPU 48 GB memory at least Front view High-bandwidth node： The scene of large file sequential read and write Rear view 4 U high 36 disk slots 2-channel quad-core CPU 16 GB memory at least Front view Large-capacity node： Near-line storage Rear view Acceleration node 1 U high

  11. N9000 Software — Wise Series WushanFS • Self-developed distributed file system • Mandatory, one-off purchase • Wise Data Protection (WiseDP) 1 WiseLink • Cluster load balancing software • Support for multiple load balancing policies • Optional, allowing for separate purchase 2 WushanFS WiseQuota • Quota management software that sets space usage limits on directories or users. • Optional, allowing for separate purchase 3 WushanFS: Wushan File System WiseTier • DST software • Support for policy-based automatic migration • Optional, allowing for separate purchase 4

  12. Extensive Flexibility: WiseDP Switch +2 ECC +1 ECC Original data counts (ODC) Redundant data counts (RDC) Redundant data nodes (RDN) +3 ECC +4 ECC N+1 toN+4 data protection File-leveldata protection 1 hr/TB data recovery

  13. Outstanding Performance: WiseLink Automatic load balancing among existent and newly added nodes ... Single node expansion within 60s Without human intervention and manual modifications

  14. Outstanding Performance: WiseTier • Enables DST to work among diverse types of nodes. • Adopts tiering policies based on I/O access popularity. • Adopts hotspot anti-shake policy to prevent unnecessary data migration. • Seamlessly adapts to service changes without interrupting ongoing applications. • Consolidates multiple storage systems into a single storage system, eliminating the possibilities of frequent data migration and simplifying management. High-OPS nodes Large-capacity nodes Hot data Cold data

  15. Simplified Management: Refined Space Management WiseQuota: sets quotas for subdirectories to facilitate space management. 100 TB 20 TB 20 TB 10 TB 10 TB 30 TB 40 TB Fast automatic deployment Comprehensive performance monitoring User group User group User User Unified topology display Centralized management of a single file system of 100 PB User User

  16. HUAWEI 9000 — Adapting to Changes and Converging for Easy Management Extensive Flexibility Outstanding Performance Optimal Integration Streamlined Management • Distributed scale-out architecture • Integration of scale-out NAS, scale-out DB, and scale-out backup • Unified management of all nodes on a single interface • LCM • Scaling from 3 to 288 nodes • Support up to 100 PB of capacity • Linear scaling of performance and capacity • World's leading file system access performance of more than 3 million OPS

  17. Contents 1 Definition of Big Data 2 HUAWEI Big Data Storage Solution 3 Application Scenarios of HUAWEI Solution

  18. HUAWEI OceanStor N9000 Scenario— Media Asset Management Focusing on five aspects Animation rendering Non-linear editing system TV station non-linear editing system solution • Stable bit steams and low single-stream bandwidth, with a strong need for data sharing Animation rendering solution • Frame image file storage, with a strong need for high performance in sequential read of small-sized I/Os VOD solution • Concurrent reads by multiple users, with a strong need for high bandwidth Media asset management solution • Large capacity, high bandwidth, and data sharing Movie post-production solution • Stable bid streams and high-bandwidth throughput, providing data sharing ... … … … HPC HD editing SD editing Cataloging N9000high-performance nodes VOD system N9000 high-bandwidth nodes and large-capacity nodes LAN VOD server cluster TV set + set-top box PC PC Smart phone Media asset management system Post-production system Query/Cataloging Audio Editing Effect …

More Related