150 likes | 276 Views
ESPC CIP Update. November 7, 2008. COPC Action Item 2008-1.1.
E N D
ESPC CIP Update November 7, 2008
COPC Action Item 2008-1.1 • NESDIS is currently in the process of reviewing and planning upgrades for the critical infrastructure protection in the Environmental Satellite Processing Center (ESPC) (including the portions related to the Shared Processing Program). This includes various backup capabilities for priority Level 1 and 2 products within the ESPC. To date, only partial backup tests have been conducted. There is concern within the OPCs that NESDIS should evaluate full backups (i.e., “hot” backups). NESDIS is asked to provide a full plan to the other OPC directors on the backup procedures for the ESPC CIP. This plan would include, e.g., a “HOT” backup, and working with the other OPCs, determine the degradation expected, and length of test. Briefing of this plan is due at the next COPC. • CSAB 22 October, 2008. CSAB recommends closing this action with thebriefing to review the CIP implementation at Fall COPC.
COPC Action Item 2008-1.1 (cont) • CIP concept of operations is to operate as a “warm backup” as opposed to “hot backup” • All equipment is on but in an idle state, not producing products for distribution • Priority 1 and 2 products to be generated within 24 and 48 hours of management decision to activate CIP • Once fully spun up CIP provides all critical products to customers with latencies comparable to the primary production system at NSOF • Re-engineering CIP as a hot backup • Additional costs arise from • continuous synchronization with primary system at NSOF • increased O&M activities at CIP to monitor and maintain continuous data flows • Increased licensing costs
Project Overview • ESPC/CIP is a phased implementation • Phase 1 – CEMSCS (POES) • Phase 1.5 – Metop • Phase 2 – SATEPS (GOES) • Phase 3 – SATEPS and Metop (IASI, ASCAT) • Provide backup capability at Wallops for all Priority 1 & 2 “critical products” in the event ESPC at NSOF becomes non-operational • Priority 1 Products within 24 hours after decision to activate • Priority 2 Products within 48 Hours after decision to activate
CIP Phases • Phase 1 Capabilities • The CIP Phase 1 capabilities focus on providing POES products that have no dependence on data from Metop sources • Does not include NOAA-18 Blind GAC data • Connectivity • NWS (Telecom Gateway) • Suitland/NSOF • FCDAS (Fairbanks AK) • WCDAS (Wallops Main Facility) • NOC (Silver Spring, MD) – Internet Access
CIP Phases (Cont.) • Phase 1.5 Capabilities • Backup of Metop Data Processing System • Create and distribute products produced by NOAA instruments on board the Metop satellite (IASI and ASCAT Phase 3) • Receive Generic File Transfer Data (GOME and GRAS products) • Ability to Process Blind GAC from NOAA-18 • Connectivity • NWS (Telecom Gateway) • Suitland/NSOF • FCDAS (Fairbanks AK) • Darmstadt (Germany) • WCDAS (Wallops Main Facility)
CIP Phases (Cont.) • Phase 2 • Backup of ESPC SATEPS (GOES) Phase 1 • QuickSCAT • LRIT • MTSAT • Family of Services • Phase 3 • Backup of ESPC SATEPS (GOES) Phase 2 • Primarily a SW effort • IASI • ASCAT • Okeanos (Ocean Color)
Current Status • Project Reviews • Phase 1 Operational Readiness Review (May 28) • Phase 1 Commissioning Review (July 31) • Phase 1.5 Operational Readiness Review (September 8) • Acceptance Testing (July 28 to Sept 3) • Operational Exercise (Aug 25 – Sept 3) • User Test (October 27 – November 3) • Phase 1.5 Commissioning Review (November 18) January 2009 • Phase 2 Operational Readiness Review (March 2009) May 2009 • Phase 2 Commissioning Review (April 2009) June 2009
Current Status (cont) • Systems • Servers operational • Network established locally and to NSOF • AFEP configured to ingest NOAA-KLM, N-18 BGAC and MetOp data • Communications • External (Silver Spring, NWS, NSOF, FCDAS, Darmstadt, Internet) • Internal (WCDAS) • Interconnectivity between main facility and OBF • National Weather Service Telecommunications Gateway(NWSTG) Connections
Current Status (cont) • Facility • Power and HVAC operating as planned • Security System operating as planned • Fire Suppression System operating as planned • Quiet Room Occupancy Underway • SARSAT Backup occupancy date is Nov ‘08 • ESPC space Occupancy delayed • additional phone lines and electrical circuit
Current Status (cont) • Phase 1.5 User Test -- Complete • Results: • Successfully tested the transfer and receipt of data at CIP • Did not meet all test objectives because of data receipt problems
Current Status (cont) • Based on CIP Activation and User Test results 1.5 Commissioning Review delayed until completion of the following corrective actions • Complete all CIP 1.0 Commissioning Review Requests for Actions (RFA’s) • Complete all CIP 1.5 Commissioning Review Requests for Actions (RFA’s) • Ensure all computer operator monitoring procedures and start-up procedures are complete and up to date • Write procedures for METOP data connection until Global patch is ready • Test NOAA-18 Blind GAC • Schedule pre-activation test • Schedule another phase 1.5 CIP Activation and User Test
Next Steps • Phase 2 Operational Readiness Review Requirements have been identified and draft document generated for review • Hardware identified and procurements initiated • Build-out of hardware in progress • User Testing for Phase 2 tentatively scheduled for March 16-24 • Gather requirements for Phase 3
User 1.5 Testing -- Results • Test Results: • NCEP • User testing occurred without MetOp data for 24 hours • from 10:51 am 10/ 28 to10:33 10/29 • User testing occurred without NOAA-18 Blind GAC until 10/31/08 • DOD • Testing commences on schedule • User testing occurred without MetOp data for 24 hours • from 10:51 am 10/ 28 to10:33 10/29 • User testing occurred without NOAA-18 Blind GAC until 10/31/08 • All required products were available • FNMOC received the following non-optimal products: • ATOVS Soundings Data • MSPPS • AFWA received the following non-optimal products: • ATOVS Soundings Data • Level 1b MetOP-A FRAC data • Scheduled Deactivation (10:00 am on 10/30) schedule was not met: • 11:41 – AFWA distribution to CIP Turned off • 13:02 – sppgw1_cip to diamond_cip turned on • 14::35 – AFWA and NAVO data forwarded from sppgw1 to sppgw1_cip • 15:50 – FNMOC distribution to CIP Turned off • 19:28 – FNMOC data forwarded from sppgw1 to sppgw1_cip