180 likes | 290 Views
EMC ControlCenter 6.0 Train-the-Trainer. Using Last Discovered Time implementation to identify data collection problems. Agenda. What is Last Discovered Time (LDT) ? LDT processing flow How LDT can be used to identify data collection problems. What is Last Discovered Time ?.
E N D
EMC ControlCenter 6.0Train-the-Trainer Using Last Discovered Time implementation to identify data collection problems
Agenda • What is Last Discovered Time (LDT) ? • LDT processing flow • How LDT can be used to identify data collection problems
What is Last Discovered Time ? • The last time the agent “successfully” discovered an object • Property of an object • Applicable (principal) objects: • e.g. Symmetrix, Connectivity Device, Host etc. • Pre 6.0 Agents are not LDT-capable
LDT processing flow • LDT processing is built on top of data collection processing • LDT processing overview: • Agent transmits LDT to the server at the time of STRSVC • Server persists LDT raw and supporting information in DB • During Store data processing, LDT is updated in DB • LDT, together with other object’s properties, is available (for example, for Console)
LDT processing flow 2 Store assignment Infra LDT Info Server 1 STRSVC Agent LDT Info 1a Ticket DB Agent 3 Ticket Info 4a Discovery Data Store 4 DATAPULL 5 DB Update LDT Infra Info LDT
How LDT can be used to identify data collection problems • Typical data collection problems: • Object updates by user are not reflected in Console • It appears as if scheduled DCPs are not firing • Sample questions: • When was the last time object was updated in DB? • When did the agent request a Store, to send data to? • Was data sent to a Store or not? • Was Store processing successful or not?
How LDT can be used to identify data collection problems (continued) • Where to find answers: • Properties view in the Console • Trace files • What trace file to look at • Database
What to look for in database • MODATACOLLECTIONINFO table • Potentially more than one record per ticket • More than one record per object • Important columns: • MOCanonicalName: • canonical name of the object • DataCollectionEndTimeServer: • time when the Agent completed data collection. Reported in STRSVC. Under normal circumstances, this time will represent LastDiscoveredTime for the corresponding object • DataPersisted: • “true” if Store completed DATAPULL processing successfully; “false” otherwise • IsLatest: • “true” if MODataCollectionInfo record represents the latest record for a given MOCanonicalName; “false” otherwise
What to look for in database (continued) • Useful columns: • DataCollectionStartTimeServer: • time when the Agent started data collection. Reported in STRSVC • StoreServiceTicketID: • “Ticket” used track data collection process • STRSVCSendTimeAgent: • time when the Agent sent STRSVC • STRSVCReceivedTimeServer: • time when the Server received STRSVC
What to look for in database (continued) • Useful columns (continued): • DATAPULLStartTimeAgent: • time when the Agent replied to DATAPULL request • DATAPULLStartTimeStore: • time when the Agent replied to DATAPULL request • DATAPULLStartTimeServer: • time when the Agent replied to DATAPULL request • DATAPULLProcessingEndTimeServer: • time when the Store finished DATAPULL processing
What to look for in database (continued) • Useful columns (continued): • ServerAgentTimeDiscrepancy: • discrepancy between time settings on Server and Agent machines in milliseconds: • StoreAgentTimeDiscrepancy: • discrepancy between time settings on Store and Agent machines in milliseconds • Message: • information about DATAPULL processing outcome • AgentName: • name of the agent
What to look for in database (continued) • Useful columns (continued): • StoreName: • Name of the store assigned to perform DATAPULL • AgentTimeZone: • Time zone of machine where agent is running (format +/- hhmm), for example, “-0500” for “(GMT-05:00) Eastern time (US & Canada)” • ServerTimeZone: • Time zone of machine where server is running • StoreTimeZone: • Time zone of machine where store is running
Case samples (case: normal) • MOCanonicalName: Host=abc.lss.emc.com • DataCollectionEndTimeServer: 9/7/2006 11:36:28 AM • DataPersisted: 1 • IsLatest: 1 • DataCollectionStartTimeServer: 9/7/2006 11:36:23 AM • StoreServiceTicketID: 2387 • STRSVCReceivedTimeServer: 9/7/2006 11:36:28 AM • DATAPULLStartTimeAgent: 9/7/2006 11:36:31 AM • DATAPULLStartTimeStore: 9/7/2006 11:36:31 AM • DATAPULLStartTimeServer: 9/7/2006 11:36:31 AM • DATAPULLProcessingStartTimeServer: 9/7/2006 11:36:32 AM • DATAPULLProcessingEndTimeServer: 9/7/2006 11:36:32 AM • ServerAgentTimeDiscrepancy: 774 • StoreAgentTimeDiscrepancy: 805 • Message: SUCCESS • AgentName: abc.lss.emc.com_MNR
Case samples (case: store processing failure) • MOCanonicalName: Host=abc.lss.emc.com • DataCollectionEndTimeServer: 9/7/2006 11:36:28 AM • DataPersisted: 0 • IsLatest: 1 • DataCollectionStartTimeServer: 9/7/2006 11:36:23 AM • StoreServiceTicketID: 2387 • STRSVCReceivedTimeServer: 9/7/2006 11:36:28 AM • DATAPULLStartTimeAgent: 9/7/2006 11:36:31 AM • DATAPULLStartTimeStore: 9/7/2006 11:36:31 AM • DATAPULLStartTimeServer: 9/7/2006 11:36:31 AM • DATAPULLProcessingStartTimeServer: 9/7/2006 11:36:32 AM • DATAPULLProcessingEndTimeServer: 9/7/2006 11:36:32 AM • ServerAgentTimeDiscrepancy: 774 • StoreAgentTimeDiscrepancy: 805 • Message: Failed: Error: Transaction failed • AgentName: abc.lss.emc.com_MNR
Case samples (case: current time 9/7/2006 2:15:26 PM: potential processing problem) • MOCanonicalName: Host=abc.lss.emc.com • DataCollectionEndTimeServer: 9/7/2006 11:36:28 AM • DataPersisted: 0 • IsLatest: 0 • DataCollectionStartTimeServer: 9/7/2006 11:36:23 AM • StoreServiceTicketID: 2387 • STRSVCReceivedTimeServer: 9/7/2006 11:36:28 AM • DATAPULLStartTimeAgent: • DATAPULLStartTimeStore: • DATAPULLStartTimeServer: • DATAPULLProcessingStartTimeServer: • DATAPULLProcessingEndTimeServer: • ServerAgentTimeDiscrepancy: • StoreAgentTimeDiscrepancy: • Message: • AgentName: abc.lss.emc.com_MNR
Case samples (case: current time 9/7/2006 2:15:26 PM: potential processing problem - continued) • Subcase #1: store is not assigned • StoreName • Subcase #2 store was assigned but DATAPULL has not finished • StoreName eccstore1 • Time zone information: • ServerTimeZone -0400 • StoreTimeZone -0400 • AgentTimeZone +0300
Case samples (case: current time 9/7/2006 5:15:26 PM & user action was performed 0.5 hour ago - potential agent problem) • MOCanonicalName: Host=abc.lss.emc.com • DataCollectionEndTimeServer: 9/7/2006 11:36:28 AM • DataPersisted: 1 • IsLatest: 1 • DataCollectionStartTimeServer: 9/7/2006 11:36:23 AM • StoreServiceTicketID: 2387 • STRSVCReceivedTimeServer: 9/7/2006 11:36:28 AM • DATAPULLStartTimeAgent: 9/7/2006 11:36:31 AM • DATAPULLStartTimeStore: 9/7/2006 11:36:31 AM • DATAPULLStartTimeServer: 9/7/2006 11:36:31 AM • DATAPULLProcessingStartTimeServer: 9/7/2006 11:36:32 AM • DATAPULLProcessingEndTimeServer: 9/7/2006 11:36:32 AM • ServerAgentTimeDiscrepancy: 774 • StoreAgentTimeDiscrepancy: 805 • Message: SUCCESS • AgentName: abc.lss.emc.com_MNR