1 / 85

Hands-On Lab: Drilling Down into Domino Statistics

Hands-On Lab: Drilling Down into Domino Statistics. Andy Pedisich Technotics. What We’ll Cover …. Using console commands to get statistics Calculating the space saved by using DAOS Working with a problem server’s surging CPU Setting up for spreadsheet based data drilling

sai
Download Presentation

Hands-On Lab: Drilling Down into Domino Statistics

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Hands-On Lab:Drilling Down into Domino Statistics Andy Pedisich Technotics

  2. What We’ll Cover … • Using console commands to get statistics • Calculating the space saved by using DAOS • Working with a problem server’s surging CPU • Setting up for spreadsheet based data drilling • Comparing server load with average server transactions per hour • Creating other charts using custom Statrep and spreadsheets • Wrap-up

  3. Perpetual Statistics • Domino servers constantly generate statistics • They track data on a surprising level • On almost every aspect of server operations • Agent manager • Mail and calendaring • The server’s platform • SMTP and Notes mail • LDAP • HTTP • Network • And lots more, too

  4. Server Statistics Are Organized Hierarchically • Stats are gathered into major categories like these • And then each one has a multitude of subcategories

  5. Subcategories of Statistics • Here’s a snapshot from the Administrator client showing some of the statistical hierarchy • This gives you a snapshot of the stats on your server • Use Refresh to get another snapshot

  6. Statistics Come in Basic Types • The basic types of statistics are: • Stats that never change once the server is started • Snapshot stats – reflect what’s going on right now • Cumulative stats that grow from the moment server is started • These stats are available to you for: • Your Domino servers • The platform your server is running on • Your network environment

  7. Static Statistics • Statistics that don’t change usually represent the operating environment of the server • Server.Version.Notes = Release 8.5.3FP3 • Server.Version.OS = Windows NT 5.0 • Server.CPU.Type = Intel Pentium • Disk.D.Size = 71,847,784,448 • Mem.PhysicalRAM = 527,433,728

  8. Amazing Detail, Yours Free! • This includes OS platform, Domino version, RAM • Lots of information about disks in use • Platform.LogicalDisk.TotalNumofDisks = 3 • Platform.LogicalDisk.2.AssignedName = E • Disk.C.Size = 80,023,715,840 • And even Network Interface Card (NIC) information • Platform.Network.1.AdapterName = Intel[R] PRO_1000 MT Server Adapter • Platform.Network.2.AdapterName = Broadcom NetXtreme Gigabit Ethernet _2 • Platform.Network.3.AdapterName = Broadcom NetXtreme Gigabit Ethernet

  9. What Good Are These Static Stats? • Think these static stats aren’t helpful? • Guess again • They are extremely valuable • If you are collecting stats correctly from all your servers, you can take a pretty detailed server inventory • Without leaving your desk • From servers all around the world, just by looking at the data we’re going to collect in the Monitoring Results database • This database is also know by its filename: STATREP.NSF

  10. Snapshot Statistics • Snapshot stats show what’s happening at the moment youask for them • They are changing all the time • Disk.E.Free = 18,679,414,784 • Server.Users = 280 • Mem.Free = 433,614,848 • MAIL.Waiting = 250 • The best part about this is that you get lots of Domino-related stats you wouldn’t get by looking at the operating system’s performance tools

  11. Cumulative Stats • Some stats are cumulative • They start counting from zero when you start the server • Server.Trans.Total = 31915 • SMTP.MessagesProcessed = 966 • Stats, like averages and maximums, are calculated from the cumulative ones • Server.Users.Peak.Time = 02/21/2006 07:50:33 MST • Platform.Memory.PagesPerSec.Peak = 1,364.1

  12. Resetting Statistics • Some of these cumulative stats can be reset using the following console command: • Set Statistics statisticname • You can’t use wildcards (*) with this argument! • Here’s an example of why you might want to reset a stat: • Set Stat Server.Trans.Total • Resets the Server.Trans.Total statistic to 0 • You might want to reset this stat if: • You are starting to benchmark a new application • You are debugging an agent and want to see if it is more efficient after changes to its design

  13. Platform Stats, Too • Platform stats vary widely from OS to OS • Getting platform stats from within Notes has great value • Track Domino server performance on an OS level even if your servers run on a variety of operating systems • For example, it’s very common to have a mix of AIX and Wintel servers • In a few minutes we’ll be discussing threshold tracking • You’ll be able to set notification thresholds universally from within Notes to track these platform stats

  14. Getting to Platform Statistics • Domino releases 6, 7, and 8 track platform stats automatically • In earlier versions they had to be explicitly enabled and many times were disabled due to problems with servers crashing • These problems are gone • To see all platform stats – enter this console command • Show stat platform

  15. See Server Statistics • Quickest way to see all server stats is to enter console command: • Show stat • Any place you can get to a console, you can access stats that can tell you a lot about the current state of the server • A SHOW STAT command gives you every statistic the Domino server has • Several hundred of them! • That’s really too many to deal with at once

  16. Can I See That in a Smaller Size? • Get a better view of the stats showing just what you’re looking for using the asterisk wildcard • You can ask directly for the top level of the hierarchy • Show stat server • That shows all of the stat hierarchy under “server”

  17. You Might Want Only Part of the Data • To get a select list of just the stats under the top-level requires the use of wildcards in your console commands • If you only want Server.Users hierarchy, use the global “*” • Show stat server.users.*

  18. What We’ll Cover … • Using console commands to get statistics • Calculating the space saved by using DAOS • Working with a problem server’s surging CPU • Setting up for spreadsheet based data drilling • Comparing server load with average server transactions per hour • Creating other charts using custom Statrep and spreadsheets • Wrap-up

  19. Let’s Have Some Fun With a Spreadsheet and DAOS • Domino Attachment and Object Service can save you disk space • When using DAOS, Domino saves attachments in an internal repository • If any other document in any other file or files have that same document, Domino saves a reference to the file • Attachments are only stored once on the server even though they might be used in hundreds of databases • That’s why DAOS saves disk space • But now that you’ve implemented DAOS, how much disk space have you saved • Wouldn’t you like to know how much precious disk space you’ve saved?

  20. How to Calculate DAOS Disk Space Savings • Your DAOS data must be on a separate drive outside of the data directory of the server to use this formula • Here’s the formula to determine DAOS disk space savings • DAOS Savings =Logical Size of all databases – (Physical size of data directory + Physical size of DAOS directory) • You can get all three of these statistics using the Notes Administrator client • You have to dig a little to get them, but they are all there

  21. Using the Notes Administrator Client to Get These Stats • We need three things: • Logical size of all databases • Physical size of data directory • Physical size of DAOS directory • Go to the Notes Administrator client and select the Files tab • The file columns will list the logical and physical size on a per-file basis • But how can you get the totals for all the files?

  22. Getting a List of All the Files • In the Notes Administrator Client using the Files tab, click on the All button on the top right of the screen • This will change the way the files on the server are displayed • You’ll no longer see folders

  23. The Tree View Changes into a List of Every File • You’ll see a complete list of all files on the server • But you won’t see the sum of each column • This data needs to be taken into a spreadsheet for further analysis • Note in the screen shot that every directory is listed

  24. Select All • Use the Edit menu to select all records being displayed on the Notes Administrator Client • Or you can select one of the records and do a Ctrl A • All the records will be selected

  25. Copy the Records to the Clipboard • Use the Edit  Copy menu or do a Ctrl C on the keyboard to copy all the records • Then open your spreadsheet

  26. Paste into Your Spreadsheet • Use the Edit  Paste menu sequence or a Ctrl V to paste the contents of your clipboard • The contents of the File tab which listed the disk stats for all of the files on your server will be pasted into the spreadsheet • Now we can use formulas to do some calculations

  27. You Might Want to Widen a Few Columns • Widen the columns for Logical Size and Physical Size just to see what you are working with • Remember, you need the total Logical Size, which is how big the databases would be if all the attachments were contained in the database • And you need the total Physical Size, which is how big the databases actually are

  28. Add @Sum Formulas to Logical and Physical Columns • Use an @sum formula to total the disk size at the bottom of the logical size and physical size columns • You can do this also by going to the bottom of each of the columns and clicking the autosum formula option • This is an Office 2007 screen shot • You now have 2 of the 3 stats you need

  29. You Need the Size of the DAOS Storage • To get the size of the DAOS storage, you have to query the statistics on the server • Do this by entering the following using the Notes administrator client server console • Show stat disk.* • Among the disk statistics will be the free space and size of the disk that houses the DAOS store • Copy them to the clipboard

  30. Using the Excel Import Wizard • If you are using Excel 2007 or better you can use Paste Special, and use the text import wizard to allow the disk size values to be pasted into cell • Otherwise you will have to copy the numbers out of the line of statistics you pasted in

  31. Paste into the Spreadsheet • Paste them into the bottom of the spreadsheet • Use a formula to subtract the DAOS free space from the total DAOS space allocated • That will give you the space used by DAOS

  32. Create the Final Formula • Create a formula • DAOS Savings = Logical Size of all databases – (Physical size of data directory + Physical size of DAOS directory) • It’s something great to show management

  33. What We’ll Cover … • Using console commands to get statistics • Calculating the space saved by using DAOS • Working with a problem server’s surging CPU • Setting up for spreadsheet based data drilling • Comparing server load with average server transactions per hour • Creating other charts using custom Statrep and spreadsheets • Wrap-up

  34. Dealing with Problematic Servers • Sometimes there are servers with issues that crop up • We would like to collect statistics for analysis from these systems more frequently than we do from the standard statistics collection interval • If you try to add a second collection interval on a server you’ll get this:

  35. Each Server Is Allowed to Collect Stats with Only One Interval • A server can only have one collection interval • You must create a 2nd collection document for another server • Don’t forget to add the “collect” task to servertasks= in NOTES.INI • Let’s look at a server that has CPU spikes • First we create a statistics collection document for a second server to take statistics from our problem server

  36. Set the Collection Interval for Five Minutes • Set collection interval for 5 minutes • Do not check any filters!!! • They tell the collector to ignore the statistics you checked • Note that stats are being logged to a database called ProblemServer.NSF • Used exclusively to track CPU util of Traveler task • Note that the data in this example has been fictionalized for effect

  37. Create a Special View That Tracks CPU Utilization for Traveler • In this case it’s the Traveler CPU we want to track • We create a custom view for the collecting database that only has the server name, the time of collection, and the statistic called Platform.Process.Traveler.1.PctCpuUtil • This will be used to easily create a graph of the CPU activity

  38. Collect the Data, Copy It as a Table from the Custom View • After collecting a week’s worth of data, we experience the CPU utilization • All the data in the view is selected using Ctrl A • It is copied as a table • Copying views as a table is my favorite feature in Notes • A Monitoring Results template is posted on my web site • A URL to this template is included at the end of the presentation

  39. Data Has Been Copied to a Spreadsheet • A simple paste of the data puts it into a spreadsheet where we are ready to turn it into a chart

  40. Use the Tools in Your Spreadsheet to Create a Graph • Select the columns Collection Time and Traveler CPU • Create a graph from the data • In this example, a scatter chart type with smooth lines is being used

  41. The Resulting Graph • This produces an excellent graph of the CPU utilization over a ten day period with samples being taken at intervals of 5 minutes • And it took less than 5 minutes to make this chart • One adjustment was made to the x-axis formatting and the legend was removed

  42. What We’ll Cover … • Using console commands to get statistics • Calculating the space saved by using DAOS • Working with a problem server’s surging CPU • Setting up for spreadsheet based data drilling • Comparing server load with average server transactions per hour • Creating other charts using custom Statrep and spreadsheets • Wrap-up

  43. The Statrep Template’s Only Export View • The default Lotus Statrep template’s Spreadsheet Export view just doesn’t seem to give us enough power • Pulling the data into Excel, then analyzing and graphing the data can often give you amazing insight into usage patterns • This information will be invaluable when: • Trying to consolidate servers • Troubleshooting performance issues

  44. Everything Is Everywhere • Keep in mind that every single statistic that is generated is contained in every document in the Monitoring Results database • You just can’t see it all because it’s not in views or documents • And views are the most important place to have it because that’s where it gives you the ability to compare samples • And analyze trends

  45. The Stats Are There, Now You See Them • There is a new, customized version of the Monitoring Results database • TechnoticsR85Statrep.ntf • It has all the views that are on the original Statrep • Plus over a dozen additional views to help you analyze the stats your servers generate • You can download this from my blog • www.andypedisich.com • Look for the link to Admin2013 resources

  46. Analysis Tools • Let’s cover the basics of the Statrep views used in the data export process • And the special Excel spreadsheet that contains custom formulas

  47. You Need a Better View of the Situation • The data export views are designed to be exported as CSV files • Each has key fields that are important to the export • Hour and Day generate an integer that represents the hour of the day and a day of the week • Hour 15 = 3:00 PM • Day 1 = Sunday, Day 7 = Saturday • These are used in hourly and daily calculations in pivot tables

  48. Export Views Are All Flat Views • Any view that is used for exporting data is flat, not categorized • This makes it easier to manipulate in pivot tables in Excel • There are columns in the export views that appear to have no data • They will be filled with a formula when brought into Excel

  49. Formulas Are Already Available • There is a spreadsheet containing my formulas to help you develop charts for all of this data • It’s available on SocialBizUG.org • Master Formula XLS Stat Exports- Technotics -V 2-4.xls • The views and spreadsheet will all fit together in a few moments

  50. What We’ll Cover … • Using console commands to get statistics • Calculating the space saved by using DAOS • Working with a problem server’s surging CPU • Setting up for spreadsheet based data drilling • Comparing server load with average server transactions per hour • Creating other charts using custom Statrep and spreadsheets • Wrap-up

More Related