Open Source Network Monitoring Tools

1. Open Source Network Monitoring Tools YasirIqbal 22-May-2010

2. Introduction • What are Network Monitoring Tools • Bandwidth Monitoring Techniques/Services • Setting up some monitoring Tools • Conclusion In this presentation

3. Cost of Bandwidth is expensive for developing countries • Bandwidth in developing countries is expensive. In a report for the Partnership for Higher Education in Africa, Mike Jensen calculates that Makerere University pays about $22,000/month for 1.5Mbps/768Kbps (in/out), Eduardo Mondlane pays $10,000/month for 1Mbps/384Kbps, while the University of Ghana pays $10,000/month for 1Mbps/512Kbps. These figures indicate that African universities, outside of South Africa, are paying over $55,000/month for 4Mbps inbound and 2Mbps outbound. These figures are about 100 times more expensive than equivalent prices in North America or Europe. Introduction:- Why do we need to monitor and measure Bandwidth

4. To Know if the ISP is providing us with the required bandwidth paid for. • To be able to optimize the available bandwidth • 59% of institutions do not monitor or manage bandwidth at all (Belcher) Cont…

5. Upgrade infrastructure, to install faster, larger, and higher performing systems, lines and facilities. • Look for cheaper provider and Increase/upgrade your bandwidth. • Alternative approach • is to recognize that ‘bandwidth’ is a valuable institutional resource or asset that needs to be managed, conserved, and shared as effectively as possible. Ways to improve network performance

6. Network Monitoring Tool How do we measure Bandwidth?

7. Allows the administrator to know the health status of the network. • It provides information about collected data and the analysis of such raw data with a view to using scarce or limited resources effectively. • Uses network probe. Probes let you isolate traffic problems and congestions slowing your network to a crawl. What are Network Monitoring Tools?

8. Identifying unofficial services or servers • Monitoring usage and traffic statistics • Troubleshooting your network • Investigating a security incident • Keeping logs of users activities for accountability What can we use the tools for?

9. Who is accessing your network? • students, academics, staff, visitors or others • What are they accessing your network for? • academic study, social use, business use, illegal use • Where are they accessing your network from? • internal, external • How are they accessing your network? • remote user, local Ethernet, WAN, dial-up, Wi-Fi, VPN • When did they access your network? • today, yesterday, last week, last month… Who? What? Where? How? When?

10. Active tools • Ping – test connectivity to a host • Traceroute – show path to a host • MTR – combination of ping + traceroute • SNMP collectors (polling)‏ • Passive Tools • MRTG • Nagios • Cacti • Ntop • Webalizer Network Monitoring Tools

11. Multi-Router Traffic Grapher Is a tool for monitoring traffic loads on a network link. MRTG generates HTML pages that provide a live, visual representation of the network traffic. It can be used to monitor any SNMP MIB. Limitations • It cannot provide information that shows which host or application may be causing a traffic bottleneck. • MRTG does not provide information about traffic type or protocol statistics Passive Network Monitoring Tools

13. TCPdump • Uses the packet capture library (libpcap). • Prints the headers of packet on a network interface, user analyses network status using this header manually • Has many option for capturing raw data, but it does not provide any analysis capability for the captured data. CONT…

14. IPTraf • IPTraf is a console-based network statistics utility for Linux. It gathers a variety of figures such as TCP connection packet and byte counts, interface statistics and activity indicators, TCP/UDP traffic breakdowns, and LAN station packet and byte count • Protocols Recognized • IP • TCP • UDP • ICMP • IGMP • IGP • IGRP • OSPF • ARP • RARP CONT…..

16. Webalizer • The Webalizer is a fast, free web server log file analysis program. It produces highly detailed, easily configurable usage reports in HTML format, for viewing with a standard web browser. • http://seecs.nust.edu.pk/stats/apr_2010/usage_201004.html CONT…

18. http://www.nagios.org/ • an enterprise-class network and server monitoring system. • Useful for: • Monitoring of network services. • Monitoring of host resources (processor load, disk usage, system logs) • Contact notifications when service or host problems occur and get resolved (via e-mail, SMS). • You can define event handlers that execute when triggered by certain events. (Proactive problem resolution) Nagios

23. http://www.opennms.org • Functionalities • High performance • A single instance of OpenNMS supports monitoring of a large number of nodes. • Automation • OpenNMS minimizes the amount of manual configuration. • Rule-based configuration • Flexible rules can be used to specify what services are polled on what devices. OpenNMS

25. http://www.cacti.net • Similar to MRTG. • Based on RRDtool. • Offers excellent graphing capabilities. • Has extensive templates. Cacti

26. Cacti is written as a group of PHP scripts. • The key script is “poller.php”, which runs every 5 minutes (by default). It resides in /usr/share/cacti/site. • To work poller.php needs to be in /etc/cron.d/cacti like this:MAILTO=root • */5 * * * * www-data php /usr/share/cacti/site/poller.php >/dev/null 2>/var/log/cacti/poller-error.log • Cacti uses RRDtool to create graphs for each device and data that is collected about that device. You can adjust all of this from within the Cacti web interface. • The RRD data is stored in a MySQL database along with descriptions of each device that is monitored. • The RRD files are located in /var/lib/cacti/rra. General Description of Cacti

27. You can measure Availability, Load, Errors and more all with history. • – Cacti con view your router and switch interfaces and their traffic, including all error traffic as well. • – Cacti can measure drive capacity, CPU load (network h/w and servers) and much more. It can react to conditions and send notifications based on specified ranges. • Graphics • – Allows you to use all the functionality of rrdgraph to define graphics and automate how they are displayed. • – Allows you to organize information in hierarchical tree structures. • Data Sources • – Permits you to utilize all the functions of rrdcreate and rrdupdate including defining several sources of information for each RRD file. Advantagess

28. Data Collection • – Supports SNMP including the use of php-snmp or net-snmp • – Data sources can be updated via SNMP o by defining scripts to do this. • – An optional component, cactid, implements SNMP routines in C with multi-threading. Important for very large installations, but not tested formally. • Templates • – You can create templates to reutilize graphics definitions, data and device sources • User Management • – You can manage users locally or via LDAP and you can assign granular levels of authorization by user or groups of users. Advantages cont.

29. Configuration of Interfaces is Tedious • – The first time you add an interfaces, add graphics for each interface and place these graphics correctly on a hierarchical menu requires considerable time and effort. • – It’s very important that you keep your Cacti configuration up-to-date with your network. You must either assign someone to do this, or create appropriate scripts and data shares for this purpose. • – If you make a configuration error it can be tedious to correct it. • But, in reality, for continuous use or large installations it is likely that you will be using scripts and tools to automate the configuration of Cacti. Disadvantages

30. Cacti requires that the following software is installed on your system. • RRDTool1.0.49 or 1.2.x or greater • MySQL 4.1.x or 5.x or greater • PHP 4.3.6 or greater, 5.x greater highly recommended for advanced features • A Web Server e.g. Apache • Net-Snmp • Mysql, PHP, Apache and SNMP packages are already installed on your machine if not installed through yum utility. • yum install mysql-server mysqlphp-mysqlphp-pear php-common php-gdphp-develphpphp-mbstringphp-cliphp-snmpphp-pear-Net-SMTP php-mysqlhttpd Setting up Cacti on CentOS 5

31. Install rrdtool manually by downloading the latest version at the following URL • http://oss.oetiker.ch/rrdtool/ • SCP the tarball into the /usr/src directory on your linux box. From a command prompt, change into the /usr/src directory, and un-tar the tarball: cd /usr/src tar -xzvf rrdtool-1.0.45.tar.gz Change into the newly created directory: cd rrdtool-1.0.45 Compile and install RRDTool: ./configure make make install rrdtool: Installation

32. The default installation location is /usr/local/rrdtool-VERSION, so make some symbolic links to the executables: ln -sf /usr/local/rrdtool-1.0.45/bin/rrdtool /usr/bin/rrdtool • ln-sf /usr/local/rrdtool-1.0.45/bin/rrdupdate /usr/bin/rrdupdateln -sf /usr/local/rrdtool-1.0.45/bin/rrdcgi /usr/bin/rrdcgi • The RRDTool Perl library simplifies things when using RRDTool from a Perl script, so to compile and install the Perl library for RRDTool: make site-perl-install • Create a directory for RRDTool databases, and a directory for the web images which it'll generate: mkdir /var/lib/rrdmkdir /var/www/html/rrdtool rrdtool: Installation

33. Extract the distribution tarball. • shell> tar xzvf cacti-version.tar.gz Create the MySQL database: • shell> mysqladmin --user=root create cacti Import the default cacti database: • shell> mysql cacti < cacti.sql • Optional: Create a MySQL username and password for Cacti. • shell> mysql --user=root mysql • mysql> GRANT ALL ON cacti.* TO cactiuser@localhost IDENTIFIED BY 'somepassword'; • mysql> flush privileges; cacti: Installation

34. Edit include/config.php and specify the MySQL user, password and database for your Cacti configuration. • $database_default = "cacti"; • $database_hostname = "localhost"; • $database_username = "cactiuser"; • $database_password = "cacti"; • Set the appropriate permissions on cacti's directories for graph/log generation. You should execute these commands from inside cacti's directory to change the permissions. • shell> chown -R cactiuserrra/ log/ (Enter a valid username for cactiuser, this user will also be used in the next step for data gathering.) • Add a line to your /etc/crontab file similar to: • */5 * * * * cactiuserphp /var/www/html/cacti/poller.php > /dev/null 2>&1 cacti: Installation

35. Now use a web browser and open the following address: • http://localhost/cacti • You will see the following... cacti: Installation

36. Press “Next >>” cacti: Installation

37. Choose “New Install” and press “Next >>” again. cacti: Installation

38. Your screen should look like this. If it does not ask your instructor for help. Press “Finish” Note! Be sure that “RRDTool 1.2.x” is chosen and not “1.0.x”. cacti: Installation

39. First time login use: User Name: admin Password: admin cacti: First Login

40. Now you must change the admin password. Please use the workshop password. cacti: Password Change

41. Management -> Devices -> Add • Specify device attributes • Choose a device template and this will ask you for additional information about the device. • You can add additional templates when, or if, you want. Add Devices: 1

42. Add Devices: 2

43. Choose SNMP version 2 for this workshop. • At your own location you can use SNMP version 3 if your devices support this. • SNMP access is a security issue: • Version 2 is not encrypted • Watch out for globally readable “public” communities • Be careful about who can access r/w communities. Add Devices: 3

44. Chose the “Create graphs for this host” • Under Graph Templates generally check the top box that chooses all the available graphs to be displayed. • Press Create. • You can change the default colors, but the predefined definitions generally work well. Create Graphics

45. Create Graphics: Step 1

46. Create Graphics: Step 2

47. Place the new device in its proper location in your tree hierarchy. • Building your display hierarchy is your decision. It might make sense to try drawing this out on paper first. • Under Management  Graph Treesselect the Default Tree hierarchy (or, create one of your own). View the Graphics

48. First, press “Add” if you want a new graphing tree: Second, name your tree, choose the sorting order (the author likes Natural Sorting and press “create”: Graphics Tree

49. Third, add devices to your new tree: Once you click “Add” you can add “Headers” (separators), graphs or hosts. Now we'll add Hosts to our newly created graph tree: Graphics Trees

50. An Example…