1 / 52

Brief Overview of Networking Evaluation Methods and Tools

Brief Overview of Networking Evaluation Methods and Tools. Outline. Tools Simulation ns2 Cluster based network emulation Emulab Live Distributed Testbed PlanetLab Questions What is the right tool(s) for my research? What does it take to get up and running?. Outline.

arich
Download Presentation

Brief Overview of Networking Evaluation Methods and Tools

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Brief Overview ofNetworking Evaluation Methods and Tools Evaluation Tools

  2. Outline • Tools • Simulation • ns2 • Cluster based network emulation • Emulab • Live Distributed Testbed • PlanetLab • Questions • What is the right tool(s) for my research? • What does it take to get up and running? Evaluation Tools

  3. Outline • For each tool/testbed • How does it work? • A “hello world” in ___ • What kind of research is it good for? • Tradeoffs between tools Evaluation Tools

  4. Simulation(ns2) Evaluation Tools

  5. What is ns? • Network simulator • a discrete event simulator • focused on modeling network protocols • wired, wireless, satellite • TCP, UDP, multicast, unicast • Web, telnet, ftp • Ad hoc routing; sensor networks • Infrastructure: stats, tracing, error models etc. Evaluation Tools

  6. ns --- what is it good for? Used to: • Evaluate performance of existing • network protocols. • Prototyping and evaluation of new protocols. • Large-scale simulations not possible • in real experiments. Evaluation Tools

  7. ns How does it work: • Event-driven simulator • Model world as events • Simulator has list of events • Process: take next one, run it, until done • Each event happens in instant of virtual time, but takes arbitrary real time • Single thread of control • Packet level Evaluation Tools

  8. Ns models • Traffic/applications • CBR, FTP, telnet, web • Routing/Queuing • Drop-tail, FQ, SFQ, RED, DRR • Wired routing, adhoc routing etc • Transport • TCP (variants), UDP, multicast (SRM) Evaluation Tools

  9. ns - software structure • Object oriented (C++, OTcl) – code reuse • Scalability + Extensibility • Control/”data” separation • Split C++/OTcl object • C++ for packet-processing (fast to run) • OTcl for control - (fast to write) • Simulation setup and configuration Evaluation Tools

  10. Pure C++ objects Pure OTcl objects C++/OTcl split objects OTcl C++ ns otcl and C++: The Duality Your ns-script Evaluation Tools

  11. Outline • Overview • Tcl, OTcl basics • ns basics • Extending ns • ns internals Evaluation Tools

  12. Basic structure of ns-scripts • Creating the event scheduler • [Tracing] • Creating network topology • Creating Transport Layer - Agents • Creating Applications - Applications • Events! Evaluation Tools

  13. Creating Event Scheduler • Create scheduler • set ns [new Simulator] • Schedule event • $ns at <time> <event> • <event>: any legitimate ns/tcl commands • Start scheduler • $ns run Evaluation Tools

  14. “Hello World” in ns simple.tcl set ns [new Simulator] $ns at 1 “puts \“Hello World!\”” $ns at 1.5 “exit” $ns run bovik@gs19% ns simple.tcl Hello World! bovik@gs19% Evaluation Tools

  15. Creating Network • Nodes • set n0 [$ns node] • set n1 [$ns node] • Links & Queuing • $ns duplex-link $n0 $n1 <bandwidth> <delay> <queue_type> • Queue type: DropTail, RED, CBQ, FQ, SFQ, DRR Evaluation Tools

  16. Routing + traffic • Unicast • $ns rtproto <type> • <type>: Static, Session, DV • Multicast support also. • Traffic • Simple two layers: transport and application. • Transport: TCP, UDP etc. • Applications: web, ftp, telnet etc. Evaluation Tools

  17. The transport layer: UDP • UDP • set udp [new Agent/UDP] • set null [new Agent/NULL] • $ns attach-agent $n0 $udp • $ns attach-agent $n1 $null • $ns connect $udp $null Evaluation Tools

  18. The transport layer: TCP • TCP • set tcp [new Agent/TCP] • set tcpsink [new Agent/TCPSink] • $ns attach-agent $n0 $tcp • $ns attach-agent $n1 $tcpsink • $ns connect $tcp $tcpsink Evaluation Tools

  19. Creating Traffic: On Top of TCP FTP • set ftp [new Application/FTP] • $ftp attach-agent $tcp • $ns at <time> “$ftp start” Telnet • set telnet [new Application/Telnet] • $telnet attach-agent $tcp Evaluation Tools

  20. Creating Traffic: On Top of UDP • CBR • set src [new Application/Traffic/CBR] • Exponential or Pareto on-off • set src [new Application/Traffic/Exponential] • set src [new Application/Traffic/Pareto] • Trace driven traffic • Inter-packet time and packet-size Evaluation Tools

  21. Attaching a traffic source • set cbr [new Application/Traffic/CBR] • $cbr attach-agent $udp • $ns at <time> “$cbr start” Evaluation Tools

  22. Tracing Trace packets on all links: • set f[open out.tr w] • $ns trace-all $f • $ns flush-trace • close $f <event><time><from><to><type><size>--<flags>--<flow id><src><dst><seqno> <pckt id> + 1 0 2 cbr 210 ------- 0 0.0 3.1 0 0 - 1 0 2 cbr 210 ------- 0 0.0 3.1 0 0 r 1.00234 0 2 cbr 210 ------- 0 0.0 3.1 0 0 Is tracing all links always the best thing to do? Evaluation Tools

  23. More Tracing • Tracing specific links • $ns trace-queue $n0 $n1 $f • Tracing variables • set cwnd_chan_ [open all.cwnd w] • $tcp trace cwnd_ • $tcp attach $cwnd_chan_ Evaluation Tools

  24. Controlling object parameters • Almost all ns objects have parameters • ex. Application/Traffic/Exponential has rate and packetSize • set parameters in OTcl • set etraf [new Application/Traffic/Exponential] • $etraf set rate_ 1Mb • $etraf set packetSize_ 1024 Evaluation Tools

  25. Putting it all together set ns [new Simulator] set n0 [$ns node] set n1 [$ns node] $ns duplex-link $n0 $n1 1.5Mb 10ms DropTail Creating Topology set tcp [$ns create-connection TCP $n0 TCPSink $n1 0] Creating Transport layer set ftp [new Application/FTP] $ftp attach-agent $tcp Creating Applications $ns at 0.2 "$ftp start“ $ns at 1.2 ”exit“ Schedule Events 1.5Mb 10ms n0 n1 FTP/TCP $ns trace-queue $n0 $n1 $f Evaluation Tools $ns run

  26. Example: XCP vs CSFQ • Compare utilization of links • Compare throughput of flows Evaluation Tools

  27. nam – the network animator n0 n1 • set nf [open out.nam w] • $ns namtrace-all $nf • exec nam out.nam & … Evaluation Tools

  28. ns “components” • ns, the simulator itself • nam, the Network AniMator • Visualize ns output • GUI input simple ns scenarios • Pre-processing: • Traffic and topology generators • Post-processing: • Simple trace analysis, often in Awk, Perl, or Tcl Evaluation Tools

  29. Network Dynamics: Link failures • $ns rtmodel-at <time> <up|down> $n0 $n1 • $ns rtmodel Trace <config_file> $n0 $n1 • $ns rtmodel <model> <params> $n0 $n1 <model>: Deterministic, Exponential Evaluation Tools

  30. Agent/FTP Agent/FTP Agent/FTP Agent/FTP Issues in Simulations • Suppose you want to study the way TCP sources share a bottleneck link… Which topology? Background Traffic? Which traffic sources? When to start sources? What else affects results? Evaluation Tools

  31. Cluster Based Emulation(Emulab) 368 nodes 3 big Ciscos Slides based on SOSP poster by Jay Lepreau et. al. University of Utah www.emulab.net Evaluation Tools

  32. Why? • “We evaluated our system on five nodes.” -job talk from university with 300-node cluster • “We evaluated our Web proxy design with 10 clients on 100Mbit ethernet.” • “Simulation results indicate ...” • “Memory and CPU demands on the individual nodes were not measured, but we believe will be modest.” • “The authors ignore interrupt handling overhead in their evaluation, which likely dominates all other costs.” • “You have to know the right people to use the cluster.” • “The cluster is hard to use.” • “<Experimental network X> runs FreeBSD 2.2.x.” • “February’s schedule for <Testbed Y> is…” • “<Network Z> is tunneled through the Internet” Evaluation Tools

  33. What? • An instrument for experimental CS research • A completely configurable “Internet emulator” in a room • At its core, it’s bare hardware, with… • … complete remote access and control • But, also simple to use • Lots of fast tools for common case • Automatic topology, node and link configuration • Automatic traffic generation • Universally available • Universities, research labs, companies • Zero-penalty for remote research Evaluation Tools

  34. Key Design Aspects • Allow experimenter complete control • Configurable network properties • link bandwidth, Latency, and loss rates via transparently interposed “traffic shaping” nodes that provide WAN emulation • Configurable OS image • Linux, FreeBSD, Windows • Virtualization • of all experimenter-visible resources • node names, network interface names, network addrs • e.g., node-0.esmexp1.esm.emulab.net Evaluation Tools

  35. Emulab Architecture Internet Web/DB/SNMP Switch Mgmt Users PowerCntl Control Switch/Router Serial Sharks Sharks PC PC 168 160 Programmable “Patch Panel” Evaluation Tools

  36. Experiment Creation Process Evaluation Tools

  37. Using Emulab • Submit ns script via web form • Specify number of nodes • Specify link properties • Relax while emulab … • Generates config from script & stores in DB • Maps specified virtual topology to physical nodes • Provides user accounts for node access • Assigns IP addresses and host names • Configures VLANs • Loads disks, reboots nodes, configures OSs • Two ways to run experiments • Interactive • Works only if there are enough free nodes right now • Batch • Runs experiment when enough free nodes available Evaluation Tools

  38. Using Emulab • Time sharing model for nodes • If you check out 50 nodes, they are exclusively yours until you release them (or forced to release them!) Evaluation Tools

  39. D D D D D D Using Emulab set ns [new Simulator] source tb_compat.tcl ##setup the core nodes for {set i 0} {$i <= 4} {incr i} { set node($i) [$ns node] tb-set-node-os $node($i) FC4-STD } ## setup the core links for {set i 0} {$i < 4} {incr i} { set j [expr $i + 1] $ns duplex-link $node($i) $node($j) 100Mb 0ms DropTail } $ns duplex-link $node(4) $node(0) 100Mb 0ms DropTail ## setup the edge nodes and links for {set i 5} {$i <= 10} {incr i} { set node($i) [$ns node] tb-set-node-os $node($i) FC4-STD set j [expr $i % 5] set link($i) [$ns duplex-link $node($i) $node($j) 1000Kb 0ms DropTail] tb-set-link-simplex-params $link($i) $node($i) 0ms 500Kb 0.0 } # Go! $ns rtproto Static $ns run Evaluation Tools

  40. What is Emulab Good For?(creators words) • Simulation • Fast prototyping, easy to use, but less realistic • Small static testbeds • Real hardware and software, but hard to configure and maintain, and lacks scale • Live networks • Realistic, but hard to control, measure, or reproduce results Emulab complements and also helps validate these environments Evaluation Tools

  41. What is Emulab Good For? • Studying routing protocols • Difficult to create richly connected nodes • Experiments that need access to the network core • Studying peer-to-peer protocols • But, lacks scale • Scaling experiments with “raw” machines with no link emulation • Good controlled environment for testing before going to PlanetLab Evaluation Tools

  42. Live Distributed Testbed(PlanetLab) 644 nodes at 304 sites www.planet-lab.org Evaluation Tools

  43. Resource Sharing • Statistical sharing of resources • Each application has a “slice” of the overlay resources • Bandwidth, CPU, memory • No guarantees on bandwidth, CPU or memory fairness • Slices guaranteed 1Mbps of bandwidth, rest shared by all slices Evaluation Tools

  44. PlanetLab Emulab Resource Sharing Evaluation Tools

  45. Slices Evaluation Tools

  46. Slices Evaluation Tools

  47. Slices Evaluation Tools

  48. Per-Node View Node Mgr Local Admin VM1 VM2 VMn … Virtual Machine Monitor (VMM) Evaluation Tools

  49. PlanetLab Virtualization: VServers • Kernel patch to mainstream OS (Linux) • Gives appearance of separate kernel for each virtual machine • Root privileges restricted to activities that do not affect other vservers • Some modification: resource control (e.g., File handles, port numbers) and protection facilities added Evaluation Tools

  50. How to Use PlanetLab • Get a slice • Request local PI • Setup SSH keys (no passwords on PlanetLab) • Add nodes to your slice through the web interface • Write (or borrow) scripts to deploy and run your system on PlanetLab and gather data from PlanetLab • Pssh/pscp (parallel versions of ssh/scp) • Remember, you’re running on a live network! • Test your code before you deploy Evaluation Tools

More Related