1 / 51

Load Testing Using Perl

National Aeronautics & Space Administration. Load Testing Using Perl. Homer Hummel Jet Propulsion Laboratory, California Institute of Technology Homer.Hummel@jpl.nasa.gov 2008-07-25. Presentation Overview. UnixLoadGen Load Test Configuration, RSA Auth and Unix fork

noel
Download Presentation

Load Testing Using Perl

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. National Aeronautics & Space Administration Load Testing Using Perl • Homer Hummel • Jet Propulsion Laboratory, • California Institute of Technology • Homer.Hummel@jpl.nasa.gov • 2008-07-25

  2. Presentation Overview • UnixLoadGen • Load Test Configuration, RSA Auth and Unix fork • Typical LoadGen Run Scenario • LoadGen directory structure • File formats • Perl scripts • Real-world experiences • General comparison with COTS • References

  3. UnixLoadGen • A general framework for load testing application servers. • Written in Perl. • Uses “fork” feature of Unix. • Uses “RSA Authentication” feature of SSH. • UnixLoadGen distributed on website [TBD] • UnixLoadGen – hereinafter referred to as LoadGen.

  4. Typical Load Test Configuration Generator 1 Test Server Generator 2 Control Workstation Generator X

  5. Use of RSA Authentication Generator 1 tester Public Key Control Workstation Generator 2 tester tester Private Key Public Key Public Key Generator X tester Public Key

  6. Control Workstation and Alternate Control Workstn Generator 1 tester tester Public Key Public Key Private Key Public Key Generator 2 tester Public Key Public Key Control Workstn (alt) tester Private Key Public Key Generator X tester Public Key Public Key

  7. Use of Unix “fork” Generator Test Server startN.pl Y load_script load_script (1) load_script (2) load_script (Y)

  8. Typical LoadGen Run Scenario Generator 1 Control Workstn Test Server load_script (1) load_script (2) Edit scripts & data push_files.pl loadgen.pl process_results.pl Control file: load_script (Y) Generator 2 load_script (1) load_script (2) load_script (Y) Gen1 cmd Gen2 cmd GenX cmd Generator X load_script (1) load_script (2) load_script (Y)

  9. Typical LoadGen Run Scenario Generator 1 Control Workstn Test Server Edit scripts & data push_files.pl loadgen.pl process_results.pl Control file: Generator 2 Gen1 cmd Gen2 cmd GenX cmd Generator X

  10. Typical LoadGen Run Scenario Generator 1 Control Workstn Test Server scripts data files Edit scripts & data push_files.pl loadgen.pl process_results.pl Control file: Generator 2 Gen1 cmd Gen2 cmd GenX cmd Generator X

  11. Typical LoadGen Run Scenario Generator 1 Control Workstn Test Server scripts data files Edit scripts & data push_files.pl loadgen.pl process_results.pl Control file: Generator 2 scripts data files Gen1 cmd Gen2 cmd GenX cmd Generator X

  12. Typical LoadGen Run Scenario Generator 1 Control Workstn Test Server scripts data files Edit scripts & data push_files.pl loadgen.pl process_results.pl Control file: Generator 2 scripts data files Gen1 cmd Gen2 cmd GenX cmd Generator X scripts data files

  13. Typical LoadGen Run Scenario Generator 1 Control Workstn Test Server scripts data files Edit scripts & data push_files.pl loadgen.pl process_results.pl Control file: Generator 2 scripts data files Gen1 cmd Gen2 cmd GenX cmd Generator X scripts data files

  14. Typical LoadGen Run Scenario Generator 1 Control Workstn Test Server load_script (1) load_script (2) Edit scripts & data push_files.pl loadgen.pl process_results.pl Control file: load_script (Y) Generator 2 Gen1 cmd Gen2 cmd GenX cmd Generator X

  15. Typical LoadGen Run Scenario Generator 1 Control Workstn Test Server load_script (1) load_script (2) Edit scripts & data push_files.pl loadgen.pl process_results.pl Control file: load_script (Y) Generator 2 load_script (1) load_script (2) load_script (Y) Gen1 cmd Gen2 cmd GenX cmd Generator X

  16. Typical LoadGen Run Scenario Generator 1 Control Workstn Test Server load_script (1) load_script (2) Edit scripts & data push_files.pl loadgen.pl process_results.pl Control file: load_script (Y) Generator 2 load_script (1) load_script (2) load_script (Y) Gen1 cmd Gen2 cmd GenX cmd Generator X load_script (1) load_script (2) load_script (Y)

  17. Typical LoadGen Run Scenario Generator 1 Control Workstn Test Server raw_results (1) raw_results (2) Edit scripts & data push_files.pl loadgen.pl process_results.pl Control file: raw_results (Y) Generator 2 raw_results (1) raw_results (2) raw_results (Y) Gen1 cmd Gen2 cmd GenX cmd Generator X raw_results (1) raw_results (2) raw_results (Y)

  18. Typical LoadGen Run Scenario Generator 1 Control Workstn Test Server raw_results (1) raw_results (2) Edit scripts & data push_files.pl loadgen.pl process_results.pl Control file: raw_results (Y) Generator 2 raw_results (1) raw_results (2) raw_results (Y) Gen1 cmd Gen2 cmd GenX cmd Generator X raw_results (1) raw_results (2) raw_results (Y)

  19. Typical LoadGen Run Scenario Generator 1 Control Workstn Test Server raw_results (1) raw_results (2) Edit scripts & data push_files.pl loadgen.pl process_results.pl Control file: raw_results (Y) Generator 2 raw_results (1) raw_results (2) raw_results (Y) Gen1 cmd Gen2 cmd GenX cmd Generator X raw_results (1) raw_results (2) raw_results (Y)

  20. Typical LoadGen Run Scenario Generator 1 Control Workstn Test Server Edit scripts & data push_files.pl loadgen.pl process_results.pl Control file: Generator 2 Summary Test Report Gen1 cmd Gen2 cmd GenX cmd Generator X

  21. Typical LoadGen Run Scenario Generator 1 Control Workstn Test Server load_script (1) load_script (2) Edit scripts & data push_files.pl loadgen.pl process_results.pl Control file: load_script (Y) Generator 2 load_script (1) load_script (2) load_script (Y) Gen1 cmd Gen2 cmd GenX cmd Generator X load_script (1) load_script (2) load_script (Y)

  22. LoadGen Directory Structure Home directory of local user account ‘tester’ control (on control workstation only) data test_type1 (e.g., dir) test_type2 (e.g., pop3) [etc.] raw_results test_type1 test_type2 [etc.] results (on control workstation only) scripts

  23. Unique Input Data Lists names000 (file): names001 (file): … names199 (file): testuser1 testuser51 testuser9951 testuser2 testuser52 testuser9952 … … … testuser50 testuser100 testuser10000 list_of_names_params_5.list00 (unique param list example): -names data/dir/names000 -names data/dir/names001 -names data/dir/names002 -names data/dir/names003 -names data/dir/names004 list_of_names_params_5.list00 (one file per generator) list_of_names_params_5.list01 … list_of_names_params_5.list09

  24. Control File Format(General) Generator_1 command_string Generator_2 command_string … Generator_X command_string Example: file jabber.ctl: gen-int-000 'export TERM=vt100; cd /home/tester/jabber/jabsimul/jab_simul; ./jab_simul > /tmp/jabber_output_gen-int-000.txt' gen-int-001 'export TERM=vt100; cd /home/tester/jabber/jabsimul/jab_simul; ./jab_simul > /tmp/jabber_output_gen-int-001.txt‘ … gen-int-019 'export TERM=vt100; cd /home/tester/jabber/jabsimul/jab_simul; ./jab_simul > /tmp/jabber_output_gen-int-019.txt'

  25. Control File Format(Y-instances of cmd_str) Generator_1 startN.pl Y command_string Generator_2 startN.pl Y command_string … Generator_X startN.pl Y command_string

  26. Control File Format(y-instances with unique params) Generator_1 startNupl.pl Y list_of_names_params_5.list00 cmd_string Generator_2 startNupl.pl Y list_of_names_params_5.list01 cmd_string … Generator_X startNupl.pl Y list_of_names_params_5.listxx cmd_string Example: file ldir_auth_mix_edg.ctl: gen-int-001 scripts/startNupl.pl 10 data/dir/list_of_names_params_10.list00 scripts/lmodattr_lnx.pl -h ldaptest –b dirtest -w xxx -a jplalias -v tester1 -c dirtest -r -iterations 200 gen-int-002 scripts/startNupl.pl 10 data/dir/list_of_names_params_10.list01 scripts/lmodattr_lnx.pl -h ldaptest -b dirtest -w xxx -a jplalias -v tester1 -c dirtest -r -iterations 200 … gen-int-010 scripts/startNupl.pl 10 data/dir/list_of_names_params_10.list09 scripts/lsgetentry_lnx.pl -h ldaptest -b dirtest -w xxx -p -c uid=celachi -iterations 200

  27. Raw Results File Naming & Format Filename: test_run_id~servername~generator~log.pid File contents: start_time,stop_time start_time,stop_time … (one line for each iteration of load script transaction) Example: File: mix1~ldaptest~gen-int-005~log.12403 1212105348.215921,1212105349.307710 1212105349.307803,1212105351.421984 1212105351.422039,1212105353.107550 1212105353.107605,1212105356.026773 …

  28. Summary Report File Example Server: ldaptest Load test: lsgetentry Test run: mix1 Date/time of beginning of test: 2008-05-29 16:55:50 Date/time of end of test: 2008-05-29 17:05:57 Test duration: Hours: 0 Minutes: 10 Seconds: 7 Values for minimum, average, maximum and standard deviation are in seconds. Min = 0.256 Ave = 2.894 Max = 6.293 Std Dev = 1.354 Total number of transactions = 5000 Concurrent Test Clients Users Iterations Errors gen-int-003 5 1000 0 gen-int-006 5 1000 0 gen-int-007 5 1000 0 gen-int-012 5 1000 0 gen-int-018 5 1000 0

  29. push_files.pl snippets my $ctrl_c = 0; # initialize $ctrl_c flag $SIG{INT} = sub { $ctrl_c = 1 }; # set the $ctrl_c flag on interrupt open CONTROL, "<../control/$control_file" or die "Can't open $control_file: $!"; while (<CONTROL>) { next if ($_ =~ /^#/); # skip comment lines (# in column 1) ($mach, $remaining) = split /\s+/, $_, 2; push @machines, $mach; push @cmd_strings, $remaining; } close (CONTROL);

  30. push_files.pl snippet my $localhome = $ENV{HOME}; foreach my $mach (@machines) { my $remotehome = `ssh $mach 'echo \$HOME'`; chomp $remotehome; if ($scripts_flag == 1) { print "Pushing scripts directory...\n"; `scp -r $localhome/scripts $mach:$remotehome`; } if ($data_flag == 1) { print "Pushing data directory...\n"; `scp -r $localhome/data/$test_types[$i] $mach:$remotehome/data`; } print "Push files completed on $mach.\n"; last if ($ctrl_c == 1); }

  31. push_files.pl snippet my $localhome = $ENV{HOME}; foreach my $mach (@machines) { my $remotehome = `ssh $mach 'echo \$HOME'`; chomp $remotehome; if ($scripts_flag == 1) { print "Pushing scripts directory...\n"; `scp -r $localhome/scripts $mach:$remotehome`; } if ($data_flag == 1) { print "Pushing data directory...\n"; `scp -r $localhome/data/$test_types[$i] $mach:$remotehome/data`; } print "Push files completed on $mach.\n"; last if ($ctrl_c == 1); }

  32. loadgen.pl snippet my $errors = 0; my @pid; foreach my $i (0..$#machines) { FORK: { if ($pid[$i] = fork) { # Parent process, child's pid in the array next } elsif (defined $pid[$i]) { # Child process print "Starting command string on $machines[$i]\n"; exec "ssh $machines[$i] $cmd_strings[$i] -test_run_id $test_run_id"; die "Exec failed after fork\n"; } elsif ($! =~ /EAGAIN/) { # EAGAIN is the supposedly recoverable fork error sleep 5; redo FORK; } else { die "Can't fork: $!\n"; } } sleep $ramp_up_delay; }

  33. loadgen.pl snippet my $errors = 0; my @pid; foreach my $i (0..$#machines) { FORK: { if ($pid[$i] = fork) { # Parent process, child's pid in the array next } elsif (defined $pid[$i]) { # Child process print "Starting command string on $machines[$i]\n"; exec "ssh $machines[$i] $cmd_strings[$i] -test_run_id $test_run_id"; die "Exec failed after fork\n"; } elsif ($! =~ /EAGAIN/) { # EAGAIN is the supposedly recoverable fork error sleep 5; redo FORK; } else { die "Can't fork: $!\n"; } } sleep $ramp_up_delay; }

  34. process_results.pl snippet my $rawdir = "$localhome/raw_results/$test_type"; # get all raw_results files for test_type if (-e $rawdir) { my @all_raw_results_files = `ls -1 $rawdir`; # -1 (one) option to # 'ls' cmd gives just file names } else { die "No raw results of test type: $test_type\n"; } # collect relevant raw results file names by test run identifier foreach my $file (@all_raw_results_files) { chomp $file; my @file_name_portions = split /~/, $file; if ($file_name_portions[0] =~ /$test_run_id/) { push @raw_results_files, $file; } }

  35. load_script_template.pl (part 1) #!/usr/bin/perl use strict; use warnings; use Getopt::Long; use Sys::Hostname; use Time::HiRes; my $ctrl_c = 0; # initialize $ctrl_c flag $SIG{INT} = sub { $ctrl_c = 1 }; # set the $ctrl_c flag on interrupt my $iterations = 3; my $delay_iteration = 0; # delay between iterations in seconds my $test_run_id; # test run identifier string e.g. run01, aurnn14, or xyz my $servername; # servername (unix hostname) my $outdir = "raw_results/"; my @start_times = (); # start timestamps in floating point seconds my @stop_times = (); # stop timestamps in floating point seconds

  36. load_script_template.pl (part 2) &GetOptions( "iterations:i" => \$iterations, "delay_iteration:i" => \$delay_iteration, # delay 'tween iterations "test_run_id:s" => \$test_run_id, # test run identifier "servername:s" => \$servername, # servername ); # if test_type subdir of raw_results does not exist, create it. # substitute "test_type" with the real one e.g. "dir" mkdir ($outdir . "test_type") unless -e ($outdir . "test_type"); $outdir .= "test_type/"; my $logfile = $outdir . $test_run_id . "~$servername~" . hostname() . "~log.$$"; # $logfile example: raw_results/dir/run01~devldap~gen-01~log.3257

  37. load_script_template.pl (part 3) # main processing loop foreach my $iter (1..$iterations) { push @start_times, Time::HiRes::time; # save start timestamp # Perl code for client-server transaction goes here push @stop_times, Time::HiRes::time; # save stop timestamp last if ($ctrl_c eq 1); # exit loop on interrupt sleep $delay_iteration; # delay in seconds before starting next iter } # output raw timing results open LOG, ">$logfile" or die "Can't create $logfile: $!"; for my $i (0..$#start_times) { printf LOG ("%.6f,%.6f\n", $start_times[$i], $stop_times[$i]); } close (LOG); exit;

  38. loadad.pl (part 1) #!/usr/bin/perl # load Active Directory server use strict; use warnings; use Getopt::Long; use Sys::Hostname; use Time::HiRes; use Authen::Simple::ActiveDirectory; my $ctrl_c = 0; # initialize $ctrl_c flag $SIG{INT} = sub { $ctrl_c = 1 }; # set the $ctrl_c flag on interrupt my $iterations = 3; my $delay_iteration = 0; # delay between iterations in seconds my $test_run; # test run identifier string e.g. run01, aurnn14, or xyz my $servername; # servername (unix hostname) my $domain; my $names;

  39. loadad.pl (part 1) #!/usr/bin/perl # load Active Directory server use strict; use warnings; use Getopt::Long; use Sys::Hostname; use Time::HiRes; use Authen::Simple::ActiveDirectory; my $ctrl_c = 0; # initialize $ctrl_c flag $SIG{INT} = sub { $ctrl_c = 1 }; # set the $ctrl_c flag on interrupt my $iterations = 3; my $delay_iteration = 0; # delay between iterations in seconds my $test_run; # test run identifier string e.g. run01, aurnn14, or xyz my $servername; # servername (unix hostname) my $domain; my $names;

  40. loadad.pl (part 2) my $username; my $password = "xxxxxxxx"; my $outdir = "raw_results/"; my @start_times = (); # start timestamps in floating point seconds my @stop_times = (); # stop timestamps in floating point seconds &GetOptions( "iterations:i" => \$iterations, "delay_iteration:i" => \$delay_iteration, # delay 'tween iterations "test_run:s" => \$test_run, # test run identifier "servername:s" => \$servername, # servername "names:s" => \$names, # pathname of file of user names "domain:s" => \$domain, # Active Directory domain ); # if test_type subdir does not exist, create it. # substitute "test_type" with the real one e.g. "dir" mkdir ($outdir . "loadad") unless -e ($outdir . "loadad"); $outdir .= "loadad/"; my $logfile = $outdir . $test_run . "~$servername~" . hostname() . "~log.$$"; # $logfile example: raw_results/dir/run01~devldap~client01~log.3257 open NAMES, "<$names" or die "Can't open $names: $!";

  41. loadad.pl (part 2) my $username; my $password = "xxxxxxxx"; my $outdir = "raw_results/"; my @start_times = (); # start timestamps in floating point seconds my @stop_times = (); # stop timestamps in floating point seconds &GetOptions( "iterations:i" => \$iterations, "delay_iteration:i" => \$delay_iteration, # delay 'tween iterations "test_run:s" => \$test_run, # test run identifier "servername:s" => \$servername, # servername "names:s" => \$names, # pathname of file of user names "domain:s" => \$domain, # Active Directory domain ); # if test_type subdir does not exist, create it. # substitute "test_type" with the real one e.g. "dir" mkdir ($outdir . "loadad") unless -e ($outdir . "loadad"); $outdir .= "loadad/"; my $logfile = $outdir . $test_run . "~$servername~" . hostname() . "~log.$$"; # $logfile example: raw_results/dir/run01~devldap~client01~log.3257 open NAMES, "<$names" or die "Can't open $names: $!";

  42. loadad.pl (part 3) # main processing loop foreach my $iter (1..$iterations) { $username = <NAMES>; # read a line from file of user names if (!($username)) { # if end-of-file seek NAMES,0,0; # point to beginnning of file $username = <NAMES>; # and read the first line } chomp $username; # discard newline push @start_times, Time::HiRes::time; # save start timestamp my $ad = Authen::Simple::ActiveDirectory->new( host => $servername, principal => $domain, ); if ( $ad->authenticate( $username, $password ) ) { # print "Good authentication!\n"; # for debugging single instance } else { print "Bad authentication!\n"; } push @stop_times, Time::HiRes::time; # save stop timestamp last if ($ctrl_c eq 1); # exit loop on interrupt sleep $delay_iteration; # delay in seconds before starting next iter }

  43. loadad.pl (part 3) # main processing loop foreach my $iter (1..$iterations) { $username = <NAMES>; # read a line from file of user names if (!($username)) { # if end-of-file seek NAMES,0,0; # point to beginnning of file $username = <NAMES>; # and read the first line } chomp $username; # discard newline push @start_times, Time::HiRes::time; # save start timestamp my $ad = Authen::Simple::ActiveDirectory->new( host => $servername, principal => $domain, ); if ( $ad->authenticate( $username, $password ) ) { # print "Good authentication!\n"; # for debugging single instance } else { print "Bad authentication!\n"; } push @stop_times, Time::HiRes::time; # save stop timestamp last if ($ctrl_c eq 1); # exit loop on interrupt sleep $delay_iteration; # delay in seconds before starting next iter }

  44. Some utility scripts in the distribution stop.pl psignal.pl check_generators.pl makelists.pl makelistoflists.pl servermon.pl swingbench.pl

  45. Real-world Experience • Ability to integrate load scripts written in other languages. • World’s first load test of Kerberos authentication servers. • Driver for other simulators. • LoadGen used for LDAP Directory, Active Directory, pop3, imap4, Remedy, Kerberos and Jabber servers.

  46. General Comparison with COTS

  47. Some Relevant References UnixLoadGen User’s Guide, by Hummel (included in distribution) SSH The Secure Shell by Barrett & Silverman, publisher O’Reilly Linux System Programming by Love, publisher O’Reilly Programming Perl 3rd Ed. by Wall, Christiansen & Orwant, publisher O’Reilly

  48. Acknowledgements CSC Management Perl Advisors Mike Gross Peter Scott Todd Lucas Patrick Ward Eric Gerritsen Thomas Berry Virinder Dhillon JPL Management Kevin Klenk cpan.org sponsors & Sarala Rajeshuni authors, specifically Henry Dillard Christian Hansen Luke Dahl

  49. National Aeronautics & Space Administration Load Testing Using Perl • Homer Hummel • Jet Propulsion Laboratory, • California Institute of Technology • Homer.Hummel@jpl.nasa.gov • 2008-07-25

More Related