Software Tools Design

Software Tools Design

Classical architecture Design Representations plan, elevation, perspective drawings/models Architectural Styles Victorian Colonial Spanish Requirements,Constraints circulation patterns acoustics lighting air flow Software architecture Design Representations multiple views models for different roles (customer, developer) Architectural Styles Client/server Layered Object-Oriented Requirements,Constraints reliability, fault tolerance performance maintainabililty Architecture – the design and and specification of overall system structure

As size/complexity of software increases, design/spec of overall system structure becomes more significant than the choice of algorithms/data structures Currently not a well-defined terminology available for software arch. but good software engineers do use architectural principles Software architecture: defines a system in terms of computational components and interactions among them. Software Architecture

Architectural Elements • components: define computation,data storage • clients/servers, databases, filters,... • connectors: define the interactions between components • procedure calls, pipes, event multicast, … • configurations: collections of interacting components and connectors • diagrams of boxes and arrows, formal mathematical models • constraints: provide semantic understanding of the style requirements • what makes the style correct, legal, efficient, secure, reliable

Architectural Styles • Dataflow systems • batch sequential • pipes and filters • Call-and-return systems • main program and subroutine • OO systems • Hierarchical layers • Independent components • communicating processes • event systems

Architectural Styles, cont. • Virtual machines • interpreters • rule-based systems • Data-centered systems (repositories) • databases • hypertext systems • blackboards

This class covers: • Pipes and Filters: Unix shell • Object-Oriented: Java • Client/Server: • Web Applications and XML, Javascript • CGI programming with Perl • Applets, Servlets

Architectural Style Questions: • What is the design vocabulary? (components, connectors?) • What are the allowable structural patterns? • What is the underlying computational model? • What are the essential invariants of the style? • What are some common examples of its use? • What are the advantages/disadvantages of using the style? • What are some common specializations?

Common Framework • Use a common framework in order to compare different arch. styles • framework: collection of computational components (components), description of interactions (connectors), set of (constraints) on how they can be combined • For each style: • structural pattern • underlying computation model • essential invariants • common examples • advantages/disadvantages • common specializations

Pipes & Filters • Components: • filters (processing components): incrementally transform data (stream to stream) from a set of inputs to a set of outputs. • Connectors: • pipes: conduits for the data stream. transmit output of one filter to input of next filter • Invariants: • filters (components) must be independent entities; • little context info is utilized, no state info preserved between instantiations • filters do not know the identity of their upstream and downstream filters. • Examples: unix shell, compilers

Pipes & Filters • Advantages: • neat design; support reuse; easy to maintain • hierarchical decomposition • black box approach • easy to reuse; support concurrency • support specialized analysis (deadlock, throughput) • Disadvantages: • batch organization of processing • not interactive • difficult to synchronize pipes, order filters • performance penalty : forces lowest common denominator of data (often ASCII) • parsing/unparsing overhead • infinite queues may be required • possible deadlock with finite buffers

Data abstraction, OO • Components: objects/instances of ADT • Connectors: procedure invocation • Variations: • allow multiple interfaces • (member function access) • allow concurrency • Advantages • encapsulation/info hiding • can change implementation without affecting clients • inheritance: shared definitions of functionality • polymorphism: actual operation determined at runtime • reuse/maintenance: promotes separation of concerns • Disadvantages • objects must know id for interaction, change of id requires notification

Event-based, Implicit Invocation • Components: objects/processes with interfaces that: • define allowable incoming, outgoing events • Connectors: event procedure bindings • procedures are registered with events • components interact by announcing events • announcing component does not know who registered for the event • upon event occurrence, associated procedures are implicitly invoked • order of invocation is non-deterministic • Main invariant: event announcer does not know who will respond • Examples: GUIs, databases • Advantages: • strong support of reuse (event registration) • eases system evolution (replace components without affecting interfaces) • Disadvantages: • less control of computation, data exchange difficult • difficult to reason about correctness • harder to understand flow, test

Client/Server • Components: server contains processing and data resources required by clients • Connectors: packets of client requests and server replies • Examples: distributed systems, web applications • Advantages: • strong support of reuse • support parallel computation • Disadvantages: • complexity • performance (data transmission) • security

Major Unix OS features: Kernel Shell File System Kernel Manages memory and allocates it to each process Schedules work done by the CPU Organizes transfer of data from one part of machine to another Accepts instructions from shell and carries them out Enforces access permission on the file system The Structure Of a Unix OS

UNIX OS • Shell • Command interpreter • Create customized environments • Write shell scripts • Define command aliases • Manipulate command history • File and command completion • Edit the command line

Unix OS • File System • Logical method for organizing and storing large amounts of information. • Easy to manage. • File: basic storage unit. • Types: • ordinary file (stores info) • directory (holds other files and directories) • special file (represents physical devices like printers, terminals, etc) • pipe (temporary file for command linkage)

Unix Operating System Structure • OS mediates between the user and the computer User Application Programs Kernel Shell Hardware

UNIX: Multi-user Multi-tasking • More than one user can run at the same time and more than one task can run at the same time • In Unix, each program is started as a process. • A process is a program in execution. • Usually only one copy of a program, but there may be many processes running the same program. • To each interactive user (window): • only one process in foreground • may have several processes in background

Unix Process • The fundamental unit of a multi-user UNIX operation. • A program in execution, doing some work. • Multiple users and multiple processes. • CPU is switched rapidly from one process to the other. • The shell uses the UNIX process structure to carry out user commands.

Command Execution From Shell: • Typically a shell process forks a child process to execute the command. • The child process execs the utility program named in the command. • Now two processes are running for the user - the shell (parent) and the utility (child) process. • Running the command in the foreground: shell waits until the child process is complete. • Running the command in the background: • the shell prints the process id of the child process and waits for the next command.

Processes Process 0: Kernel bootstrap. Start process 1. kernel kernal mode user mode Process 1: create processes to allow login. /etc/init fork exec fork exec inetd lpd condition terminal for login /etc/getty /etc/getty httpd exec exec check password /bin/login /bin/login exec exec command interpreter shell shell

Unix Process • Init process • last step in booting procedure • create other processes to allow the users to login • Getty process • conditions for terminal connection • wait for user-id • display login on the screen • Login process • check password with the uid • execute .profile or .login (depends on default shell) • display shell prompt • Shell process (command line interpreter) • Shell prompt ($, %)

UNIX Process • Process environment • Process id, parent-process-id, process-group-id • Opened files • Working directory • File creation mask • User ID, Group ID • Resource limits • Environment variables • Code • A child process inherits parent’s environment.

Processes use ps to see the processes that you are running. $ ps PID TTY TIME CMD 221 pts/4 4:15 netscape 201 pts/4 0:05 bash 215 pts/4 1:15 emacs-19 • use “&” to execute a task in background • Example: $ sort infile > outfile & • use ctrl-Z to suspend the foreground task, and then use bg. • use fg to move a background task to foreground.

Shell: Command Interpreter • Bourne Shell: the default shell (sh) • original unix shell • does not have interactive features of newer shells • widely used for writing shell scripts • standard on Unix System V • C Shell (csh): available on BSD Unix and most other systems • with syntax similar to the C language • with many enhancement over the Bourne shell. • Korn Shell (ksh): AT&T’s answer to C Shell • combine features of Bourne and C shells • very efficient • Other shells: tcsh, bash

Day-to-Day Use Command Function Meaning cat Display a file conCATenate cp Copies a file CoPy mv Renames a file or moves it MoVe rm Delete files ReMove lpr Sends a file to a printer Line Printer lp (Sys V) ls Lists the contents of a directory LiSt chmod Changes the “mode” of permissions Change MODe pwd Shows the current directory Print WorkingDir cd Change current directory Change Dir mkdir Create a directory MaKe DIR rmdir Delete a directory ReMove DIR ps Shows the processes on the system Process Status man Shows info. about commands Manual df Shows file system status Disk File du Shows the space allocation of files Disk Utilization grep Search for patterns in files

Standard Command Format • command [options] [arguments] wc [-c | -m | -C] [-lw] [file …] • stuff in brackets is optional • boldface words are literals (must be typed as is) • italicized (or <> enclosed) words are args (replace appropriately) • Options modify how the command works • command [options] [--] [<file> …] options ::= -option white-space options* option ::= noargoption* argoption | noargoption+ noargoption ::= letter argoption ::= letter whitespace string $cc -po zap zap.c

Some Examples • ls [-alRF…] file-list • a for listing all files including the dot files • l for long format (file type, permissions, #links, owner, group, etc) • R for recursive, list subdirectories. • F for listing directories with a trailing / • ps [<options>] • List the information about running processes • Example: %ps -el # print the info about all processes (e) in the long format (l)

I/O Redirection • Redirection and Pipe “>“ redirects standard output (screen) to a file E.g. ls > dirlist “<“ redirects standard input (keyboard) from a file E.g. sort < infile > outfile “|” pipe output of program 1 to input of program 2 E.g. who | wc Or getit < in | check_it | process_it | format_id > out “>>“ appends output to a file E.g. ls -l >> oldfile Exercise: find out the definition of “tee”.

Sequential vs. Concurrent Process • Sequential: %date %ps -ef OR %date; ps -ef; who %who • Concurrent: %pgm1 & prgm2 > file1 & pgm3 %make > log & %sort +1 pdir; ((pr dir | lpr) & sort +1 local))

File Name Expansion • Each shell program recognizes a set of special characters called meta characters. • The metacharacters are used to create patterns to match filenames and command names. • Bourne and Korn shell meta/wildcard characters • * matches any string (including null) • ? matches any one character • [a-dA-D] matches any one character in the range • [!AB123] matches a char not in this range • \ escape • ~<username> : (not bourne shell) the home dir of the user.

File Name Expansion • Assume we have the following files under the current directory: 120, 235, 2#5, a.out, c22, c*2, Doc.1, Doc.2, Doc.3, one.c,two.c, three.c ls *.c ls c*2 ls [a-z]*[!0-9] a.* *.* ls ??? cd ~foo ls *

Filters • Most UNIX utilities are filters • A filter is a program which • reads input (text) from standard input only • writes output to standard output only • writes error to standard error only • may use temporary files for intermediate results • Filters can be combined to work together using pipes • Pipe: takes stdout of one command and uses it as stdin of another command ls | wc

Command Alias • Assign your own name for a command • Syntax is shell dependent alias ll ‘ls -l’ C shell alias ll=‘ls -l’ Korn, Bourne shell • Displaying the value of an alias alias ll (displays value)

Unix File Systems • File: a collection of data • Just a sequence of bytes • no record or block structure required • Directory • A file that contains information for files • distinction between a directory and a file • system can alter the contents of a directory • rooted tree file structure (inverted tree) • directories can contain both files and other directories info.

Unix File System Road Map • Special files: /dev/* represents I/O devices. / /dev /etc /var /bin /tmp /usr /mnt /home passwd hosts …... spool adm include lib sue john etc bin 5bin printer mail(in) mail(out) uucp messages wtmp …

File Systems and the I-nodes • Each disk drive contains one or more file systems • Each file system occupies a number of cylinder groups. • Each file system has a superblock, an i-node table and files • The superblock is at a fixed location relative to the beginning of the file system. It contains a description of the file system. • One can find the location of the I-node table thru superblock. • Each entry of the I-node table is an I-node, which uniquely represents a file in the file system. • An I-node contains uid, gid, time of creation, modification, and access, permissions, location of the file on the disk. superblock I-node table file1 file2 free file3 free

An I-node includes the following information: • File access permissions • Owner and group ids • File link count • Time of last modification • Time of last file access • Location of blocks for each regular file and directory file • Device id for special files I-node 2 contains the location of the block with the root directory

chmod • Change the access permissions of a file • chmod <permissions> <filename> • permissions can be specified as 3 octal digits, <user,group,others>, the three bits of an octal means r,w,x • Example: chmod 755 test • permissions can be specified as +x, or u+x, or g+r, … • “chmod +s test” sets the set-uid bit for file test. • If a directory has “x” in its permission, the dir is searchable, ie., one can do “ls” on the directory.

Pathnames • Every file and directory in the file system can be identified by a “full path name” (route from root to file) /home/sue/email/f1 • Relative path name • location relative to current directory . Current directory .. Parent directory • if cwd is /home/sue: ls email ls ./email cd .. ls ../fred / home sue fred email docs f2 f1

Some Details • cp [-ir…] file1 file2 cp [-ir…] file-list directory • i for interactive. prompt use whenever a file will be overwritten • r for recursive. copy a directory tree • ls [-alRF…] file-list • a for listing all files including the dot files • l for long format • R for recrusive. list the all subdirectories. • F for listing directories with a trailing / • date [+format] • % date ‘+%h %d, 19%y’

Some Details (cont.) • wc file-list • display the number of lines, words and characters • more file-list • Browse through text files on page at a time. • head [-n …] file-list • Display the first n lines of the files (default=10) • tail [+n|-n| -f| …] • Display the last few lines in the files (default = 10) • Example: %tail +5 foo # display the last parf of foo starting from line 5 %tail -5 foo # display the last five lines of foo %tail +30 foo | head -15 | more #display line 30-45 of foo %tail -f foo # wait and display the new lines appended to foo

Some Details (cont.) • cut -c list file • cut -f list [-dChar] file • Cut out selected characters or fields from each line of a file • Examples: %cut -c 1-5,15-20 foo # extract chars 1-5 and 5-20 from each line of foo. %cut -f 1,3 -d” “ moo # extract field 1 and 3 from each line of moo. • paste file1 file2 • Concatenate corresponding lines of the given input files • Example (reverse two fields of the file abc) %cut -f1 abc > abc1 %cut -f2 abc > abc2 %paste abc2 abc1 > xyz

Some Details (cont.) • grep, egrep, fgrep grep [-nv...] pattern [file-list] • Search the input text files for lines matcing the pattern %grep Unix doc.1 # Display the lines in doc.1 that contains “Unix” %grep -n Unix doc.* # Display the lines with line numbers %grep -v Unix doc.1 # Display the lines which do not contain “Unix”

Some Details (cont.) • sort [-tC…] [-o outfile] [field-list] [file-list] • sort the files %sort +1 list1 # sort list 1 starting from field 2 to the end of the line %sort +2-3 list2 # sort list2 on the third field %sort -n -o list4 list3 sort list3 numerically and place the output in list4

diff • diff file1 file 2 • Display different lines that are found when comparing two files • It prints a message that users ed-lide notation (a - append, c - change, d -delete) to describe how a group of lines has changed. • It also describes what changes (additions, deletions and changes) need to be made to the first file tomake it the same as the second file.

diff Example file1 file2 file3 apples apples oranges oranges oranges bananas bananas kumquat s kiwis peaches

diff (cont.) %diff file1 file2 3c3 <bananas ----------------- >kumquats %diff file1 file3 1d0 <apples 3a3,4 >kiwis >peaches

comm file1 file2 • Takes two sorted text fields and print common lines and lines which appear exclusively in one file on separate columns. • column1: lines only in file1, column 2: lines only in file2; col 3: comm • Example file1 file2 %comm file1 file2 apple apple apple banana cantaloup banana grape grade cantaloup orange kiwi grape lemon kiwi %comm -12 file1 file2 apple grape

Software Tools Design