1 / 51

Intermediate Unix

Intermediate Unix. Presented July 29 th , 2001 by: “Robin” R. Battey (zanfur@cs.washington.edu) Evgeny Roubinchtein (evgenyr@cs.washington.edu) with tips, suggestions, and corrections by: Hannah Tang (hctang@cs.washington.edu) http://www.cs.washington.edu/people/acm/tutorials/.

saxon
Download Presentation

Intermediate Unix

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Intermediate Unix Presented July 29th, 2001 by: “Robin” R. Battey (zanfur@cs.washington.edu) Evgeny Roubinchtein (evgenyr@cs.washington.edu) with tips, suggestions, and corrections by:Hannah Tang (hctang@cs.washington.edu) http://www.cs.washington.edu/people/acm/tutorials/

  2. The “File System” • Under UNIX, (almost) everything is a “file”: • Normal files • Directories • Hardware • Sockets • Pipes • Things that are not files: • Users • Groups • Processes

  3. Ownership • Files have two owners • Every file has exactly one user owner • Every file has exactly one group owner • Everyone is a user • Users are in at least one group • Processes have owners, too (known as an “id”) • Every process has exactly one user id • Every process has at least on group id • Users and groups are really just numbers with names • Every username is mapped to a single numeric “uid” • Every groupname is mapped to a single numeric “gid”

  4. Who am I? • Commands that tell you who you are: • whoamidisplays your username • iddisplays your username and groups • Commands that tell you who others are: • finger [<name>]displays info for <name> • id [<username>]displays info for <username> • Commands that change who you are: • su <username>“switch user” to <username> • login login as a different user

  5. File Permissions • Every file has three access levels: • user (the user owner of the file) • group (the group owner of the file) • other (everyone else) • At each level, there are three access types: • read (looking at the contents) • write (altering the contents) • execute (executing the contents)

  6. Strange Things • There are three “strange” permissions: • setuid (run program as user owner) • setgid (run program as group owner) • text (stay in swap after executing) • Directories act differently • “write” (creating/deleting files) • “execute” (cd-ing to that directory) • “setuid” (ignored) • “setgid” (created files have same group owner) • “text” (deletion restricted to user owned files)

  7. Examining Permissions • A “long” ls listing shows file permissions: [zanfur@odin zanfur]$ id uid=8774(zanfur) gid=100(users) groups=100(users),101(mp3) [zanfur@odin zanfur]$ ls -l total 524 -rwxr-xr-x 1 zanfur users 512668 Jul 31 00:18 bash prw-r--r-- 1 zanfur users 0 Jul 31 00:24 fifo -rw-r--r-- 1 zanfur users 0 Jul 31 00:18 file drwxr-xr-x 2 zanfur users 4096 Jul 31 00:16 normal drwxrws--T 2 zanfur mp3 4096 Jul 31 00:14 shared drwxrwxrwt 2 zanfur users 4096 Jul 31 00:14 tmp drwxrwxr-x 2 zanfur www 4096 Jul 31 00:15 www [zanfur@odin zanfur]$ _

  8. Permissions Listing Breakdown

  9. What You Can Do With Permissions

  10. Changing Ownership • Changing user ownership: • The command is “change owner”: chown <username> <filename> • but, you can only do it if you are root • so just copy it instead • Changing group ownership: • The command is “change group”: chgrp <groupname> <filename> • but, you can only do it if you are in group <groupname> • and you must be the user owner of the file

  11. Changing Permissions • The “change mode” command: chmod <level><op><permissions>[,…] <filename> <level> string of: u, g, o, a (user, group, other, all) <op> one of +, -, = (gets, loses, equals) <permissions> string of: r, w, x, s, t, u, g, o (read, write, execute, set-id, text, same as user, same as group, same as other), • Examples: chmod u+rwx,go-w foobar chmod g=u,+t temp/ chmod u=rwx,g=rwxs,o= shared/

  12. Process Management • What can you do with it? • Start programs in the background • Run more than one program per terminal • Kill bad and/or crashing programs • Suspend programs mid-execution • List all jobs running in a shell • Move foreground jobs to the background • More …

  13. Three States of a Process • Foreground • Attached to keyboard • Outputs to the screen • Shell waits until the process ends • Background, running • Not attached to keyboard • Might output to the screen • Shell immediately gives you another prompt • Background, suspended • Paused mid-execution • Can be resumed in background or foreground

  14. Background Processes • Listing jobs: • jobs lists background “jobs” and job #’s • ps lists processes and their process id (“pid”) • %<job#> expands to the process id of the job • Stopping foreground jobs • Press ^Z (Ctrl-Z) in the terminal window • Starting a process in the background • Append a & character to the command line • Examples: ls –lR > ls-lR.out & xemacs my_program.cc & • Resuming a stopped job • In the foreground: fg [<pid>] • In the background: bg [<pid>]

  15. Killing Processes • The “kill” command: kill [-<signal>] <pid> Send <signal> to process <pid> • The “killall” command: killall [-<signal>] <command> Send <signal> to all processes that start with <command> • Useful signals (kill –l for the complete list): TERM the default, “terminate”, kills things nicely KILL will kill anything, but not nicely HUP “hangup”, used to reload configurations STOP stops (suspends) a running process

  16. What are environment variables? • The environmentis an array of strings of the formNAME=VALUE • Each process (program) has its own copy of the environment • Processes started by the shell get a (“deep”) copy of the shell’s current environment • Each process can examine and modify its own (and only its own) environment • The “meaning” of environment variables is merely a matter of established conventions

  17. Viewing your shell’s environment • To view a single environment variable’s value:echo $<name> • For example:echo $HOMEwill display/homes/iws/evgenyr • To view all environment variables at once: • tcsh/csh/bash: printenv • sh/ksh: set

  18. Setting environment variables • tcsh/csh: • setenv <name> <value> • Example: setenv PRINTER ps329 • bash/ksh: • export <name>=<value> • There is no space on either side of ‘=’! • Example: export PRINTER=ps329 • sh: • <name>=<value>; export <name> !

  19. Appending to environment variables • Appending /uns/bin/ to your PATH • bash/ksh: export PATH=$PATH:/uns/bin • tcsh/csh: setenv PATH=$PATH:/uns/bin • Prepending is similar: • bash/ksh: export INFOPATH=/uns/info:$INFOPATH • tcsh/csh: setenv INFOPATH /uns/info:$INFOPATH

  20. Common environment variables • The “meaning” of environment variables is purely a matter of convention. • However, these environment variables are quite common: • PATH, INFOPATH, MANPATH, EDITOR, VISUAL, PAGER, HOME, MAIL, USER • The man page for a program will usually list the environment variables the program pays attention to.

  21. Useful shell features (a preview of upcoming attractions) • Aliases • Redirecting input and output • Command substitution • Scripts

  22. Shell as a work-saver: aliases • Aliases: textual substitution, similar to C-preprocessor macros. • Syntax: • bash/ksh: alias <text>=’<replacement>’ • tcsh/cshalias <text> ’<replacement>’ • When you type <text>, the shell “substitutes” <replacement> • Typing alias by itself lists all your current aliases • unalias <text> removes the alias • Check the csh/tcsh man page for extra features

  23. Line continuation character, just like in C/C++! Alias examples • Always print postscript files double-sided • tcsh/csh:alias lpr ’lpr –Zduplex’ • bash/ksh:alias lpr=’lpr –Zduplex’ • Collect a few tree-friendly options to enscript: • tcsh/csh:alias print \ ’enscript –2rhB –SDuplex=DuplexNoTumble’

  24. What is input/output redirection? • Normally, a program’s standard output is displayed on user’s terminal, and its standard input comes from the keyboard. • Redirecting the output of a program means asking the shell to put the program’s output (stdout [C++’s cout]) into a file. • Redirecting the input of a program means asking the shell to feed a file as the program’s standard input (stdin [C++’s cin]). • Note: redirection works with files.

  25. Why redirect program’s output? • You may want to save output and examine it at your leisure: • if the program generates a lot of output • if the program takes a long time to run • You may want to present the output to another person • Having the program write to standard output may make the program simpler to write

  26. Standard output vs Standard error • By convention, “normal” output of a program is sent to standard output (stdout [C++’s cout]), while debugging or error output is sent to standard error (stderr [C++’s cerr]).

  27. How to redirect program’s output? • To redirect just the standard output:<program> > <FILE> • To redirect just the standard error:sh/ksh/bash: <program> 2> <FILE>csh/tcsh: ( <program> > STDOUT ) >& STDERR • To redirect both standard output and standard error: csh/tcsh/bash: <program> >& <FILE>sh/ksh/bash: <program> > <FILE> 2>&1

  28. > vs. >> • Both > and >> will create the output file, if it doesn’t already exist • If the file does exist, then: • Using > to redirect output will overwrite the output file: • ls > newlisting • printenv > my_environment • Using >> to redirect output will append to the output file • cat ch1 ch2 ch3 > book • cat ch4 ch5 ch6 >> book

  29. Why redirect program’s input? • To run the program repeatedly with the same (or similar input) • Having the program read from standard input may make the program simpler to write.

  30. How to redirect program’s input? • Simple!<program> < <FILE> • Examplesort < my_grades.txthead < really_long_book.txt

  31. Piping • Piping is connecting programs together by using the output of one program as the input to the next. • Syntax:<program1> | <program2> | … | <programN> • A simple example (view a sorted file-listing a page at a time):ls | sort | less • Note: piping deals with the input/output of programs (that is, stdin, stdout, stderr)

  32. Why piping? • Because “the whole is bigger than the sum of its parts. • By combining Unix utilities in a pipeline, you can build tools “on-the-fly” as you need them.

  33. Piping examples • How many .c files are in this directory?ls *.c | wc –l • What files were modified most recently?ls –t | head • What processes am I running?ps auxw | grep <mylogin> • Make an alias for the above, for other Linux boxen too:alias myps=’ ps auxw | grep `id –un`’

  34. Thecatutility • Especially useful: cat • When used with pipes, “copies” stdin to stdout • cat can be used to redirect out of a pipe! • Example: make a file containing currently running processesps aux | grep evgenyr| cat > my_processes • When used with a file, “copies” the file into stdout • cat can be used to place a file’s contents into a pipe! • Example: list all the items in a list in alphabetical ordercat my_grocery_list.txt | sort | uniq

  35. Command substitution(aka “what’s up with the backquotes?”) • Command substitution means that the shell substitutes a command’s output for the command itself. • To get the shell to perform command substitution, enclose the command in backquotes (`)

  36. Shell as a work-saver: scripts • Instead of typing the same series of commands over and over again, put them in a file and have the shell execute the (commands in the) file! • The file must have execute permission • The first line of the file should be:#! <YOURSHELL> (e.g., /bin/bash) • There must be no space (or any other character) between ‘#’ and ‘!’. • Whether to put space after ‘!’ is a matter of style • Shell also has control flow, tests, etc. The shell tutorial on the ACM web page goes into much more detail.

  37. Intermezzo: quoting • Problem: some characters are special to the shell (*, ?), but you would like to pass those characters to the programs the shell runs as-is. • So you quote the character(s): • A single character: by prefixing it with a \ (backslash) • A string of characters: by enclosing them in • Single quotes: no characters are special. None • Double quotes: some characters (notably $ and `) are still special. You’ll need to prefix those with a backslash to pass to the program as-is.

  38. Quoting examples • List all the files in the current directory:ls * • List the file or directory called ‘*’ (or complain if it’s not there):ls \* • Print $HOME:echo ’$HOME’ • Print the path to your home directory:echo ”$HOME” • Print `id –un`: echo ’`id –un`’ • Print your login:echo ”`id –un`”

  39. Next stop: utilities • (No, not electricity, water, and sewer) • diff and patch • grep • find and xargs

  40. diff and patch • You have two versions of a file; how do you see what’s changed between the two versions? • diff shows the differences between two files; you’ll want to use the –u or –c option to diff to produce output in human-friendly format:diff –u my_file my_file.orig | less • patch applies differences to a filediff –u my_file my_file.orig > my_patchpatch my_file < mypatchmy_file is now the same as my_file.orig

  41. grep – search for patterns • grep searches for patterns in texts:grep <string> <FILE> • grep uses regular expressions to describe the string(s) to search for • In a regular expression, most characters are treated as-is, but a few are magic • Ordinary characters just represent themselves • Magic characters do special things

  42. grep’s magic characters • A few different kinds of magic characters: • Some characters “anchor” (parts of) a regular expressions to specific places in the string:^ - The beginning of a line$ - The end of a line • A dot (.) matches “any one character whasoever” (except for newline) • [ ]form a character class: • [aeiou] – any single vowel • [a-zA-z0-9] – Any single letter or digit • [^0-9] – Any single character that isn’t a digit Here, ‘^’means “not”

  43. Even more of grep’s magic characters • Quantifiers say how many times the preceeding “thing” should be repeated: • * means “zero or more times” • ? Means “zero or one time”

  44. Frequently used grep options -i : do case-insensitive search -n : print line numbers -v : print lines that do not match -l : only list the files from which output would have been printed

  45. grep examples • Print all non-blank lines:grep –v ’^$’ my_file.txt • Print all the lines on which my_function is called (or declared):grep –n ’my_function *(’ my_code.c • Show all the xterms I am running:ps auxw | grep ”`id –un` .*xterm”

  46. find find <PATHS> <OPTIONS> <EXPRESSION> • Traverse the tree(s) rooted at <PATHs>, and for each “thing” found, evaluate the <EXPRESSION> until the result of <EXPRESSION> is known. • The evaluation of <EXPRESSION> “short-circuits”, just like expressions in C/C++. • false && some_expression can never be true • Options apply to the entire find command, not the individual expression

  47. Components of a find expression • An expression is zero or more primaries connected by operators • Two kinds of primaries: • Tests: just return true or false • Actions: do something, in addition to returning true or false • Operators work just as they do in C/C++. • ! / -not • -a / -and • -o / -or • , (comma) • Parentheses are used for grouping, just as in C/C++.

  48. find examples • Print all the *.c* and *.h* files in and below the current directoryfind . –name ’*.[ch]*’ –a –print • Show line numbers myfunction calls in the above *.c* filesfind . –name ’*.c*’ | xargs grep –n ’myfunction.*(’ • Change permissions on files under your home directory so they’re inaccessible to everyone but you (except for files in the www directory)cd; find . –path ”./www*” –prune –o –exec chmod go-rwx {} \; • How big are my *.c* files?expr `find –name ’*.c*’ –printf ”%k + ”` 0 • Read the find manual (info find) for more examples

  49. xargs: combine arguments xargs <OPTION> <COMMAND> <INITIAL_ARGS> • Feeds standard input as arguments to <COMMAND> • Often used with find(find … | xargs grep) • But it doesn’t have to be used with find:cat filelist | xargs cat {} > concatenated

  50. Finding more information – at CSE • Use info and man • Peruse the “see also” section of the man pages • Peruse info’s i (search index) command • The uw-cs.lab-help group • Look at your .login, .cshrc, etc. to see how support has set things up

More Related