UNIX Chapter 10 Advanced File Processing

1. UNIX Chapter 10Advanced File Processing Mr. Mohammad Smirat

2. Regular Expression • Some of the commonly used tools that allow the use of regular expressions are awk, ed, grep, egrep, and vi, but the level of support for regular expression isn’t the same for all these tools.

3. Regular Expressions (con…)

4. Regular Expression (cont…) • The following are some of vi commands which illustrated the use of regular expressions: • / [0-9] / Do a forward search for a single stand alone digit.?\.c[1-7]?Do a backward search for words or strings that end with .c followed by a single digit between 1 and 7. • :1,$s/:$/./ search the whole file and substitute colon (:) at end of a line withperiod(.). • :.,$s/^[Hh]ello /Greetings/ from the current line to the end of file, substitute the word hello and Hello, starting a line with the word Greeting. • :1,$s/^ *// eliminates one or more spaces at the beginning of all the lines.

5. Regular Expression (cont…) • A friend helped you type a very long report, but he consistently typed “thorogh” instead of “thorough”, he typed line 223 through 456 fix the document in one command? :1,$ s/thorogh/thorough/g :223,456s/thorogh/thorough/g • The phone company has moved the 710 exchange to a new area code, so you must include the area code023 before you dialing all 710 exchange numbers, fix this problem to the phone company file in one command?

6. Compressing Files • compress[options][file-list]The output will be compressed .Z file or standard output in input is from standard input. • Options:-c write compressed file to the display screen instead of a .Z file.-f force compression (no prompt)-v display compression percentage and the names of compressed files. $compress -v t1 t2 t1: -- replaced with t1.Z compression: 14.50% t2: -- replaced with t2.Z compression: 17.89% $uncompress -v t1.Z t2.Z t1.Z: -- replaced with t1 t2.Z: -- replaced with t2

7. Compressing Files (cont..) • zcat[options][file-list]Concatenate compressed files in their original form and send them to standard output. $zcat t2.z This file is being used to explain test two. $ • The file t2.z remains.

8. Sorting Files • sort[options][file-list]sorted lines in ASCII files in file list • Options:-b ignore leading blanks-d sort according to usual alphabetical order, ignore all chars except letters.-f consider lowercase and uppercase to be equivalent+n1[-n2] specify a field as the sort key, starting with +n1 and ending at -n2, or end of line if n2 is not specified.-r sort in reverse order

9. Sorting Files (cont…) • $ sort studentsfile will be sorted by all chars from left to right. • $sort +1 studentsfile will be sorted starting with the second field of the file (field # 1), because the first field is field #0. • $sort +3 -r -b studentsfile will be sorted starting with the fourth field, in reverse order and ignoring the leading white spaces. • $sort +1 -2 +3 -b studentfile will be sorted starting with the second field and ending with the third field (primary key), the secondary key starts at the fourth field and ends at the end of line, ignore leading white spaces.

10. Searching for commands and Files • find directory-list expressionSearch the directories in the list to locate files that meet the criteria described by the expression. • Options:-inum N search for files with inode number N-name pattern search for files that are specified by the pattern.-user name search for files owned by user name.-links N search for files with N links.

11. Searching for commands and Files (CONT…) • $find ~ -name JUST.gif will search the home directory and display the pathname of the directory that contains JUST.gif. • $find /usr/include -name socket.h will search directory /usr/include for socket.j and print the absolute pathname of the file. • find /usr . -inum 2342 will search /usr and . (pwd) directories for all the files that have an inode number 2342 and print the absolute pathname of all such files.

12. Searching Files • grep[options] pattern [file list]egrep[options][string][file list]fgrep[options][expression][file list]Search the files in file list for given pattern, string, or expression, if no file list, take input from standard input. • Options:-c print the number of matching lines only-i ignore the case of letters-l print only the names of files -n print line # along with matching lines -vprint non matching lines-wsearch for given pattern as a string

13. Searching Files (CONT…) • $grep Ali studentsdisplays line in file students that contain the string Ali. • $grep -n Ali studentsdisplays the line number along with the matching lines for string Ali in the file students. • $grep -n include *.csearch the pwd for the string include in all file with extension .c • $grep ‘^[A-H]’ studentsdisplays the lines in the file students that start with the letter A through H. the ^ specifies the beginning of the line.

14. Searching Files (CONT…) • $egrep “^J” studentsdisplays the lines in the student file that start with letter J, the ^ indicate the beginning of the line. • $egrep “^J|^K” studentsdisplays the lines in the file students that start with letters J or K. • $grep ‘\<Ke’ studentsdisplays the lines that contains words starting with the string Ke. Not \< is used to indicate start of a word. • $egrep -v “Ali|Nabeel” studentsdisplays the lines that do not contain Ali or Nabeel in the file students.

15. Cutting and Pasting • cut –blist [-n][file list]cut –clist [file list]cut –flist [-dchar][-s][file list]cut out fields of a table in a file. • Options:-b listTreat each byte as a column and cut bytes specified in the list.-c listTreat each char as a column and cut char specified in the list.-d charUse the char instead of the <Tab> char as field separator. -f listCut fields specified in the list.-n Do not split chars (used with -b option)-s Do not output lines that do not have the delimiter character.

16. Cutting and Pasting (cont…) • $cut -f1,2 studentsspecify the first and second field of the student file. • $cut -f1,2,5 studentspecify the first, second and fifth field of the student file. • $cut -f1-3 studentsspecify fields one through three from the file name students. • $cut -d: -f5,1,6 /etc/passwdthe -d is used to specify : as the field separator, then display or cut field 5,1, and 6 from the directory /etc/passwd.

17. Cutting and Pasting (cont…) • Paste[options] file listHorizontally concatenate files in file list. • Options:-d list Use list chars as line separators. • $paste students student_recorddisplay on the screen both file concatenated horizontally. • $cut -f1-3 students > table1$cut -f4 student_record > table2 $paste table1 table3$rm table1 table 2display the first three field of students and the seventh field of file student_record.

18. File Encryption and Decryption • Crypt [options]Encrypt (decrypt) standard input file and send it to standard output file. • Options:key password to be used to perform encryption and decryption.-k use the value of the environment variable CRYPTKEY.