Introduction to *nix Systems

1. Introduction to *nix Systems HKOI Training Team Sidney Fong

2. Overview of this presentation • What to expect • Brief introduction to UNIX-like systems • Brief history of Linux • Introduction to basic Linux commands and environment • Linux as competition environment • What not to expect • Detailed instructions on how to install Linux • Setting up Linux as a server

3. What is *nix? • “*nix” refers to a class of UNIX-like operating systems • The term “UNIX” is sometimes used to refer to *nix systems • Examples of *nix systems: • Linux (or GNU/Linux) • BSD (FreeBSD, NetBSD, OpenBSD, etc) • Solaris • Mac OS X (Darwin) • GNU Hurd

4. Typical Features of *nix Systems • Geared towards being a “server” / “networking” OS • Multi-user • Multi-tasking • Tends to focus more on command line interface • The Unix Philosophy • Write programs that do one thing and do it well • Write programs to work together • Write programs to handle text streams, because that is a universal interface

5. Linux - Overview For the rest of the session, we will focus on Linux

6. Linus Torvalds Creator of Linux Kernel Richard M. Stallman (RMS) Founder of GNU and the Free Software Movement Important Figures

7. Names • Pronunciation of “Linux” • “lee-nix” ( /linəks/ ) • Pronunciation of “GNU” • “guh-noo” ( /gnu/ ) • GNU • recursive acronym for “GNU’s not UNIX” • “Linux” vs “GNU/Linux” debate

8. The GNU Project • Started by RMS to create a complete UNIX-like system in 1983 • Kernel, system tools, compilers, documentation, editors, etc. • Ideology of Free Software Movement • Free: “Free” as in Freedom • “Copyleft” Software: GNU General Public License (GPL) • Related: Open Source Movement • Most GNU tools were ready by the early 90’s. • … Except the kernel: “GNU Hurd” • Hurd was supposed to be a microkernel • Over-design? Wrong approach?

9. BSD • At the same time, another open source UNIX distribution, the BSD, was being sued by UNIX System Labs. (USL v. BSDi) • The lawsuit cast doubt upon the legal status of the BSD’s. The case was finally settled in 1994, with both parties agreeing not to litigate further. • This, together with the lack of a kernel in the GNU project, created a demand for a free UNIX-like kernel, which was fulfilled by Linux

10. Linux • Started by Linus Torvalds as a hobby project • Inspired by Minix, an OS used as an example in teaching principles of operating systems • First release of Linux in September 1991 • Free of the problems of GNU Hurd and BSD • Eventually joined by other developers • More: • Just for Fun: The Story of an Accidental Revolutionary

11. A Typical Linux System • Linux Kernel • User space programs (many are GNU software) • Shell (command line): bash • Utilities: ls, cp, mv, rm, cat, grep, ... • Compiler: gcc • … and more … • X Window System (for desktops/workstations) • Various daemons (a.k.a. services) • Eg.: syslogd, httpd, cron, inetd

12. A Rough Graphical Representation of a Linux System Hardware Kernel User space programs Daemons X Server Shell X Client

13. Linux “Distributions” • Since most software on Linux can be freely modified and re-distributed, people began packaging these software into “distributions” • A “distribution” is responsible for most of the software installed on the Linux system, including the base “operating” system and any other applications the user installs • rpm, deb/apt, tgz • Each distribution has it’s own “character”, differing in • Purpose : Server, Workstation, Firewall, etc… • Installation / upgrade method • Choice of software • Way of packaging software (tgz, rpm, deb) • Default configuration • Aims and Ideology – Commercial, Promote Free Software, etc

14. Linux Distributions – Redhat / Fedora • The “standard” Linux • Has a relatively high level of support from third party vendors • Geared towards commercial / corporate environments • Initial set up is generally easy • Rather expensive if you buy the enterprise edition (hundreds of USD!) • Maintenance is a nightmare! ;-p www.redhat.com

15. Mandriva (was “Mandrake”) • Originally based on Redhat, therefore somewhat similar to Redhat • Geared towards desktop and newbies www.mandrivalinux.com

16. Slackware • Stable, simplistic • One man distribution • Zipslack, which works without having separate Linux partition(s) • Relatively UNIX-like compared to other distributions www.slackware.com

17. Gentoo • Source based installation! • The standard way of installing new software on the system is to compile it from source (using the “portage” system) • This is equivalent to the BSD ports system • Takes LoOOong time to install / upgrade www.gentoo.org

18. Debian • Made by a worldwide non-profit organization, with thousands of volunteers helping to maintain packages • Quality and quantity of packages are generally quite high • Slow release cycle – the last “stable” release was in July 2002!! www.debian.org

19. Linux - Basics An introduction to using Linux as a normal user

20. Post-Login Screen “Message of the day” Usually contains important notices Username Hostname Current directory Hint: You may press <Ctrl+Alt+F1> to <Ctrl+Alt+F6> for multiple terminals if you are using Linux on the local machine (instead of over network) Why is the username and hostname displayed in the command prompt?

21. Basic Commands • “ls” – List directory contents • “cd” – Change directory • “cp” – Copy file(s) • “rm” – Remove file(s) • “mv” – Move file(s) • “echo” – Display a line (or lines) of text • “mkdir” – Make (i.e. Create) Directory • “cat” – Concatenate files and print to stdout • “exit” / “logout” – Exit the shell

22. Example I • Try the following commands • ls # list directory • echo HKOI Cows # print a message on screen • ls -l # list directory (long format) • mkdir hello # create directory “hello” • cd hello # change directory to hello • cp /etc/motd . # copy file to current dir • ls -la # list directory • cat motd # display contents of file • rm /etc/motd # try to remove file (fails ;-p) • mv motd .. # move file to parent dir • cd .. # change to parent directory

23. Linux File System Hierarchy • In Linux, the file system is a (single) tree • Everything begins from the root directory: / • Non-exclusive listing of commonly used directories • /home – User home directories • /bin, /usr/bin – Binaries (executables, programs) • /lib, /usr/lib – Libraries (similar to “DLL”s in Windows) • /etc – System configuration files • /var, /usr/var – Stores files that are often modified (eg. log files, database files, mail spool, print spool) • /dev – special device files • /tmp – Temporary files • In general, a normal user cannot modify the files outside his home directory (except /tmp)

24. Exploring the system • Try to navigate your way around the system with the above commands • You may view the contents of a file with the “cat” command • You can return to your home directory with the “cd” command (no arguments) • Make good use of tab completion

25. Text Editors • Vi (VIM) • Visual Editor • VIM: Vi Improved. • Emacs • Editor Macros • Nickname: Esc-Meta-Alt-Control-Shift • Nano • “Notepad”-like • Easy to use but less powerful than Vi / Emacs

26. Text Editors (How to quit) • Some programs may launch text editors that you may not know how to use -- even quitting is difficult! • VIM • <Esc> :q! (Quit without saving) • Emacs • <Ctrl-X> <Ctrl-C> (Quit) • Although not required, it is strongly recommended that you learn one of the above editors since they can increase your productivity quite significantly

27. Man(ual) pages • Example: man cp Purpose of command Different usages Hint: Press ‘q’ to quit Detailed description of command and its options

28. *nix Quirks (compared to MSDOS) • Path separator is a slash “/” instead of backslash “\” • Files are case sensitive • “File extensions” are “meaningless” • The “.” character in a file has no special meaning unless it is in the beginning of a filename • “.file” – hidden file • “.” – current directory • “..” – parent directory • Executable files have an “executable bit” set, instead of a “.exe” extension • <Ctrl-D> is End of File (<Ctrl-Z> in DOS)

29. *nix Quirks II • There must be a space between the command and subsequent arguments • Eg. “cd /” works, “cd/” does not • Command line arguments start with “-” or “--” instead of “/” • DOS: dir /OD • Linux: ls -t • Unix text format vs MSDOS text format • A line in *NIX systems ends with a ‘\n’ (ASCII 10) • A line in MSDOS ends with ‘\r\n’ (ASCII 13, 10) • The relevant functions of the C library in DOS/Windows converts ‘\n’ to ‘\r\n’ automatically

30. *nix Quirks III • Unless the path the command is specified, the shell will only run commands located in the $PATH environment variable (why?) • Question: how to run a program on the current directory? • ./command • To see what $PATH contains, you may issue the command: echo $PATH • To edit the value of $PATH: PATH=(new value)

31. Task I • Write a program in Pascal that • Reads an integer n from standard input • Writes n random integers to standard output • Save it as numbers.pas • Compile the program • Execute the program • Keep a backup copy in ~/backup/numbers.pas

32. I/O Redirection, Pipes • “A > B” redirects standard output from program A to file B. This operation will overwrite original contents of B (if any) • “A >> B” redirects standard output from program A and appends it to the end of file B. • “A < B“ redirects standard input from file B to program A • “A | B” redirects standard output of program A to standard input of program B • Try the following commands: • echo 100 | ./numbers • echo 10 > ten.txt • ./numbers < ten.txt • echo 20 | ./numbers | sort –n Hint: Also works in MSDOS and cmd.exe

33. Task II • Download http://www.hkoi.org/~sydneyfong/public/openq.zip to home directory and unzip the contents (use wget / w3m) • Change working directory to ~/prog0 • Write a program to solve the HKOI “Open Question” (see HKOI2005 website) • (You may try to write a “wrong” one (i.e. “incorrect solution 4”) for checking answers) • Save the program as “program0.pas” • Run the program using inputs from the files input1.txt to input5.txt, redirect the output to output1.txt to output5.txt • Check your answers by using `diff`

34. More on bash shell • alias – substitute a string as a command • Eg. alias ls=‘ls --color’ • You may list current aliases by simply issuing the command “alias” without arguments • The script ~/.bash_profile is run during login • The script ~/.bashrc is run in non-login shells • You may set aliases, environment variables and run commands in these files to customize your shell • You may want to make the two files identical • Aliases that you may want to have by default: • alias cp=‘cp -i’ # asks for confirmation before overwriting a file • alias mv=‘mv -i’ # asks for confirmation before overwriting a file • alias rm=‘rm -i’ # asks for confirmation before deleting a file

35. There are different sections of man pages Section 1: commands Section 2: system calls Section 3: C library calls Section 4: special files (in /dev) Section 5: (config) file formats Section 6: “games” Section 7: Misc Section 8: System administration commands If there are conflicts in names, the section of lower value is displayed. For a specific section: man <section> <page> Note that you may find section 3 very useful as a reference to standard C functions (eg. printf, fopen, string functions, etc) More on man pages

36. Special files (in /dev) • /dev/zero – returns an endless stream of ‘\0’ characters • To create an file of size 1MB: • dd if=/dev/zero of=myfile.img bs=1k count=1024 • /dev/null • Always return EOF when read • cat /dev/null > abc – creates empty file of size zero • All data written to /dev/null is discarded • ls > /dev/null – mutes output of ls

37. Tips in competitions… Practical advice for competition environments

38. Useful Commands in Competitions • bc – Arbitrary precision calculator • sort – Sort lines of text files (note the “-n” option) • ps – list running processes on the machine • kill – Sends a signal to running programs (terminates it by default) • date – Print system date/time • time – reports execution time of a command • diff – compares files • head, tail – prints the first/last N lines of the input • wc – word count (also counts characters and lines) • more – pager (allows you to scroll the input for easy viewing) • less – better version of more • grep – print lines matching a pattern • find – search for files • factor – factorize numbers • tsort – perform topological sort • seq – print a sequence of numbers • file – determine file type (sometimes displays other useful information) • tar – Archive files (“glue” multiple files into one, and vice versa)

39. Compilers - fpc • In most cases, fpc <file> would do • Useful options • -Co (check for integer overflow) • -Cr (range check) • -O1, -O2, -O3… • In some systems the man page is outdated

40. Compilers – gcc/g++ • use -o <output> to specify output file, otherwise the default is “a.out” • Most gcc options apply to g++ • Normally you would do: • gcc -O2 -o program1 program1.c • g++ -O2 -o program1 program1.cc • Remember to compile your program using the flags given by the judge/competition. (In particular, the optimization flags sometimes causes trouble) • If you want, you may also run `make <prog>`