SPMA & SWRep: Basic exercises

1. SPMA & SWRep: Basic exercises HEPiX hands-on, NIKHEF 5/03 German Cancio http://cern.ch/wp4-install

2. Agenda • Exercise environment • Software Repository exercises • SPMA exercises

3. Exercise environment (I) The demo setup has been simplified compared to the real production environment. • 1 server node hosting a single SWRep (lap-163.nikhef.nl) • N client nodes (one per alumni pair) • Only base SPMA/SWRep functionality is showed in the exercises. • The rest is left as exercise to the students ;-) • SWRep to CDB communication (for inventory) simulated with NFS mounts from lap-163.nikhef.nl to each client node • Local configuration cache (CCM) has to be updated manually • Wrapper script provided

4. Mapping between usernames and student nodes door

5. Exercise environment (II) • Preparing for the exercises: • Please log in to your client node (see username/password on blackboard), and open (at least) two windows: 1. A root window (needed for SPMA execution) 2. A normal user window (spmuserX) (needed for ‘pan’ compilation and SWRep manipulation) • Hands-out include: • This slides • A printed copy of the SPMA and SWRep client man pages • Solutions: • http://lap-163.nikhef.nl/Solutions.txt

6. Exercise environment (III) • Checking the home directory contents: • ~spmuserX/bin: • pan_wrapper.sh: wrapper for invoking pan with appropriate includes • spma_wrapper.sh: wrapper for invoking CCM, SPMA config generator and SPMA • pan.rh7: ‘pan’ binary • ~spmuserX/RPMS: • RPMS/: contains RPM demo packages, used for the exercises • SPECS/: the corresponding .spec files (not used in this demo) • ~spmuser/templates: • *.tpl: the ‘pan’ template files for the SPMA configs • xml/: directory where generated XML profiles will be stored

7. Software Repository Exercises • Quick introduction to edg-swrep-client • Adding and removing packages to the SWRep • Querying the SWRep for information • Area (ACL) management: adding, removing, changing areas for packages

8. SWRep client interface • Edg-swrep-client is the SWRep client interface • /opt/edg/bin/edg-swrep-client • Run edg-swrep-client --help for setup help • Run edg-swrep-client –-list-commands for the list of commands • Most important commands: • Package upload and deletion: • put • pull • remove • Repository query: • query • find • listplatforms • “Area” (==ACL) query and manipulation: • chtag • addarea • rmarea

9. Setup verifications • SWRep to client interface • /opt/edg/bin/edg-swrep-client listrights should produce something like: Areas: The SWRep ACL mechanism • Every student has a ‘private’ area (/hepix/userX), and there is a shared area (/hepix/common) • Note that each area determines who can modify which package, but the areas do not represent directories (rather hierarchical ACL tags) • Sub-areas can be created (ACL rights are cumulative) $ /home/spmuserX > /opt/edg/bin/edg-swrep-client listrights You are spmuserX, with rights to change packages with tags: /hepix/common /hepix/userX You have rights to remove packages

10. SWRep exercise 1 Exercise 1: Add / remove packages to/from Software Repository i. Add the package ~spmuserX/RPMS/RPMS/spma-test4-userX-0-1.i386.rpm to the SWRep, to your private “area”, for the platform i386_redhat73. • use the ‘put’ command (not: pull) of edg-swrep-client for package uploads ii. Remove that package from the SWRep. • use the ‘remove’ command iii. what happens if you try to add/delete a package which is not in your private ‘area’? Hints: • Use the ‘edg-swrep-client’ man page! • Check which is your private area with the ‘listrights’ command

11. SWRep exercise 2 Exercise 2: querying the SWRep for information i. list all platforms available on the SWRep. ii. list all packages belonging to the i386_redhat73 platform. iii. find all packages matching ‘kernel’ on the SWRep iv. get the package information for the ‘kernel’ and ‘kernel-smp’ packages. Hints: • Read the man page… ;-)

12. SWRep exercise 3 Exercise 3: Area management i. move one (pick one) of your packages to the /hepix/common area. How can you verify if somebody else can delete/modify this package? ii. Add a new area called ‘test’, which is a sub-area of your private area (/hepix/userX/test). Can you add further sub-areas to this sub-area? iii. Remove the area created in B). Does it also remove sub-areas of the created area? iv. Create a sub-area as in B), put some packages on it, and rename the whole sub-area to a different name. Hints: • Use the ‘chtag’, ‘addarea’, ‘rmarea’, and ‘charea’ commands of edg-swrep-client

13. SPMA exercises • Quick introduction to the PAN package templates • Adding/removing packages to the node • Local packages, and user priority switch • Handling simultaneous package versions with the SPMA

14. SPMA setup: package templates (I) • The Pan templates for the SPMA contain the list of packages to be installed by the SPMA. • pro_software_packages_hepix.tpl and • pro_software_packages_asis.tpl • By overlaying the templates using ‘include’, we get the desired list of packages in the XML. • see next slide • ‘pan’ is invoked using a wrapper to use the right directories and includes: • ~/bin/pan_wrapper.sh

15. External (generated by SWRep) ‘read-only’ structural templates editable templates Object templates (XML) Template structure for Software packages – HEPiX simplified setup Pro_software_ packages_hepix / asis software_ types functions includes Pro_default_hepix creates value repository_cern_cc_ I386_redhat73 Pro_software_hepix (external object template to speed up compilation, but no XML generated) Object templates (generate XML) dummynode

16. SPMA setup: package templates (II) Some functions are provided for manipulating the package list in the profile, which are used in the templates: • pkg_add(“packagename”,”version-release”,”arch”); • Adds a package to the profile. • pkg_del(“packagename”,[“version-release”]); • Removes a package version (or all if none specified) from the profile . • pkg_repl(“packagename”,”new ver-rel”,”arch”[,”old ver-rel”]); • Replaces the package version ‘old’ by ‘new’ in the profile. If ‘old’ is not specified, it replaces all other versions. • It is important to understand that the ‘add’,’del’ and ‘repl’ functions do only modify the final list of desired packages. • Eg. ‘pkg_del’ does not instruct to delete any package, but removes it from the list of desired packages • Useful when modifying inherited profiles • For the exercises, we will use only the ‘pkg_add’ function.

17. SPMA setup: command line interface • ensure SPMA runs in ‘light’ mode (not managing all packages on the node, but only a subset) • /opt/edg/bin/spma --help should include: • Most important options: • --noaction • --verbose • --userpkgs (yes/no) [run in ‘light’ or ‘full’ mode] • --userprio (yes/no) • In this tutorial, the SPMA is invoked using a wrapper (~/bin/spma_wrapper.sh) which passes the options trough. The wrapper is necessary for updating the CCM cache and generating the SPMA configuration file out of the ‘pan’ XML profile. --userpkgs <string> (default: 'yes') Allow user (== non-SPMA) installed packages

18. SPMA exercise 1 Exercise 1: Adding/removing packages to the node i.As spmuserX, edit the ‘templates/pro_software_packages_hepix.tpl’ template (with your favorite editor), and add to it the following package: • GNU.LANG_gcc-alt-2.95.2-6.i386.rpm ii. Save the template, and compile the XML profile with: ~/bin/pan_wrapper.sh ~/templates/dummynode.tpl iii. Now, use the SPMA wrapper to update the package list (downloading and installing the missing package), in dry run mode: (as root!)~/bin/spma_wrapper --noaction iv. And now run the SPMA: (as root!)~/bin/spma_wrapper

19. SPMA exercise 1. (contd) v. What happens if you re-run the SPMA again? vi. What happens if you remove the package by hand (rpm -e GNU.LANG_gcc-alt-2.95.2), and run the SPMA afterwards? vii. Now comment out the line added to the template in i.) and add another line for adding the following package: • JAVA_colt-1.0b3-0.i386.rpm and re-run the SPMA. What happens? viii. Which packages would be added if the template ‘pro_software_packages_asis.tpl’ is include’d into ‘pro_software_hepix.tpl’? Notes: • The generated SPMA local configuration file (containing the ‘target’ package list) is kept in /var/lib/spma-target.cf • The list of packages managed by the SPMA is stored in /var/lib/spma-managed-packages • SPMA log file: /var/log/spma.log Note: bug triggered during presentation fixed in edg-spma-1.0.4!

20. SPMA exercise 2. Exercise 2: Local packages and user priority switch i. Install (by hand) the following package: GNU.LANG_gcc-alt-3.2.3 (as root) rpm -Uvh http://lap-163.nikhef.nl/swrep/i386_redhat73/GNU.LANG_gcc-alt-3.2-3.i386.rpm This package will be considered as ‘local’ or ‘managed by the user’. What happens if the SPMA is told to use a different version? Edit ‘pro_software_packages_hepix.tpl’, and add a line for gcc 2.95: ”/software/packages”=pkg_add(“GNU.LANG_gcc-alt”,”2.95.2-6”,”i386”); Recompile the profile and re-run the SPMA ~/bin/pan_wrapper.sh, then as root: ~/bin/spma_wrapper.sh The package will be downgraded since the SPMA gives by default priority to the SPMA selected packages. But this can be changed…

21. SPMA exercise 2 (contd.) ii. Edit the SPMA config file /etc/spma.conf (as root) for enabling priority to user packages: uncomment the line # userprio = no and change it to userprio = yes Reinstall the GNU.LANG_gcc-alt-3.2.3 package (see previous slide) rpm -Uvh http://xxxx.nikhef.nl/swrep/i386_redhat73/GNU.LANG_gcc-alt-3.2-3.i386.rpm Recompile the profile and re-run the SPMA as root: ~/bin/spma_wrapper.sh This time, the user installed package will be kept, as the user version has priority over the SPMA selected version.

22. SPMA exercise 3 Exercise 3: multiple versions of packages How to install both gcc 3.2 and 2.95 simultaneously? The functions for package management allow to specify an optional parameter for this: ”/software/packages”=pkg_add(“GNU.LANG_gcc-alt”,”2.95.2-6”,”i386”,”1”); ”/software/packages”=pkg_add(“GNU.LANG_gcc-alt”,”3.2-3”,”i386”,”1”); The last parameter means: ‘do not remove other versions of the package in the profile’) (See the pro_declaration_functions.tpl file for more info) i. Add the above two lines to your pro_software_packages_hepix.tpl template, regenerate the profile, and run the SPMA. What happens? (check with rpm –qa the list of installed packages) Note: for sure, having multiple versions requires that the files of the versions do not conflict!

23. Other SPMA exercises Exercises to the students (studying the manual page): • Using the SPMA in full mode – no user packages (careful!) • Running in “update only” mode • Using the package local cache • Study the package flags (not available in this demo setup) • Using other protocols than HTTP (not available in this demo setup) Your feedback is most welcome for further developments!!!