How to Get Started Using XSEDE

1. How to Get Started Using XSEDE Phil Blood, PSC Marcela Madrid, PSC

2. OUTLINE • What is XSEDE? • How do I get started? • XSEDE User Portal • User Responsibility & Security • Applying for an Allocation • Accessing Resources • Managing Data and File Transfers • Other Resources • Running Jobs • Managing Your Software Environment • Getting Help • Next Steps

3. Extreme Science and Engineering Discovery Environment (XSEDE): most powerful integrated advanced digital resources and services in the world. Includes computing, visualization, storage resources and advanced support services. Funded by NSF. • A single virtual system that scientists can use to interactively share computing resources, data, and expertise. • XSEDE lowers technological barriers to the access and use of computing resources. Using XSEDE, researchers can establish private, secure environments that have all the resources, services, and collaboration support they need to be productive. XSEDE partnership: 16 supercomputers across the country (Service Providers): Univ. of Illinois at Urbana-Champaign, Carnegie Mellon Univ./Univ. of Pittsburgh, Univ. of Texas at Austin, Univ. of Tennessee Knoxville, Univ. of Virginia, Shodor Education Foundation, Southeastern Universities Research Association, Univ. of Chicago, Univ. of California San Diego, Indiana Univ., Purdue Univ., Cornell Univ., Ohio State Univ. Univ. of California Berkeley, Rice Univ., and the National Center for Atmospheric Research. Led by the Univ. of Illinois's National Center for Supercomputing.

4. How do I get started using XSEDE? To get started using XSEDE a researcher needs to: • apply for an XSEDE allocation or • request to be added to an existing one. You do either of these through the XSEDE User Portal.

5. XSEDE User Portal • Web-based single point of contact that provides: • Continually updated information about your accounts. • Access to your XSEDE accounts and allocated resources: The Portal provides a single location from which to access XSEDE resources. One can access all accounts on various machines from the Portal. • Interfaces for data management, data collections, and other user tasks and resources • Access to the Help Desk.

6. XSEDE HOME PAGE: xsede.org Can create a Portal account at any time

7. portal.xsede.orgUser Responsibilities & Security The first time you login to the Portal, at the beginning of each allocation term, you will be asked to accept the User Responsibilities form: • Explains acceptable use to protect shared resources and intellectual property. Acknowledgment in publications, etc. • You are responsible for your account: Do not share accounts • User is responsible for protecting the passwords: • Includes not sharing passwords, not writing passwords down where they can be easily found, and not using tools which expose passwords on the network • This includes private keys: make sure they are password-protected. • Appropriate Behavior • Protecting computing, closing SSH terminals when done, logging out of the User Portal when done, etc. • Report Suspicious Activity. • If you have any suspicion that your account or personal computer has been compromised send email to help@xsede.org or call 24/7: 1-866-907-2383 immediately.

8. portal.xsede.org

9. XSEDE PORTAL TABS: Go to Allocations to apply for an allocation. The PI will receive email when the allocation is approved.

10. My XSEDE: Allocations/Usage When the allocation is active it will show in the Portal, MyXSEDE, Allocations/Usage tab:

11. portal.xsede.orgMY XSEDEProfile • Profile • View and or change your user information (organization, address). • Important because the email that is here is where XSEDE will communicate with you regarding your allocation.

12. Once your allocation is approved: The principal investigator (PI), co-PI, or an Allocations Manager can add users to an existing allocation via the XSEDE User Portal: My XSEDE->Add/Remove User Takes the portal name of the user you want to add/remove. Can use Single Sign-On (SSO) via the XSEDE User Portal to log into all resources on which you have an account. For help, email  help@xsede.org A few Exceptions: Accounts at certain Service Providers need to be activated before they can be accessed.

13. Accessing XSEDE Resources • Single Sign-On (SSO) is the default authentication method. With SSO, you need only one username and password (your User Portal one) to log into every digital service on which you have an account. • The easiest way to use SSO is via the XSEDE User Portal, but you can also use SSO via a desktop client or with an X.509 certificate. • Stand-alone client: http://grid.ncsa.uiuc.edu/gsi-sshterm/ • After you authenticate using SSO with your User Portal username and password, you will be recognized by all XSEDE services on which you have account, without having to enter your login information again for each resource. • If you wish to access resources individually via ‘ssh’, submit a ticket requesting site-specific access.

14. SSO via XSEDE User Portal • Make sure you are logged into the XSEDE User Portal • Go to ‘My XSEDE’tab • Go to the ‘Accounts’link • Resources you have access to will be indicated by a ‘login’ link • Click on the ‘login’ link of the resource you would like to login to

15. SSO via XSEDE User Portal • Clicking on login • brings up a Java Applet • Asks “Do you want to run the • application”? Run the applet, then wait a few minutes for the applet to load.

16. My XSEDE  AccountsSSO via XSEDE User Portal • Initially a blank terminal window will appear in your web browser. • The window will fill with text indicating that you have been successfully logged into the resource of your choice. • Can now connect to other machines from this terminal, using the command • gsissh machine-name

17. SSO from an XSEDE Resource If you have an account on a specific machine: Use the Unix command SSH on a terminal window to connect to the resource that you've chosen. In the SSH window that appears: login following the instructions for that resource from your account information packet. (Using the specific username and password for that machine).

18. SSO from an XSEDE Resource, continued • Since we did not initiate the SSO from the XSEDE User Portal, the certificate proxy was not generated automatically, so you must generate it yourself using the command: • myproxy-logon -l <username> • Where username is your Portal username. • You will get a prompt to enter your MyProxy passphrase. Use your current XSEDE Portal username and password for this step. • You can type ‘man myproxy-logon’ to learn more about options to this command. • Run grid-proxy-info to verify a proxy has been established. This will also tell you how long your proxy will last. Now you can access other resources without a password using gsissh • (If you encounter problems, they may be related to establishing your environment. We will cover this shortly.)

19. Example: SSO from XSEDE resource command line tg-login10 8/joeuser> grid-proxy-info ERROR: Couldn't find a valid proxy. Use -debug for further information. tg-login10 8/joeuser> myproxy-logon -l joeuser Enter MyProxy pass phrase: A credential has been received for user joeuser in /tmp/x509up_u18228. tg-login10 8/joeuser> grid-proxy-info subject : /C=US/O=National Center for Supercomputing Applications/CN=Joe User issuer : /C=US/O=National Center for Supercomputing Applications/OU=Certificate Authorities/CN=MyProxy identity : /C=US/O=National Center for Supercomputing Applications/CN=Joe User type : end entity credential strength : 1024 bits path : /tmp/x509up_u18228 timeleft : 11:59:42 (Important information) man myproxy-logon gives information on how to extend this time. tg-login10 8/joeuser> gsissh tg-login.lonestar.tacc.XSEDE.org Last login: Wed Jul 9 15:47:04 2008 from machine.psc.edu ------------------------------------------------------------------------------ Welcome to the Lonestar x86_64 Linux Cluster (Dual-Core) Texas Advanced Computing Center, The University of Texas at Austin

20. portal.XSEDE.org ResourcesFile ManagerFile Transfers: Small (< 1 GB) Files • To transfer small files between XSEDE Resources and/or your own workstation you can: • scp or sftp from the Unix command line. Easy to use. Provides poor performance for very large files.

21. XSEDE User Portal File Manager • Log into the XSEDE User Portal • Select “Resources” tab • Select “File Manager” tab • (now wait for Java Applet to load)

22. portal.xsede.orgResources  File Manager Click on the X (XSEDE logo) to list all machines. This includes: • your local machine. • XSEDE$Share: 2GB of space to collaborate. Allows you to share files with your collaborators. • Click on the ‘View’ tab and select “Show Transfers” to get information on past and ongoing file transfers and speed. • Drag files between resources.

23. portal.xsede.org  Resources  File ManagerLarge (> 1 GB) File Transfers • On the left panel of the file browser, click the XSEDE logo to display your list of available resources. • Before clicking on the resource, Right click on the resource you’re going to transfer data from and select Edit. This will bring up the file transfer parameters: • Click on the “Optimize” button • Click the checkbox next to “Stripe Transfers” • Click OK • Repeat for the other panel using the destination resource • View  show transfer • Repeat this every time you change Resources • Drag and drop the file from source to destination to transfer.

24. Large (>1 GB) File Transfers within the TeraGrid:Command line method • Uberftp or globus-url-copy: uses the GridFTP protocol (GridFTP is not a command) to transfer at rates ~100 MB/sec • Example (PSC Blacklight -> TACC Ranger): Optimized for large files: All on a single line: • globus-url-copy -stripe -tcp-bs 8388608 gsiftp:// gsiftp://gridftp.blacklight.psc.teragrid.org /scratcha/joeuser/file gsiftp://gridftp.ranger.tacc.teragrid.org/scratch/joeuser • Look here for names of gridftp servers at each site: https://www.teragrid.org/web/user-support/transfer_location • speedpage.psc.edu: provides advance information on the file transfer rate you can expect using globus-url-copy with the optimized parameters above.

25. https://mobile.xsede.org • Allows browsing of all XSEDE systems, file downloading, and third-party transfers. • It provides several features for mobile users such as one touch file publishing to the user's public folder, simple creation of shared groups for any file/folder, and one click permission management of all XSEDE systems, file downloading, and third-party transfers.

26. Portal.xsede.org->Resources-> Resource Monitor View systeminformation: TFLOPS, memory, today’s load, jobs running in queue. Status: up or down: takes you to the news announcements that tells you when the machine is expected to come back up.

27. Portal.xsede.org->Resources • Remote Visualization: Offers interactive visualization capabilities for users needing to analyze very large data sets • Science Gateways enable entire communities of users associated with a common scientific goal to use national resources through a common interface.

28. portal.xsede.org  Resources searchable, and available to a wide audience, either a collaborative group or the scientific public in general Data Collections: Organized, searchable, permanent data storage. Available to a collaborative group or the scientific public in general.

29. Portal.xsede.org->User Forums

30. Running Jobs on XSEDE Resources To learn about the specifics of how to run on a particular supercomputer, look at the User Guide. • File Systems • Batch jobs

31. File Systems on XSEDE Resources Where your data resides on XSEDE and the appropriate storage is your responsibility. In general, all resources provide: • HOME: Permanent space, but small. A good choice for building software and working file collections of small to medium sized files, where a medium sized file is less than 50 MB. • SCRATCH: More space, but TEMPORARY. Recommended for running jobs. Backup your files left here! They get deleted (purged). • Archival (mass) storage: Long-term storage of large amounts of data, is accessible from all sites, but offers slower access time. Tar files before storing.

32. Create a script • Example script for running an MPI job on Blacklight at PSC. • Actual commands are site and machine specific, but they follow general principles. • Needs to be modified to run on other XSEDE machines.

33. Batch jobs • Submit the script that you have created: • Batch system should be used to run your job. • Actual commands are machine specific, but they follow general principles. • qsub jobname • qstat –a • qstat -u username • qdel jobid • man qsub

34. Managing Your Environment: Modules • Allows you to manipulate your environment. • ‘module list’ shows currently loaded modules. • ‘module avail’ shows available modules. • ‘module show’ <name> describes module. http://modules.sourceforge.net/ % module load gcc/3.1.1 % which gcc /usr/local/gcc/3.1.1/linux/bin/gcc % module switch gcc/3.1.1 gcc/3.2.0 % which gcc /usr/local/gcc/3.2.0/linux/bin/gcc % module unload gcc % which gcc gcc not found

35. Getting Help • For continually updated information on your account, allocations, and more, login to the XSEDE User Portal. • portal.xsede.org  Help • portal.xsede.org  My XSEDE  Tickets • portal.xsede.org  Documentation  Knowledge Base Provides answers to commonly asked questions. • xsede.org  Resources: User support documentation, user guides. • portal.xsede.org  User Forums: discuss your questions with other users and XSEDE staff. • portal.xsede.org  Training: Seminars or in-person training sessions, enroll on line. • Get face-to-face help from XSEDE experts at your institution; contact your local Campus Champions. • Advanced User Support

36. Reporting and Tracking Issues • portal.xsede.org  Help • Help Desk: Submit ticket • Security Incident, for ex. your account has been compromised. • portal.xsede.org  My XSEDE  Tickets • Submit ticket • View past tickets (both open and closed) • Can also email help@xsede.org or call 1-866-907-2383, at any hour (24/7)

37. Preferred method of submitting ticketHelps ensure all relevant details are provided for quick routing

38. More “helpful” resources xsede.orgUser Services • Resourcesavailable at each Service Provider • User Guides describing memory, number of CPUs, file systems, etc. • Storage facilities • Software (Comprehensive Search) • Training • How to get help • Advanced User Support (AUSS)

39. Advanced User Support Services (AUSS) • Provides collaboration between XSEDE staff and users. The objective of the program is to enhance effectiveness and productivity. • Advanced Support for Research Teams (ASRT): In-depth staff support. Staff from the XSEDE will join the Principal Investigator’s team to collaborate for up to one year on the users’ applications. • Advanced Support for Community Capabilities (ASCC) • Advanced Support for Science Gateways (AS-SGW)

40. AUSS support can include: • Porting applications to new resources • Providing help for portal and gateway development • Implementing algorithmic enhancements • Implementing parallel math libraries • Improving the scalability of codes to higher processor counts • Optimizing codes to efficiently utilize specific resources • Assisting with visualization, workflow, and data analysis/transfer • .

41. Advanced User Support (AUSS)  Expertise is available in a wide range of areas: performance analysis, petascale optimization, gateway and web portal experience, specialized scientific software. Can solicit AUSS at any time through the Allocations tab at the XSEDE User Portal Requires written justification Inquire at help@xsede.org

42. Questions? Need Help? • Once more, we are here to help you… • First, try searching the Knowledge Base or other Documentation • If that doesn’t help, submit a ticket: Use the XSEDE User Portal ‘My XSEDE Tickets’ tab • or • Send an email to help@xsede.org • Can also call XSEDE Help Desk 24/7: • 1-866-907-2383

43. Need more training? • portal.xsede.org  Training • Course Calendar • On-line training