Application review module (ARM)

1. Application review module (ARM) Division of Nursing and Public Health Reviewers Manual

2. About ARM Once you have read and considered each application, ARM is the web-based computer system you will use to record the results of your evaluations. It provides a user-friendly environment to facilitate the HRSA/DNPH application review process while allowing reviewers to participate from remote locations. About ARM • The users of ARM that are discussed in this material are the Reviewer and the Review Administrator (RA).

3. Log-in under the “Reviewer Logon” tab at the top. Click in or Tab to User Name and enter your username and then to Password and enter your password into the appropriate boxes. Then click the button. Note: Usernames and Passwords are case sensitive. Accessing ARM Accessing ARM You will receive an email prior to your review containing your Username and Password as well as the link to the ARM website. • Each review in ARM has a unique Session ID. You may be a reviewer on more than one active review. Your Username and Password will provide you access to any active review to which you have been assigned.

4. Accessing Your Review Starting the Review Find the Session ID or Session Name that matches the review you will be working on and click on the hyperlinked Session Name. Click on the hyperlinked panel number to select your panel.

5. Application Selection Screen Click on the hyperlinked Application Name to advance to the Application Worksheet Screen and begin your review. Starting the Review This screen displays four major identifiers of the application. • Application ID • Status • Score • Application Name • If there are no applications listed, please contact your HRSA/DNPH review administrator. Note: In the score column, if the score shown is -- you have not entered any scores for this application. If an “*” is present next to the score, you have not entered all of the individual scores required for this application.

6. View Application PDF File View Application PDF File In situations where the application PDF file feature is turned on, the Reviewers will be able to view the application PDF file through the ARM system. Click on the Open PDF link to open the application PDF file. The file will open in a new window. You can also save this file for viewing at a later time. *Note: In most cases, the link will read Open PDF. This means the file has been retrieved and is ready to be opened. If the link reads Retrieve PDF, the file has not been transferred to the ARM system. Simply click on the Retrieve PDFlink to transfer the file into the ARM system. Once the PDF file has been brought over to the ARM system, the link will read Open PDF. 6

7. Application Evaluation Screen Application Worksheet Screen This screen serves as the primary control panel for evaluating that particular application. All applications will be scored by individual criterion. • The scoring criteria are listed in the first column. • The middle column is where you will click to enter your scores. The score will automatically total as you move through the criteria. • The last column is where you will be able to click and enter comments after you have entered scores. )

8. In the Score column, click on the hyperlinked -- to begin scoring the application. Using the drop-down arrow, select the appropriate score by clicking on the number. Click . Note: Reviewer will view criteria pertinent to the Nursing Scholarship Program. Criteria will be similar to the screenshot on page 7. Application Evaluation Scoring Application Scoring First, users must enter a scoreBEFOREany comments can be entered. Note: You will not be able to select a score greater than the maximum defined by the criterion.

9. Application Evaluation Scoring Note: Reviewer will view criteria pertinent to the Nursing Scholarship Program. Criteria will be similar to the screenshot on page 7. Entering Comments ONLY AFTER YOU ASSIGN A SCORE to an individual criterion can you include corresponding comments. • Again, as seen in the image, the only activated Comment links are those with a corresponding score. The link will automatically appear once you have scored the criterion. Note: If you have not entered all required scores for an application, your Actual Score will be followed by an * indicating that your scoring is not complete.

10. Enter your strength or weakness. You can type directly into this field or copy and paste from a word processing program. After entering the comment, it is strongly recommended that you Spell Check your work by clicking on the Spell Check button. Click to save your comment as part of your evaluation. Application Evaluation Comments Entering Comments Clicking the Comment link will open the Comments window. Click on the drop-down arrow to open the menu. Select whether you are describing a Strength or Weakness. Enter a page number or any page identifying information. Note: All fields in this screen are required.

11. Application Evaluation Comments Entering Multiple Comments You can enter multiple strengths or weaknesses for each criterion. • To enter multiple comments, repeat steps 9-12. • All previous comments will be visible toward the bottom of the page, below the “Add a New Comment” box. • Regardless of the order in which they were entered, comments will group by category (Strength or Weakness).

12. To edit your comments, click on the word Edit next to the comment number you would like to change. • Clicking on the Edit link will open the comment window and you will be able to edit your content. When complete click to save your changes. Application Evaluation Comments Editing Comments You can edit your saved comments at any time. Deleting Comments • Click Delete to permanently remove a comment from your evaluation.

13. Save Your Work • To save your comments as you work, you can periodically click the button. This button will save your comments, and you can easily edit them to make additions or changes. Session Time-Outs and Preventing Data Loss Session Time Out • After 30 minutes, if you are not clicking between ARM web pages, you will be logged out and required to re-enter your username and password. Any comments that have not been saved will be lost. Note: The comment window has a countdown timer (circled above). This timer will let you know how much time remains in your session and will read “Timeout” when time expires.

14. Application Evaluation Scoring Completed Comments Once you have finished entering all of your comments for a given criterion, you will need to return to the Application Worksheet Screen. Click on the hyperlinked word Evaluation in the bar at the top of your screen to return to the Application Worksheet Screen after you have included all comments for a given criterion.

15. Application Evaluation Scoring Note: Reviewer will view criteria pertinent to the Nursing Scholarship Program. Criteria will be similar to the screenshot on page 7. Application Worksheet Screen After returning to this screen continue entering comments for the remaining criteria as needed. • Repeat steps 8-12 to add comments for each criterion

16. Application Evaluation Scoring Note: Reviewer will view criteria pertinent to the Nursing Scholarship Program. Criteria will be similar to the screenshot on page 7. Entering Multiple Scores At Once • If you would like to enter all of your scores at one time, click on the hyperlinked column header Score. • The full Add Scores screen will open, and you will be able to click on the drop-down arrow for each criterion and assign a score. • When you have entered all of the scores, click .

17. To view your scores by criterion, you can click the at any time. The view will open in a new window from which you can also print the view as a report. • To view all of your strengths and weaknesses for each criterion, click . Click to view your entries for all other sections. Each of these views open in a new window and show information for that individual application, and each view can be printed as a report. Scoring Reports Note: Reviewer will view criteria pertinent to the Nursing Scholarship Program. Criteria will be similar to the screenshot on page 7. Reviewing your Work You can review your work at any time in the process.

18. Note: Reviewer will view criteria pertinent to the Nursing Scholarship Program. Criteria will be similar to the screenshot on page 7. Submit Review Submitting Your Review When you have completed scoring the application and are satisfied with your comments, you will need to submit your evaluation to your RA. Click the button to submit your application evaluation to your RA. Click OK in the confirmation box to finalize your submission. • The button will not be activated until all sections are completed. • To evaluate other assigned applications, click “Applications List” located on the blue tool bar at the top of the page. • When all applications have been evaluated and submitted to the chair, log-off of the system. Note: The Submit to Chairperson button will not be activated until ALL scores have been entered. Note: Once you submit your application evaluation to your RA, further edits CANNOT be made.

19. You can only see your own scores and comments. You must enter a score for each criterion BEFORE proceeding to enter strengths and weaknesses. The Comments link will be automatically activated after you enter a value in the score field. The button does not become active until the your review of an application is complete. You cannot submit your review to the RA until you have scored all criteria and comments. You cannot change or modify your scores, strengths and weaknesses once an application has been submitted to the RA. You can, however, still view and print your own scores, comments and entries to other sections. All Fields marked with an asterisk * are mandatory. If the Back Arrow or the blue Navigation Bar are available, use one or the other to navigate within ARM. Only close a window when neither the Back Arrow or Navigation Bar are available. Exit ARM by clicking on Logon on/off in the upper right corner or your screen. Scoring and Documents Rules Application Worksheet Rules

20. When an application has been assigned and is available to you to enter criteria scores, strengths, and weaknesses that application’s status is In Review. • When you finish entering all your scores, strengths and weaknesses, you can click the button and that application’s status changes to Submitted to Chair. • (Web-Assisted Teleconference Only) When the panel has reached consensus and the RA has recorded all of the panels recommendations regarding strengths and weaknesses, the RA will return control to all reviewers on the panel so that they may enter or modify their scores for each criteria. That application’s status is then changed to Returned by RA. Once each reviewer has entered, changed, or decided to leave his or her scores unchanged, they will again click the button, and that application’s status once again becomes Submitted to Chair. Application Status Definitions

21. Returned Applications Entering or Modifying Your Final Scores After your panel finishes their discussion of a score discrepancy application, your RA will return the application to you so you can enter or modify your scores. • You will know if an application has been returned to you because the status on your Application Selection Screen will read “Returned by Chair”. To begin the process of modifying or entering your score for an application, click on the Application Name, as you did in Step 3, to open the Application Worksheet Screen.

22. To change or enter a criterion score, click the applicable Scorelink. When you have finished updating or entering , click to send your evaluation back to the RA. Note: Reviewer will view criteria pertinent to the Nursing Scholarship Program. Criteria will be similar to the screenshot on page 7. Revise Application Evaluation Revising Your Evaluation At the direction of your Chairperson and based on your panel discussions, you will enter or modify your scores. Note: Once you re-submit, you will no longer have access to the Return History screen from this screen.

23. Please follow the instructions in this manual to process all the applications on your panel. If you have technical questions, please contact the HRSA ARM Technical Assistance Team at ArmTechAssistance@hrsa.gov If you have NSP questions, please email BCRSReview@hrsa.gov or call: Arthur Williams, Senior Public Health Analyst 301-443-9182 Dustin Price, Public Health Analyst 301-443-4874 Kamu Aysola, Management Analyst 301-443-5217 End