Perspectives from the Field Dissemination and Quality Assurance

1. Perspectives from the FieldDissemination and Quality Assurance David Joiner CSERD / Kean University

2. Quality assurance for modeling and simulation • Verification • provides assurance that the resource works as advertised on the computing platforms This helps to answer the question, does this resource "solve the problem right". • Correct derivation of solutions • Bug-free programs • Validation • provides assurance that the resource is based on current, valid scientific methods. This helps to answer the question, does this resource "solve the right problem". • Comparison of results to data • Standard cases • Accreditation • provides assurance that the resource is appropriate for the advertised audience. This helps to answer the question, does the resource "match the learner's skill level". • Connection to computational competencies, standard curriculum

3. Use of VV&A in K-16 education efforts • National Science Digital Library • CSERD • ComPadre (Physics, recent effort to increase computational physics holdings) • BEN (Biology, no specific effort to single out computational activities) • ChemED DL (Chemistry, no specific effort to single out computational activities)

4. Use of VV&A in K-16 education efforts • Computational Science Education Reference Desk • Historically focused on undergraduate education, increasingly covering materials at both K-12 and professional levels • Breadth of the catalog [multiple disciplines, both (Computational Science) Education and Computational (Science Education), grade levels from K-professional] requires accurate metadata tagging for searching/browsing/filtering of materials • VV&A process in place to review materials, in use for some time in an unsolicited mode, recent addition of solicited reviews. • Persistence of CSERD • Maintained by the Shodor Education Foundation, Inc. • Originally funded by EOT-PACI, then by NSDL • NSDL is currently reorganizing, but Shodor is committed to long term maintenance of CSERD regardless of NSDL funding.

5. CSERD’s VV&A Effort • Reviewer Registration and Authentication • Potential reviewers provide areas of expertise, authenticated by site editors • Unsolicited Reviews • Verification and Accreditation allow for a “guided” review • All allow free-form reviews • Solicited Reviews • Content experts, higher level reviews • Summary Reviews • Content editors create summary reviews for materials after full VV&A review is complete • Stored in NSDL master repository • In progress efforts to create standardized review metadata across NSDL projects

6. Recommendations from HPCU RAT • VV&A • The TeraGrid should submit all materials to CSERD for both broad dissemination as well as subject all of the resources to a formal VV&A review process. • TeraGrid should identify at least one person to be the primary editor to lead the charge of finding reviewers, organizing reviews, and publishing the resources that have passed a minimum quality level. • TeraGrid should encourage the community to contribute additional materials, to offer community comments on the resources, and to provide feedback on the resources to TeraGrid on a regular basis. • User Surveys • A common user training survey used by all RPs should be provided to all users after participating in a training session, or using training resources.

7. VV&A Breakout Discussion • Are there existing databases of materials to be mined? • How do we get people to want to do this? • Credit • $$$ • What organizations would want to take credit for doing this? • How is VV&A for HPC University resources different from VV&A for other introductory undergraduate CSE materials? • Single PC testing grid – windows, Mac, Linux • Many PC testing grid • Compilers? • Parallelization method? • Numerical libraries? • Architecture? • Public access to resources • Is the source code available? • Are commercial tools required?

8. VV&A Action Items