SPI Software Process & Infrastructure

1. SPI Software Process & Infrastructure Quality Assurance (QA) http://spi.cern.ch/qa QA reports generation - 22 June 2005 Johanne BENARD johanne.benard@cern.ch

2. QA reports generation Test coverage reports • Basics • Some examples • Other utilities Savannah Quality Assurance (QA) reports • The web interface • Capabilities • Some examples SPI - Software Process & Infrastructure

3. Test coverage reports - Basics • Aim at: • Showing what percentage of an application has been executed by the test process, • Measuring effectiveness of a system test. • Two different principles in GCOV: Statement coverage analysis: • Code break down into basic blocs, • Block solicitations counting Branch coverage analysis: • Conditional possible paths • Path usage frequency SPI - Software Process & Infrastructure

4. Test coverage reports - Basics (*) LTP: Linux Test Project LCOV is an extension of GCOV, a GNU tool which provides information about what parts of a program are actually executed while running a particular test case. Code instrumentalisation: -fprofile-arcs -ftest-coverage LTP GCOV Suite (*): lcov, genhtml, etc. Compilation time Runtime Coverage analysis File.bb File.cxx, .c , .h, etc. File.da File.info File.bbg SPI - Software Process & Infrastructure

5. Some examples - POOL SPI - Software Process & Infrastructure

6. Some examples - POOL SPI - Software Process & Infrastructure

7. Some examples - CLHEP SPI - Software Process & Infrastructure

8. Some examples - PI SPI - Software Process & Infrastructure

9. Some examples - SEAL SPI - Software Process & Infrastructure

10. Some examples - ROOT SPI - Software Process & Infrastructure

11. Test Coverage - Other utilities Tools • Genhtml: • Creates an HTML view from coverage data found in INFOFILE • Geninfo: • Converts .da files into .info files • Genpgn: • Creates an overview image for a given source code file of either plain text or .gcov file format • Gendesc: • Converts a test case description file into a format as understood by genhtml INPUTFILE SPI - Software Process & Infrastructure

12. QA Savannah Reports SPI - Software Process & Infrastructure

13. QA Savannah report – Web Form Bugs, Tasks, Support and Patches Default: Project beginning Today .ini file: header, Style sheet and options Category, Assigned To Severity, etc. SPI - Software Process & Infrastructure

14. QA Savannah report – Web Form • Potential users: projects or experiments that actively use Savannah trackers (bugs, tasks, support, patch) • Current users: JRA1 coordination, JRA1 Middleware, LCG 2 Sites • Ready for LCG projects: • POOL, • PI, • SEAL • Tried on several other projects: • Athena, • Gaudi, • Cobra, • Iguana SPI - Software Process & Infrastructure

15. QA Savannah report - Capabilities Options used Global processing Breakdown Into Savannah fields SPI - Software Process & Infrastructure

16. QA Savannah report - Capabilities Statistics on a given period Translated into graph SPI - Software Process & Infrastructure

17. Some examples – LCG 2 Sites SPI - Software Process & Infrastructure

18. Some examples – JRA1 Middleware SPI - Software Process & Infrastructure

19. Some examples - POOL SPI - Software Process & Infrastructure

20. Some examples - SEAL SPI - Software Process & Infrastructure

21. Some example - PI SPI - Software Process & Infrastructure

22. SPI Quality Assurance SPI - Software Process & Infrastructure

23. Web site: http://spi.cern.ch/qa SPI - Software Process & Infrastructure

24. Conclusion • Proposed services: • To generate customized test coverage reports • To set parameters for Savannah QA reports • To repeat this talk for projects / experiments Thank you for your attention! Question? SPI - Software Process & Infrastructure