Example: Implementing Level 4

1. Example: Implementing Level 4 SED Company

2. SED Project Characteristics Project Team • Type: • New Development - 40% • Sustainment – 60% • Applications: • Trainer/Simulator • Test Device/Tool • Command, Control & Communication • Tactical • Size of team: 4 – 25 people • Duration: 3 – 24 months • Product size: 400 – 160,000 LOC • Languages: 11 including Ada, C, Fortran. Project Lead Engineering Staff SQA CM

3. SED Process Improvement Program Dir. & Dep. Dir. • Based on the SEI’s Capability Maturity Model for Software. • Monitored by the Quality Management Board (SED senior management). • Facilitated by the Software Engineering Process Group (Government and contractor). • Supported by the training program. QMB Div. Chiefs SEPG TAC Project Teams Resources: SEPG = 9 people, 4 FTE MYs TAC = 5 people, 1.5 FTE MYs Level 1 Level 2 Level 3 Level 4 1990 1991 1994 1996 2000

4. Challenges to Reaching Level 4 • Understanding the CMM terminology. • Improving the metrics program. • Documenting the processes and procedures. • Training the personnel. • Maintaining focus during implementation.

5. Challenge #1 • Challenge: Understanding the terminology in the CMM/SW Level 4 Key Process Areas. • Responses: • Researched and studied available material (i.e. Crosstalk, SEI web site) • Participated in conferences and symposiums (i.e. SEPG, STC, SEI) • Communicated with the SEI (i.e. Paulk, Carleton, Florac). • Interfaced with high maturity organizations (Air Force).

6. CMM/SW Level 4 • Managing with facts and data. • Understanding how processes perform. • Understanding the quality of the developed products. • Using organizational business goals and projects’ issues to determine the quantitative analysis applications.

7. Challenge #2 • Challenge: Improving the metrics program. • Responses: • Employed a full-time metrics analyst with a background in statistical process control. • Established organizational goals with senior management. • Aligned the metrics with the organizational goals and process (SEPH). • Automated the collection and reporting of metrics.

8. Metrics Program • Metrics: effort, size, duration and defects. • Metrics Database contains: > 265 builds > 300 Peer Reviews > 250,000 effort hours > 455,000 lines of delivered code (11 languages) • Statistical Process Control (SPC) • applied to: • Productivity • Review (Scan) Rate for peer reviews • Defect Density for peer reviews

9. Challenge # 3 • Challenge: Documenting the processes and procedures. • Responses: • Utilized an approach appropriate for SED. • Updated the SEPH, the SEPG Operations Handbook and other documents to include the Level 4 activities. • Incorporated the quantitative process management plans into the project plans. • Modified peer review procedures and forms to include the application of SPC.

10. Standard Software Process • The SEPH (SED Engineering Process Handbook) • Contains processes, procedures and standards for new development, transition and sustainment projects. • Includes guidance for the collection of project metrics. • Tailoring of the SEPH is performed by the Project Team, reviewed by the SEPG mentor and approved by management.

11. Challenge #4 • Challenge: Training the Level 4 process and procedures. • Responses: • Modified previously developed courses. • Developed and taught an SPC application course. • Minimized class durations and maximized course offerings. • Supplemented with mentoring from SEPG members.

12. Training Program Training Plan • Required training based on role performed. • Training Database maintains records. • Training schedule, data, charts available on-line. • Management provided semi-annual progress reports. Training Program Handbook Training Report Training Totals

13. Challenge #5 • Challenge: Maintaining focus during implementation. • Responses: • Conducted regular SQA audits and an internal mini-assessment. • Reported readiness to senior management on a regular basis. • Utilized the email, newsletter and intranet as communication sources. • Held assessment preparation meetings with all projects.

14. Assessment Details • 2 week CMM Based Appraisal for Internal Process Improvement • Certified Lead Assessor – Dave Zubrow of SEI • Assessment Team members represented 6 different organizations/companies • 54 Assessment participants • 18 projects • completed CMM Compliance Checklists • compiled Artifact Notebooks • participated in interview sessions • 4 Project Leads from diverse projects

15. Maturity Profile Apr 2000 Level 4 SED Assessments Nov 1996 Level 3 May 1994 Level 2 Major Determinants in a Process Capability Rating Sept 1991 Level 1 • The Process is Documented • The Process is Taught • The Process is Used • The Process is Measured *From March 2001 SEI report, based on most recent assessment of 1012 organizations.

16. Results • Increased productivity • Since 1995, productivity has doubled. • Peer reviews are 9 times more cost effective than testing. • High quality products • Post delivery defects are below 1 per KSLOC. • In past two years, a 10% decrease in the defect density of peer reviewed products. • Organizational growth • The number of software development/sustainment projects has increased by a factor of 4 since 1992.

17. Keys to Success • Long term, consistent commitment by senior management. • Total staff (Government and contractor) involvement. • Establishment of organizational goals provided guidance for the metrics program. • Automation used to disseminate information and collect data. • Dialogue with SEI and high maturity organizations.