Scott Ambler - Background

1. Agile Software Development:The Full StoryScott W. AmblerPractice Leader Agile Developmentscott_ambler@ca.ibm.com

2. Scott Ambler - Background • Practice Leader Agile Development • Senior Contributing Editor, Dr. Dobb’s Journal • Fellow – International Association of Software Architects • www.ibm.com/rational/bios/ambler.html • www.ibm.com/developerworks/blogs/page/ambler

3. Agenda • Warning! • Agile Current Status • Common Agile Practices • Scaling Practices • Call to Action

4. Warning! • I’m spectacularly blunt at times • Many new ideas will be presented • Some may not fit well into your existing environment • Some will challenge your existing notions about software development • Some will confirm your unvoiced suspicions • Don’t make any “career-ending moves” • Be skeptical but open minded

5. Agenda • Warning! • Agile Current Status • Agile Adoption Rates • Project Success Rates • Common Agile Practices • Scaling Practices • Call to Action

6. Has Your Organization Adopted One or More Agile Techniques? 85% have run multiple agile projects 24% of “No” respondents hope to do Agile this year Source: Dr Dobb’s 2007 Agile Adoption Survey www.ambysoft.com/surveys/

7. % of Successful Agile Projects(296 co-located, 251 not co-location, 130 offshoring): Agile Adoption Survey

8. Largest Team Size Attempted vs. Successful

9. Why Agile/Lean? It’s More Successful • Quality: 87.3% believe that delivering high quality is more important than delivering on time and on budget • Scope: 87.3% believe that meeting actual needs of stakeholders is more important than building the system to specification • Money: 79.6% believe that providing the best ROI is more important than delivering under budget • Staff: 75.8% believe that having a healthy workplace is more important than delivering on time and on budget • Schedule: 61.3% believe that delivering when the system is ready to be shipped is more important than delivering on schedule Source: Dr Dobb’s 2007 Project Success Survey

10. Agenda • Warning! • Agile Current Status • Common Agile Practices • Agile Development Practices • Test-Driven Development (TDD) • Database Refactoring • Other Quality Practices • Working in Priority Order • Agile Planning • Agile Model Driven Development (AMDD) • Agile User Experience • Agile Documentation • Scaling Practices • Call to Action

11. Agile Development Practices • Regular Delivery of Working Software • Only valid measure of progress • Provides visible results to stakeholders • True earned value, not documentation-based “earned value” • Daily Stakeholder Interaction • On-Site Customer • Active Stakeholder Participation • Product Owner • Continuous Integration • Automatically compile, test, and style check your code • Continuous code integration is nice • Continuous system integration is nicer

12. Test First Design (TFD)www.agiledata.org/essays/tdd.html • With TFD you write a single test and then just enough production code to fulfill that test • Test-Driven Development (TDD) = Refactoring + TFD • TDD is a just-in-time (JIT) specification activity • TDD is a continuous confirmatory validation activity • TDD via Customer/Acceptance Tests • Specification of requirements • TDD via Developer Tests • Specification of design • TDD is also called Behavior Driven Development (BDD)

13. Other Agile Quality Practices • Non-solo development • Pair programming • Modeling with others • Effectively continuous inspections • Following guidance • Coding practices • Database standards • User interface (UI) standards • Modeling style guidelines (www.agilemodeling.com/style) • Refactoring • Small change to your code which improves the quality of the design without changing the semantics • Code refactoring • UI refactoring • Database refactoring

14. Database Refactoring • A database refactoring is a simple change to a database schema that improves its design while retaining both its behavioral and informational semantics. Examples: Move Column, Rename Table, and Replace Blob With Table. • A database schema includes both structural aspects such as table and view definitions as well as functional aspects such as stored procedures and triggers. • Important: Database refactorings are a subset of schema transformations, but they do not add functionality. • www.agiledata.org

15. Working in Priority Order: Agile Change Managementwww.agilemodeling.com/essays/agileRequirements.htm

16. Agile Planning • Create and maintain a high-level Gantt chart indicating the iterations, milestones, and major dependencies • Plan each iteration in detail at the beginning of the iteration • Done by the team, not just the manager • The people best suited to plan the work are the people who are going to do the work • Consider planning poker, www.planningpoker.com • DDJ’s 2007 Adoption survey, most valuable work products: • #5 was an iteration task list • #18 was a high-level Gantt chart • #19 (of 19) was a detailed Gantt chart

17. Agile Model Driven Development (AMDD)www.agilemodeling.com/essays/amdd.htm • Do just enough initial envisioning to understand the scope and technical direction • Model storm on a just-in-time basis to gather the details when you need them

18. Agile User Experience (UEX) • Observations: • User interface (UI) and usability issues are critical to the success of most systems • The UI is the system to the end user • Few developers have solid UEX skills, although many think they do • Advice: • Everyone should have some UEX training • Have someone with UEX expertise within your organization, and ensure that they pair regularly • Part of initial envisioning should address UEX issues • UEX issues will need to be addressed throughout development • Recognize that few of us are building the iPod, but when we tread into new territory we may need to do more up-front work than usual

19. Agile Documentation Practiceswww.agilemodeling.com/essays/agileDocumentation.htm • Maximize stakeholder ROI • Are treated as a requirement • Have a specific customer and facilitate the work efforts of that customer • Are concise • Fulfill a purpose • Describe information that is less likely to change • Describe “good things to know” • Are sufficiently accurate, consistent, and detailed – But aren’t perfect

20. Agenda • Warning! • Agile Current Status • Common Agile Practices • Scaling Practices • Challenges with Mainstream Agile • Scaling TDD via Agile Model Driven Development (AMDD) • Scaling TDD’s via Comprehensive Testing • Scaling On-Site Customer/Product Owner • Scaling via Rational Unified Process (RUP) • Portfolio Management • Enterprise Architecture • Agile Data Management • Lean Development Governance • Call to Action

21. Compliance requirement Critical, Audited Low risk Geographical distribution Entrenched process, people, and policy Co-located Global Minimal Significant Organization distribution (outsourcing, partnerships) Application complexity Simple, single platform Complex, multi-platform Third party In-house Team size Degree of Governance Under 10 developers 100’s of developers Informal Formal Challenges with Agile in the Mainstream Agile Development

22. Scaling TDD: Agile Model Driven Development (AMDD) www.agilemodeling.com/essays/amdd.htm

23. Scaling TDD: Comprehensive Agile TestingJanuary 2007 Dr. Dobb’s Magazine (www.ddj.com/dept/debug/196603549)

24. Scaling XP’s On-Site Customer and Scrum’s Product Owner • On-site customer is nice, so put them to work • Stakeholders can be active participants in modeling • Product owner is really a communication conduit between the team and stakeholders • Must have agile business analysis skills • PO gets the team access to the relevant stakeholders just in time • Negotiate, negotiate, negotiate • Dr. Dobb’s Journal, January 2008

25. Database Testingwww.agiledata.org/essays/databaseTesting.html

26. The Generic Agile Lifecycle

27. Scaling via Rational Unified Process (RUP) • RUP socialized many of the concepts taken for granted by the Agile community • RUP is really a process framework, not a process • RUP can be as Agile, or non-Agile, as you want to make it • Many organizations struggled to implement RUP effectively • RUP: • Addresses the fully development lifecycle • Is risk-driven • Contains advice for most of the challenges currently faced by Agile • RUP done right is Agile, RUP done wrong is just plain wrong

28. Portfolio Management • Driven by the enterprise vision and regulatory restrictions • Focus on collaboration and enablement, not command and control • Manage enterprise risk • Understand the as-is “IT inventory” • Identify potential projects • Choose the highest value projects • Organize similar projects into programs • Steer existing development projects and programs • Manage services contracts • Work closely with project managers • Monitor projects

29. Enterprise Architecture • Provide technical vision to the enterprise • Promote reuse and common infrastructure • Develop reference architectures • Develop guidance • Work closely with development teams • www.agiledata.org/essays/ enterpriseArchitecture.html

30. Agile Data Managementwww.agiledata.org • Traditional data management has clearly failed: • Data Warehouse Institute (DWI) estimates data quality to have a $611B annual impact in the US • DDJ found that 62% of organizations have production data quality problems yet the majority have no viable strategy for addressing them • DDJ found that the majority of organizations have no database testing strategy in place, and many haven’t even considered it • DDJ found that over 60% of development teams now go around their organizations’ data groups • This is now the “elephant in the room” for most organizations • A new vision: • Evolutionary and collaborative approaches • Test-driven approaches • Dovetail into enterprise architecture and administration efforts, no longer a silo effort

31. Align HR Policies With IT Values • Align Stakeholder Policies With IT Values Lean Development Governancewww.ibm.com/developerworks/ • Pragmatic Governance Body • Staged Program Delivery • Business-Driven Project Pipeline • Scenario-Driven Development • Iterative Development • Adapt The Process • Risk-Based Milestones • Continuous Improvement • Embedded Compliance • Simple And Relevant Metrics • Continuous Project Monitoring Organization Processes Mission & Principles Measures Roles & Responsibilities Policies & Standards • Integrated Lifecycle Environment • Valued Corporate Assets • Flexible Architectures • Promote Self-Organizing Teams • Align Team Structure With Architecture

32. Agenda • Warning! • Agile Current Status • Common Agile Practices • Scaling Practices • Call to Action

33. A Call To Action • Look beyond development trees to see the business forest • Recognize that “the age of hype” is over • Talk about everything that we do, not just the cool/extreme things that we like to talk about • Bring agile concepts to other communities • Their questions will reveal many of the challenges we still face • Invite outsiders into our community • We need more “uncomfortable” keynotes • Police mailing lists a bit better • We turn off a lot of smart people who have something to contribute

34. Keep In Touch! Scott W. Ambler www.ibm.com/rational/bios/ambler.html www.ibm.com/developerworks/blogs/page/ambler

35. References and Recommended Reading • www.agilealliance.com • www.agilemodeling.com • www.agiledata.org • www.enterpriseunifiedprocess.com • www.ibm.com/rational/agile/ • Ambler, S.W. (2002). Agile Modeling: Effective Practices for XP and the UP. New York: John Wiley & Sons. • Ambler, S.W. (2003). Agile Database Techniques. New York: John Wiley & Sons. • Ambler, S.W. (2004). The Object Primer 3rd Edition: AMDD with UML 2. New York: Cambridge University Press. • Ambler, S.W. and Sadalage, P.J. (2006). Refactoring Databases: Evolutionary Database Design. Reading, MA: Addison Wesley Longman, Inc. • Larman, C. (2004). Agile and Iterative Development: A Manager’s Guide. Reading, MA: Addison Wesley • McGovern, J., Ambler, S.W., Stevens, M., Linn, J., Sharan, V., & Jo, E. (2003). The Practical Guide to Enterprise Architecture. Prentice Hall PTR. 

36. We value: Individuals and interactions Working software Customer collaboration Responding to change Over: Processes and tools Comprehensive documentation Contract negotiation Following a plan Agile Valueswww.agilealliance.com

37. Agile Principleswww.agilealliance.com • Our highest priority is to satisfy the customer through early and continuous delivery of valuable software. • Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage. • Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale. • Business people and developers must work together daily throughout the project. • Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done. • The most efficient and effective method of conveying information to and within a development team is face-to-face conversation. • Working software is the primary measure of progress. • Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely. • Continuous attention to technical excellence and good design enhances agility. • Simplicity--the art of maximizing the amount of work not done--is essential. • The best architectures, requirements, and designs emerge from self-organizing teams. • At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.