100 likes | 235 Views
Establishing an Agile Testing Team: Our Four Favorite “Mistakes”. Kay Johansen Anthony Perkins. Our Task: Introduce a QA group to support frequent delivery at a web-hosting provider. An established company with large installed customer base and no existing QA group, delivering weekly updates!
E N D
Establishing an Agile Testing Team:Our Four Favorite “Mistakes” Kay Johansen Anthony Perkins
Our Task: Introduce a QA group to support frequent delivery at a web-hosting provider. • An established company with large installed customer base and no existing QA group, delivering weekly updates! • The “products” were really collections of open-source software customized by our developers; multiple operating systems (also customized); Some applications developed in-house • One two-year old project not gone out the door.
We intended to automate heavily and focus on the four Agile values. • Individuals and interactions • We hired programmers, known team players, gave them no process, but encouraged collaboration. • Working software • We wanted to write automated regression tests. • We made a clean test environment wth daily integrations, didn’t ask for documents. • Customer collaboration • We held bug prioritization meetings, talked with developers many times a day. • Responding to change • We reinterpreted our priorities over time.
We deliberately made 3 “mistakes” (matching “Lessons Learned in Software Testing”) • We didn’t protect the customer • “Never be the gatekeeper” • “Don’t sign-off to approve the release of a product” • We didn’t hire QA experience • “Understand how programmers think” • “Develop programmers’ trust” • We gave in easily • “Beware of becoming a process improvement group” • “Adapt your processes to the practices that are actually in use”
We made friends with the product manager and developers, and integrated daily . . . • Los of “face time” made friends out of the Product Managers (they controlled the release). • Our “programmer/testers” were respected by the product developers. • We found bugs in real code, so development paid attention to our findings. • We always knew the state of the code, so the PM could make scope vs. schedule decisions. • We refused to assign priorities to bugs, so development and the PM had to. • The 2-year old product got shipped.
However, we didn’t actually automate very much, and half our people got laid off. • We hadn’t actually automated many tests, still tested by hand. • Our test coverage was lower than we wanted (much lower!) • The product released later than we -- and management -- wanted. • The company downsized, laid off half our team. • (Hint: we were not perceived as valuable)
Our team decided to reprioritize, in order to demonstrate our value clearly. • We had been too focused on the technical: (test automation and complete coverage) • Needed more focus on the social / political: • We worked on gaining still more trust from the developers • We began status reports to management to show them what we were doing.
Even pushing trade-off decisions up made us look more valuable. • We made deliberate “Mistake” Number 4: We chose not to test everything • We gave management several alternatives of what to NOT test • They chose to drop testing of weekly releases • (From “Lessons Learned in Software Testing”:) • “Beware of testing ‘completely’ ” • “Don’t let yourself be abused”
We succeeded in being perceived as adding value. • When bugs escaped on the untested products, the reaction was "How many more resources do you need?" instead of "How could you have let that happen?" • Our programmers had a genuine interest in the developers’ products, so they went to lengths to help us in any way they could. • We were accommodating where we could be, so people gave us more support when we had an issue we wouldn't give in on. • Finally: We got to hire more people !
Reflection: Hiring programmers was great and prioritizing test coverage was useful. • We liked these things that we did: • Hiring programmers onto the test team • Making friends with the product manager • Prioritized, non-exhaustive testing • Clean test environment • Read “Lessons Learned in Software Testing” • But what happened to the automated testing? • Wasn’t as critical at that point... • We did get around to it over time.