Issuu on Google+

Increasing Responsiveness and Economy of Software Inspection Marko Komssi marko.komssi@f-secure.com 2004-12-03


Contents Problems with inspection processes and rules Generating a minimal number of inspection rules based on risks and goals Progressively increasing the precision of inspection practices

2


Traditional Inspection Process1

Entry Kickoff and Meeting Planning

Individual Checking

Logging Meeting

Specification and Process Meetings

Editing

Edit Auditing

Exit

3


Inspection Process Variables • Amount of inspection rules and phases • Number of inspectors and their roles • Duration of inspection sub-phase and total time spent • Checking rate • Proportion of product under inspection • Experience of inspectors • Rigidity of inspection entry and exit criteria

4


Inspection Rules A common way to seek errors is to verify the product against inspection rules Defects in a code or a document are violations of a rule CUSTOMER — customer requirements must be real • Does the statement in the specification describe the customer? • Is the need of the customer correctly presented?

5


Problems with Inspection Rules and Processes A rigorous and stable inspection process • Different company maturity levels and development processes • Alternative quality goals and development phases of projects • A high threshold for using a heavy inspection model

Ineffective inspection rules • Inspection rules may reveal only minor defects • Defects may have high severity but low priority

6


Effectiveness of Inspection Rules (Requirements)

7


Effectiveness of Inspection Rules (User Interface)

8


Not Too Much Precision, Please! Write long and precise specifications only if you know what you are doing • “Odd though it may sound, you will typically do less damage if you write too little rather than too much4.“ • “Even the smallest features pack more calories than you might think5.“

Take compactness as part of a inspection strategy • “Then you will have a short, readable document, which means that people will bother to read it and then ask questions4.“

Include formality in review practices just as much as is needed6

9


Goal- and Risk-Based Inspection Rules The three most effective rules cover 60–80 % of problems It is economical to inspect issues that: • support the goal of the product • are likely to harm the project in the current development phase

Rule generation can be done by imitating the concepts of Goal-Question-Metric2 (GQM) and Risk-based testing3

10


Progressive Inspection Progressive inspection (PI) is a concept for improving the traditional inspection process: • Increases the precision of review practices as the project development progresses • Uses the cheapest possible review techniques associated with the quality goals and the risks of the product6 • Weights different stages of inspection

11


Emphasis in the Beginning of Development

Entry and Planning

Kickoff Meeting

Individual Checking

Specification and Logging Meeting Process Meetings

Editing

Edit Auditing

Exit

12


Slim Quality Goals in the Beginning of Development

Entry and Planning

Kickoff Meeting

Individual Checking

Few rules

Specification and Logging Meeting Process Meetings

Editing

Edit Auditing

Exit

Soft exit criteria 13


Emphasis in the End of Development

Entry Kickoff and Meeting Planning

Individual Checking

Specification and Logging Meeting Process Meetings

Editing

Edit Auditing

Exit

14


Increasing Precision in the End of Development

Entry Kickoff and Meeting Planning

Individual Checking

A larger set of rules

Specification and Logging Meeting Process Meetings

Editing

Edit Auditing

Exit

Rigid exit criteria 15


PI inside XP and EVO In XP7, high-level goals of user stories can be questioned with targeted inspections In EVO8, inspection and its techniques can be tailored based on the needs of the project • Inspections are started by concentrating in planning and brainstorming • In each inspection cycle, practices are developed based on comments from the inspection data and the team

16


Conclusions Traditional inspection practices may be cumbersome Three inspection rules suffice in the early stages of development In PI, inspection practices increase in precision and progress during project development

17


References 1.

Gilb, T., Graham D., Software Inspection. Addison-Wesley (1993).

2.

Basili, V.R., Rombach, H.D., The TAME Project: Toward ImprovementOriented Software Environments. IEEE Transactions on Software Engineering, 14, 6 (1988), 758–773.

3.

Bach, J., James Bach on Risk-Based Testing. How to Conduct Heuristic Risk Analysis? Software Testing & Quality Engineering, 1, 6 (1999), 23–28.

4.

Cockburn, A., Writing Effective Use Cases. Addison-Wesley (2001).

5.

McConnell, S., Achieving Leaner Software. IEEE Software, 14, 6 (1997), 127–128.

6.

Wiegers, K.E., Peer Reviews in Software: A Practical Guide. AddisonWesley (2001).

7.

Beck, K., Embracing Change with Extreme Programming. Computer, 32, 10 (1999), 70–77.

8.

Gilb, T., Principles of Software Engineering Management. Addison-Wesley (1988).

18


g


Europe’s Premier Software Testing Event World Forum Convention Centre, The Hague, Netherlands

“The Future of Software Testing”

Traditional QA Meets Agile Development Dietmar Strasser, Borland, Austria WWW.QUALTECHCONFERENCES.COM


Traditional Testing meets Agile Development Dietmar Strasser Director QA, Lifecycle Quality Management dietmar.strasser@borland.com


Agenda Journey towards an Agile Team Our Environment we live in How do we provide Visibility? Q & A


Journey towards an Agile Team


Starting Point

PM

SilkPerformer Developers

PM

SilkTest Developers

PM

Test Manager Developers

QA

PM ... Product Manager PO ... Product Owner Copyright Š 2008 Borland Software Corporation.

Confidential

5

Doc


Adding Product Owner

PM

PM

PM

PO

SilkPerformer Developers

PO

SilkTest Developers

PO

Test Manager Developers

QA

PM ... Product Manager PO ... Product Owner Copyright Š 2008 Borland Software Corporation.

Confidential

6

Doc


User Story Workflow – Testers not integrated In Progress QA

Unassigned PO

QA

Drafted

QA Ready

PO

Dev PO

Approved

In Progress Dev

Not Started

In Progress

Copyright © 2008 Borland Software Corporation.

Confidential

7

QA

Drop Ready Drop Ready QA

RTM Ready

RTM Ready


User Story Workflow - „Small Waterfall“

Testing In Progress

Unassigned

QA •User story •Iteration x+1 •4 weeksQA

PO

Drafted

QA Ready Dev Coding

PO

Approved

PO •User story In Progress •Iteration x Dev

•4 weeks

Not Started Copyright © 2008 Borland Software Corporation.

In Progress Confidential

8

QA

Drop Ready Drop Ready

Release Testing RTM Ready QA

•All user stories •Last RTMIteration Ready •4 weeks


Lesson Learned

“Ask the Team”

Copyright © 2008 Borland Software Corporation.

Confidential

9


Adding Tester Skills

PM

PM

PM

PO

SilkPerformer Developers

Tester

PO

SilkTest Developers

Tester

PO

Test Manager Developers

Tester

PM ... Product Manager PO ... Product Owner Copyright Š 2008 Borland Software Corporation.

Confidential

10

Doc


Adding Documentation Skills

PM

PM

PM

PO

SilkPerformer Developers

Tester

Doc

PO

SilkTest Developers

Tester

Doc

PO

Test Manager Developers

Tester

Doc

PM ... Product Manager PO ... Product Owner Copyright Š 2008 Borland Software Corporation.

Confidential

11


Transition to Engineering Team

PM

PO

SilkPerformer Engineering Team

PM

PO

SilkTest Engineering Team

PM

PO

Test Manager Engineering Team

PM ... Product Manager PO ... Product Owner Copyright Š 2008 Borland Software Corporation.

Confidential

12


Adding QM Coach

QM Coach PM

PO

SilkPerformer Engineering Team

PM

PO

SilkTest Engineering Team

PM

PO

Test Manager Engineering Team

PM ... Product Manager PO ... Product Owner Copyright Š 2008 Borland Software Corporation.

Confidential

13


Splitting & Re-Locating Teams

QM Coach PM

PO

SilkPerformer Engineering Team

PM

PO

SilkTest Engineering Team

PM

PO

Test Manager Engineering Team

PM ... Product Manager PO ... Product Owner Copyright Š 2008 Borland Software Corporation.

Confidential

14


User Story Workflow - Agile Unassigned PO

In Progress

Drafted

Agile Scrum Team

PO

Finish user story in DONE one 4-weeks Iteration

PO

Approved

Not Started Copyright Š 2008 Borland Software Corporation.

DONE

In Progress Confidential

15


Lesson Learned

“Agile is a journey, not a destination”

Copyright © 2008 Borland Software Corporation.

Confidential

16


Distributed Team Environment QM Coach

Product Scrum Team(s)

EQC Resource Pool

Daily

EQC Contact

Quarterly

Copyright Š 2008 Borland Software Corporation.

Confidential

17


Lesson Learned

“Communicate, communicate, communicate, …”

Copyright © 2008 Borland Software Corporation.

Confidential

18


Our Environment we live in


Our Environment we live in Project Management, Reporting Management RBT Environment

Iteration Management

Requirements Management

Test Management

Scrum Team(s)

Product Owner

Scrum Team(s)

Source

Functional/

Management

Performance Testing

Scrum Team(s)

Scrum Team(s)

Copyright Š 2008 Borland Software Corporation.

Confidential

20

xUnit

Scrum Team(s)


Lesson Learned

“People are more important than processes & tools”

Copyright © 2008 Borland Software Corporation.

Confidential

21


How do we provide Visibility?


Types of Visibility • Internal Visibility • • • •

Daily Stand-Ups Iteration Review Meetings Regular Updates on Production Systems Project Dashboard

• External Visibility • Regular „Drops“ for customers and field people

Copyright © 2008 Borland Software Corporation.

Confidential

23


Project Dashboard

Goal Story Report

Scrum Team Reports Iteration-Related

Copyright Š 2008 Borland Software Corporation.

Confidential

Quality-Related

24

Getting into Details

User Story Reports

Provide Visibility

(Executive Summary)


Goal Story Report

Copyright Š 2008 Borland Software Corporation.

Confidential

25


User Story Report

Copyright Š 2008 Borland Software Corporation.

Confidential

26


Q&A

Email to:dietmar.strasser@borland.com


Inspection