Reducing S/W Development Cycle Time Through Lean Six Sigma by agileethos

Reducing S/W Development Cycle Time Through Lean Six Sigma Darian Rashid Lean Six Sigma MBB darian@agileethos.com 732-213-8522

Â© Agile Ethos. Do Not Reproduce

Agenda • The Waterfall model for software development • Introduction to Lean • Wastes in Waterfall • Designing a new system

The Coin Game – Round 1 Instructions: •

Form teams of 5 – 7 people

•

Choose one person to act as the manager

•

The team has to sit next to each-other, in a row, with the manager either first or last

•

Reset all coins (Set each one to heads)

•

Game 1: 1. The manager starts the clock and tells the team to start 2. The first player flips all coins, one at a time (with one hand only) 3. Once this is done, all coins are slid over to the next person, who flips all the coins, one by one. NOTE: A player cannot start until the previous player is finished flipping ALL coins 4. Repeat down the line until all coins are flipped by each person on the team (except for the manager) 5. The manager records the total time

Coin Game Debrief •

What did this game represent? ___________________________________________________

•

What were some important metrics we needed to track? ___________________________________________________

•

In what ways can we reduce the cycle time? ___________________________________________________ ___________________________________________________ ___________________________________________________ ___________________________________________________

Problems With Product Development •

Product development has inherent problems, no matter where a developer works: – Requirements are often • Poorly defined • Ambiguous • Incomplete • Incorrectly interpreted by the architect/developer – Schedules are too tight • Deadlines are often missed • Becomes “normal” for developers often work nights and weekends – Never enough adequate testing – Too many bugs slip into the final release – Usually too few people in any department (S/W eng., test eng., etc.)

An Example PDP â&#x20AC;&#x201C; The Waterfall Model Other Sources

Customer Input High-Level Goals

Requirements Analysis High & LowLevel Design Develop Code & Unit Test Systems & Other Tests Deployment & Maintenance ÂŠ Agile Ethos. Do Not Reproduce

What is Lean? •

An approach to improve value streams in office, service, or manufacturing processes

•

The principles of Lean: 1. Eliminate waste 2. Minimize inventory 3. Maximize flow 4. Pull from customer demand (Just in Time production) 5. Exceed customer requirements 6. Do it right the first time 7. Empower employees 8. Design for rapid changeover 9. Partner with suppliers 10. Continuous improvement

Lean Focuses On… • Maximizing – Smooth flow (eliminate bottlenecks / re-balance work) – Visibility (spot progress and problems, maintain a clear line of vision to anywhere, from anywhere) – Communication through structure (create cells, combine operations, eliminate cubes, …) – Utilization of people, space, and equipment – Flexibility (layout and structure) •

Minimizing

– – – –

Handling and strain (ergonomics) Distances (avoid walking, carrying) Clutter Storage (continuously minimize storage space)

Types of Wastes in Lean

Transportation: Movement of goods produced I nventory: Any fully or partially manufactured product M otion: of resources W aiting: Any delay that leads to idle time O ver production: Producing more than needed O ver processing: Doing more work than needed D efects: any product or service that doesn’t meet the customer’s specifications (or requirements) © Agile Ethos. Do Not Reproduce

Transportation Manufacturing/Office

•

Software Development

Moving products between locations

• •

Loading, and unloading on trucks

• • •

Retrieving samples and lab results

Moving parts in and out of inventory or storage Retrieving materials and supplies Example: Moving products from one warehouse to another

•

Handoffs

•

Switching a module back and forth between development and testing

•

Context/task switching - Multiple projects per developer at the same time

•

Reducing Transportation: •

Have dev. and test work together

•

Complete tasks serially so each one may be done as fast as possible

Inventory Manufacturing/Office

• • • • • •

Software Development

Finished goods

•

Any module that is not deployed (Software WIP)

•

Partially finished modules still being worked on

•

Any abandoned modules

•

Any piece of code written that isn’t being used

•

Reducing Inventory: Get software into production as soon as possible; WIP cap

Work in process (WIP) Raw materials Storage and supplies Emails Example: Airplanes on the ground

Batching Requirements

45%

13% 16% 19%

Always Often Sometimes Rarely Never

 Eliciting all the requirements before a project begins is proven to •

Produce bottlenecks

•

Produce waste



A study of several thousand waterfall projects showed that only about 20% of requested requirements are used in the project(Johnson)



~65% rarely or never used



Wasted time, money & resources 

Gathering requirements



Developing



Testing



Deploying



Maintaining

 Optimized approach: Start with the 10-15% of the most vital requirements and elicit more with each Sprint © Agile Ethos. Do Not Reproduce

Motion Manufacturing/Office

• •

Software Development

Any excess/unnecessary motion of resources Searching for parts, supplies, printouts, bills-of-material ...

• • •

Reaching for equipment and tools

•

Summarizing data into useful information

• •

Extra clicks, key strokes, and code

•

Traveling to meetings

•

Walking down the hall to get your question answered

•

Walking to colleagues in separate floors

Sorting, lifting, pushing, pulling, etc. • Searching and gathering data and files

Example: Messy workspaces

Transferring customers through multiple support staff

•

Movement/handoffs of documents through the development process (requirements, architecture, compiled modules, etc.)

•

Reducing Motion: Minimize distance and handoffs

Mapping Motion

ÂŠ Agile Ethos. Do Not Reproduce

Waiting Manufacturing/Office

Software Development

• • • • •

Waiting on materials and supplies

•

Delays in starting projects

Waiting on data and information

•

Delays due to massive architecture

Waiting on people

•

Delays due to redesign/rework

Waiting on equipment

•

Waiting for maintenance, set-up, inspection

Delays due to signoffs (customer and internal)

•

Delays due to late releases

• •

Waiting on parts or prints

•

Delays due to slow staffing

Waiting on a supplier, auditor, regulator, or customer

•

Reducing Waiting: Smaller and more rapid releases

• •

Waiting because of batches Waiting because of queues

Overproduction Manufacturing/Office

Software Development

• •

Inflating batch sizes to avoid set-ups

•

“Make to stock/assemble” based on sales forecast and production schedules

Adding extra features that aren’t needed or requested

•

Adding more documentation than customer needs

• •

“Make to fill time”

•

“Over-testing” features

Making copies and “FYI/CYA” distribution

•

Reducing Overproduction: Put in only those features that are absolutely necessary

•

“Distribute all” and “reply all” email distribution

•

Creating more information than customer needs

•

Creating reports no one needs or reads

Overprocessing (Unnecessary Activity) Manufacturing/Office

• •

Software Development

Paperwork and duplicate forms (e.g., doctor’s office)

•

Customer sign-offs

•

Requirements to code traceability

Over-tight safety margins and tolerances

•

Change controls

•

Development metric tracking

•

Documents done for the sake of following a process (e.g., for ISO compliance, etc.)

•

Reducing Overprocessing: See what documents people are waiting for

• •

Bad design

• •

Supplemental manual entry of data

Use of outdated standard formats and hardcopy Use of outdated or inappropriate software

Defects Manufacturing/Office

Software Development

• • • • •

Scrap, rework, and repair

•

Bugs, bugs, bugs

Field failures

•

Bugs that aren’t immediately detected (i.e., during poorly run unit tests)

•

Requirements errors/omissions

•

Architecture/design errors

• • • •

Pricing error

•

Reducing defects: Assign smaller pieces of code to develop and test

Containment and correction Metric variation Missing parts, fixtures, supplies, hardware, etc. Missing data and information Design or engineering errors Lost data, records, specifications, or approvals

The Hidden Process Step 1

•••

Step i

Rework

•••

Step n

Rework REWORK: The Hidden Process

The Coin Game – Round 2 •

Game 2: 1. 2. 3. 4. 5.

•

The manager starts the clock and tells the team to start The player 1 flips 10% of the coins (with one hand) and slides them to player 2 Once player 2 has the coins, player one immediately starts the next batch of coins Player 2 flips all their coins, hands it to player 3 and starts on the next batch The manager records the total time

Debrief: – What is different about this process compared to the one in Game 1? _____________________________________________________________ _____________________________________________________________ – How can we do better? _____________________________________________________________ _____________________________________________________________

Designing a New System • The waterfall method aims for a perfect execution the first time – Leaves no room for learning – Leaves no room for clarification of requirements – Leaves no room to be flexible when the customers “change their mind” once they see a feature – Leaves little to no room for feedback from any stage downstream to any stage above – Does not work for “unstructured” problems • There is an evolution toward solving ill-conceived problems through “learning cycles” – short bursts of exploration, experimentation, and verification

Designing a New System • The waterfall method violates fundamental Lean principles – – – – – –

Generates enormous waste Maximizes batching/bottlenecks Maximizes Work In Process Minimizes flow Push model Cannot change requirements rapidly during the development cycle

Designing a New System • Release working software within short bursts (e.g., 2 weeks) • Each burst will have: – – – – –

Requirements Designs/Architecture Development Testing Deployment (even if it is internal)

• Each burst will start with the best version of requirements gathered from the customer

Designing a New System • After each burst – A working piece of software (feature increment) is demoed to the customer – The customer can then verify or comment on what is shown – The product backlog is updated – Lessons learned during development about features, architecture, testing, etc. are shared

• We gain knowledge through each burst, even if the customer says the deliverable is not what is wanted • The next burst iterates on the knowledge gained in the current one © Agile Ethos. Do Not Reproduce

Iterative Development… •

Increases (not avoids) communication with the customer – Delights the customer – Encourages the developer – Enables the “pull” model

•

Allows tests to be run as soon as possible instead of all at the end of the waterfall process – Increase flow – Reduces batching

An Iterative Development Framework Requirements elicited and added to product backlog Initial requirements discussed, elicited and prioritized

Daily Planning Meeting

Initial product backlog produced

Sprint backlog produced and prioritized

24 hours

Initial sprint backlog produced and prioritized

Iteration

Sprint Backlog

Backlog tasks expanded by team

1â&#x20AC;&#x201C;4 Weeks High Quality Product Increment

Product Backlog as Prioritized by Product Owner

Customer Feedback & Key Learnings Team Retrospective Adapted from Agile Software Development with Scrum by Ken Schwaber and Mike Beedle

ÂŠ Agile Ethos. Do Not Reproduce

Cross-Functional Teams •

Works together as a whole – No “us and them”

•

Run like small businesses – Up to the team to respond to their customer(s) – Do more with less…

•

Foster product innovation through feedback cycles

•

Just ask George Lucas and Intuit…

Rapid Release

Feedback

The Organizational Feedback Loop t c a p m i o t e m i t e l c Cy Customers/Users Define Value

Product Managers Understand Value

Product Development Deliver Value ÂŠ Agile Ethos. Do Not Reproduce

Case Study: Joint Strike Fighter • • • Company • •

Product:

Team:

Lockheed Martin Fortune 500 140,000 Employees Located in 939 facilities in 457 cities across 45 states Present in 56 countries and territories

• • •

Autonomic Logistics Information System F-35 Lightning II (Joint Strike Fighter) Product of products

• • • •

Team of interest built software products for the fighter More than 200 people 7 Locations 3 time zones

Case Study: Joint Strike Fighter • • • •

Release rate: 1/year or less Last 2-3 months were fire drills Handoffs from phase to phase assumed 100% completion before handoff Siloed groups led to little to no communication

• •

Problems: •

Moved to • Agile: •

With the customer

Quarterly requirements updates often put development teams at square one

• •

Between groups

Business team needed to move with the market

Bottom line: Could not get the “Right product right”

Settled on moving to Agile using Scrum Deployment model: Pilot  Department  Enterprise

Case Study: Joint Strike Fighter •

83% of groups saw a significant increase in productivity

•

79% of groups had significantly fewer defects delivered to the customer

Results:

•

78% of groups had an accelerated time to market

•

66% of groups had a reduced cost

•

Moved from “big bang testing” model to continuous system integration

• • • •

Faster end-customer feedback Allows movement with changing requirements More stable code base Costs much less to fix problems

The Coin Game – Round 3 •

Repeat the game applying any and every Lean thinking tool you can – GOAL: Improve on the cycle time achieved in Game 2 to under 1 minute – Create a “Lean process” to flipping coins as a team – Map it out – Discuss roles – Execute

•

Constraints: – You can only use one hand to flip – Stacking coins and flipping the stack is also not allowed – You cannot add resources from other teams – You may not “outsource” the work – The coin must touch the table on EVERY flip

Main Points • The Waterfall model contains many forms of waste • Large-batch development leads to an inventory of unused features • The Iterative development applies immediate learning • Iteration delivers highest-value items first • Iteration Allows for change if and when needed