MINITAB®CaféJune, 2006

MINITAB® Café June, 2006 An activity developed by Blackberry & Cross™ for all MINITAB 14 users and Industrial Statistics enthusiasts in Central America

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Acknowledgment Blackberry & Cross has received tremendous help from Rodrigo Picado, (CSSBB, CQE, CQIA), who has provided us with his knowledge and insight about applied DOE in manufacturing processes (real DOE, not an academic exercise). Keep it up!

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Agenda I. II.

Introduction to MINITAB CafĂŠ (5 min) Introduction to Basic of DOE (1 hour) I. II. III.

III. IV.

Coffee Break (15 min) Application Case: Softball manufacturing (1 hour, 30 min) I. II. III.

V.

Experimental Design ANOVA importance DOE basics

Background Softball Manufacturing Process Improvement with FF DOE Conclusions

Wrap-up (30 min) I. II.

Comments Evaluation

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

MINITAB 14

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

What is DOE/DEX? • “Design of experiments (DEX or DOE) is a systematic, rigorous approach to engineering problem-solving that applies principles and techniques at the data collection stage so as to ensure the generation of valid, defensible, and supportable engineering conclusions. In addition, all of this is carried out under the constraint of a minimal expenditure of engineering runs, time, and money.” (1)

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

What is Experimental Design? (1/2) “In an experiment, we deliberately change one or more process variables (or factors) in order to observe the effect the changes have on one or more response variables. The (statistical) design of experiments (DOE) is an efficient procedure for planning experiments so that the data obtained can be analyzed to yield valid and objective conclusions.

DOE begins with determining the objectives of an experiment and selecting the process factors for the study.� (1)

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

What is Experimental Design? (1/2) “An Experimental Design is the laying out of a detailed experimental plan in advance of doing the experiment. Well chosen experimental designs maximize the amount of "information" that can be obtained for a given amount of experimental effort.

The statistical theory underlying DOE generally begins with the concept of process models.� (1)

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Process Models As a refreshment: A statistical model “is a set of mathematical formulas and assumptions that attempt to explain a real phenomenon”.

We need to look for the model that explains the precise and accurate behavior of fundamental data; then, the models “fits” perfectly.

The goal is to develop a parsimonious model (few formulas as possible) to describe the real phenomenon, situation.

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Black Box Model Controlable input factors

x1 x2

... ...

xp

output

input process

z1 z2

... ...

y

zq

Uncontrolable input factors

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Typical Uses of DOE You can use DOE, basically, when you have to: 1. Choose between options 2. Select key factors affecting a specific response (or responses) 3. Hit a specific indicator goal 4. Eliminate or reduce Variability 5. Maximize of Minimize responses 6. Design robust process/products 7. Search for multiple targets 8. Find a Regression model

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Experimental Design: Parsimonious and Simple Some process analysts tend to elucubrate about trying to find the most complex DOE, in order to explain a specific phenomenon.

The secret is to keep the model design and statistical experimentation, as simple as possible.

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

ANOVA in this MINITAB Café We want to take some minutes of this MINITAB Café to talk about, in some aspects, ANOVA, and specifically, One-way ANOVA, since it is a really useful and important tool in experimental design (statistical analysis).

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Can we perform experimental design using ANOVA? • ANOVA, as well as Hypothesis testing, and many other tools, can be used to understand the behavior of phenomena.

• If objectives for the analysis are clear, as well as the identification of the factor (factors) is correct, we are accomplishing the basics of Experimental Design.

• In this sense, ANOVA is for all intents and purposes an statistical experiment (maybe we can not say it is a Factorial DOE, but in its fundamental nature, ANOVA is an experiment)

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

ANOVA Basics – ANOVA is a statistical method used in order to compare the means of a number of populations.

– ANOVA is an acronym of Analysis of Variance.

– Typically, ANOVA and Regression Analysis are considered as advanced application methods www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

ANOVA: is about MEANS The acronym of ANOVA may create a confusion about the purpose of this method.

Remember: ANOVA is a method used with the intention of determining the existence of differences among a number of population means. www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

ANOVA and Variance

The ANOVA method requires the analysis of different sources of variation, variance, linked to the samples to be studied.

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

ANOVA “Analysis of variance (ANOVA) is similar to regression in that it is used to investigate and model the relationship between a response variable and one or more independent variables.�

MINITAB Stat Guide. MINITAB Release 14

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

ANOVA Conceptual Models •

Fixed-effects model: the assumption in this case is that the data in each population under study come from a normal population and means may or may not differ.

• Random-effects models: in this case, the assumption is t that the data describe a hierarchy of dissimilar populations whose differences are inhibited by the hierarchy.

•

Mixed models: these models may be used to explain conditions where both fixed and random effects are present.

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Bok: Fixed Model Most of the analysis required by the ASQ Bok, for CSSBB,CSSGB, CQE, among others, is referred to the Fixed Model ANOVA. If you are interested in further analysis, we will be glad to facilitate you with materials and examples. www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Assumptions of ANOVA There are two basic assumptions: 1.

Independent random sampling from each of the “j” populations under study

2.

The “j” populations are normally distributed with means μ that may or may not be equal (here is the essense of the ANOVA), however the variances σ² are equal

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Effects of Violating Assumptions Normality: The effects of violating the Normality assumption are related with the F-test.

Several authors report that the F test is outstandingly robust to deviations from normality. The main constraint is in terms of the KURTOSIS of the data.

However, the main problem comes up if the kurtosis is greater than 0 (it means the distribution is NOT mesokurtic), the F tends to be small and the power to reject the null hypothesis is reduced. In this case we can not reject, even when the null hypothesis is incorrect.

If the distribution is platykurtic, a similar effect is expected: we will not fail to reject even when this is the right option.

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Side Bar: Kurtosis Refreshment • Distributions with zero kurtosis are called mesokurtic. The most well-known example of a mesokurtic distribution is the normal distribution • A distribution with positive kurtosis is called leptokurtic. The shape of a leptokurtic distribution has a more sharp "peak" around the mean, (v.g.

Laplace Distribution and the logistic distribution) • A distribution with negative kurtosis is called platykurtic. A platykurtic distribution has a smaller "peak" around the mean. the continuous uniform distribution, and the Maxwell-Boltzmann distribution, are examples. www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Homogeneity of Variances Homogeneity of variances assumption is considering that the variances in the different groups of the design are identical Warning: Adding, putting together, the variances of two groups, where the variance of each group is different, is not suitable; It will not provide an approximation of the common within-group variance (in view of the fact that no common variance exists)�

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Homogeneity of Variances The F test is very robust and can handle this violation, however, there is a case in which the averages are correlated with variances across cells of the ANOVA.

Tip: Creating a Scatter plot of variances or standard deviations against the means will detect such correlations.

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Sphericity, Compound Symmetry, MANOVA and ANCOVA These are important concepts when discussing the assumptions of ANOVA, but out of the scope of this MINITAB Cafè.

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Practical Steps for ANOVA Some authors may differ about the sequence, depth, or necessity of some the following proposed steps.

Nevertheless, the following steps can be consider as necessary and good statistical practices.

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Steps to properly developed and apply experimental design with ANOVA 1. Description of the problem (MQC) 2. Determination of the analysis scheme 3. Statement of the null hypothesis 4. Determine Sample Size 5. Perform a Test for Equal Variance 6. Test the Normality of Data 7. Configure the ANOVA Table 8. Run the ANOVA test 9. Analyze results 10. Conduct Tukey’s, HSU’s and/or ANOM 11. Conclude from the experiment

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Sample Size for an ANOVA One Way Remember that when dealing with Power and Sample Size problems, there is always two approaches: – Prospective: before running the experiment – Retrospective: after running the experiment

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Sample Size for an ANOVA One Way Some elements (data) that are necessary: 1. Standard Deviation

2. Minimum detectable difference between the smallest and largest factor means 3. Power of the test www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Sample Size for an ANOVA One Way Assume you have to make an experiment, in order to understand if the aging time of a material affects its strength. You want to find a difference between a control group mean of 10 and a level mean that is 14 (the maximum difference is 4). The assumed average sigma has been 2.5. Your initial sample size was 8, as retrospective approach. Calculte the sample size and/or power of the test. Please, consider the assumption that your customer demands a 90% or more, as Power of the Test

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Power of the Test “The power of a test is the probability of correctly rejecting H0 when it is false. In other words, power is the likelihood that you will identify a significant difference (effect) when one exists.”

MINITAB Stat Guide

“The power of a statistical test is the probability that the test will reject a false null hypothesis, or in other words that it will not make a Type II error. The higher the power, the greater the chance of obtaining a statistically significant result when the null hypothesis is false” Wikipedia

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Sample Size for an ANOVA One way •

Power and Sample Size

•

One-way ANOVA

• •

Alpha = 0.05 Assumed standard deviation = 2.5 Number of Levels = 5

• • •

SS Sample Maximum Means Size Power Difference 8 8 0.661089 4

•

The sample size is for each level.

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Sample Size for an ANOVA One way •

Power and Sample Size

•

One-way ANOVA

•

Alpha = 0.05 Assumed standard deviation = 2.5 Number of Levels = 5

• • • • •

SS Sample Target Means Size Power 8 14 0.90 8 16 0.95 8 21 0.99

•

The sample size is for each level.

Actual Power 0.923493 0.956893 0.990934

Maximum Difference 4 4 4

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Test for Equal Variances If you are using MINITAB choose: Stat > ANOVA > Test for Equal Variances.

Assume you have a product in which you want to improve the variability of the diameter *it is a plastic device.

You know that the back pressure of the injection system contributes to create the diameter, as required.

You have the recorded some previous trials using two pressure levels: 25 psi and 41 psi (this data is coded) Can you conclude that there is no evidence of inequality of the variances?

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Test: Bonferroni and Levene’s •

Test for Equal Variances: Diameter versus Pressure

•

95% Bonferroni´s confidence intervals for standard deviations

• • •

Pressure N Lower StDev Upper 25 9 2.91929 4.55522 9.7110 41 9 3.54091 5.52519 11.7789

• F-Test (normal distribution) Test statistic = 0.68, p-value = 0.598

• Levene´s Test (any continuous distribution) Test statistic = 0.05, p-value = 0.824

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Methods to Test Equal of Variances homogeneity of variances

In a general sense, we can categorize the methods as follows: Test statistics – Bartlett's (better with Normal Distributed data) – F-test (robust against departure from normality; effect of kurtosis should be analyzed) – Levene´s (less power, but it is non-parametric)

Variance Measure – Standard deviation – Variance – Bonferroni´s confidence intervals

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Levene´s Test

[ID: 405]

[Extracted from MINITAB Answers Knowledbase]

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Bonferroni´s Test

[ID: 1008]

• MINITAB provides you with easy-to-read charts, in which you can, graphically, analyze the Bonferroni´s test.

• If you want to make the manual calculations, please refer to ID:1008 of the MINITAB Answers Knowledbase (or check your printed material, and find a copy of the ID:1008, in hard copy)

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Illustration of the Tests Test for Equal Variances for Diameter F-Test Test Statistic P-Value

Pressure

25

Levene's Test Test Statistic P-Value

41

2

4 6 8 10 95% Bonferroni Confidence Intervals for StDevs

0

5

12

25 Pressure

0.68 0.598

41

10

15 Diameter

20

25

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

0.05 0.824

In this case, since the p/value of the F-test and the Levene’s test are significantly higher, we can conclude that there’s no evidence to claim that the variances are unequal.

Normality TEST The GOF test suggest are: 1.

2.

3.

4.

Anderson Darling (AD) Kolmogorov-Smirnov (K-S) Ryan-Joiner P-Value

Some other tests are used in statistical analysis: •

Lilliefors test (an adaptation of Kolmogorov-Smirnov)

• • •

Shapiro-Wilk test (similar to Ryan-Joiner) Normal probability plot Jarque-Bera (based on kurtosis and skewness analysis)

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Exercise • The following exercise has been solved, using MINITAB Release 14.

• Please, refer to Montgomery’s Hardwood Concentration Example.

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

GOF in MINITAB 14

STAT>Quality Tools>Individual Distribution Identification

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Normality Probability Plot of Streng Normal 99

Mean StDev N AD P-Value

95 90

Percent

80 70 60 50 40 30 20 10 5

1

5

10

15

20

25

30

Streng

STAT>Basic Statistics > Normality Test

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

15.96 4.723 24 0.326 0.505

Equal Variances Test for Equal Variances for Streng Bartlett's Test Test Statistic P-Value

1

1.14 0.769

Lev ene's Test Test Statistic P-Value

Hardw

2

3

4

0

1 2 3 4 5 6 7 8 9 95% Bonferroni Confidence Intervals for StDevs

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

0.60 0.623

Sample Size • Assume a maximum difference of 10 units *strength. The assumed pooled standard deviation is: 4.723. • The expected power of the test will be calculated as a retrospective data, since the original sample size was 6 trials, per hardwood concentration

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Power and Sample Size • One-way ANOVA • Alpha = 0.05 Assumed standard deviation = 4.723 Number of Levels = 4

• SS Sample Maximum • Means Size Power Difference • 50 6 0.805586 10 • The sample size is for each level.

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Power and Sample Size •

One-way ANOVA

•

Alpha = 0.05 Assumed standard deviation = 4.723 Number of Levels = 4

• • • • •

SS Sample Target Maximum Means Size Power Actual Power Difference 50 8 0.90 0.927034 10 50 9 0.95 0.957199 10 50 12 0.99 0.992446 10

•

The sample size is for each level.

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Next Step • Run the ANOVA (we have already done it) • At this you should be familiar, with HSU’s tests and Tukey’s.

• We will introduce ANOM.

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

ANOM • Analysis of means *ANOM, is a graphical statistical tool.

• It used, mainly, as a analog method to ANOVA. • However, it is important to emphasize that ANOVA tests whether the factor means vary from each other. ANOM tests against the grand MEAN. www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Why ANOM? There are several scenarios, but one of the main reasons is that ANOVA assumes normality.

ANOM works with data following binomial, Poisson or normal distributions.

Additionally, under certain circumstances, ANOVA may not detect some fluctuation from the mean, but ANOM may detect it.

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

ANOM One-Way ANOM for Streng by Hardw Alpha = 0.05 22.5 20.0 18.38 Mean

17.5 15.96 15.0 13.54 12.5 10.0

1

2

3 Hardw

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

4

Suggested Steps in DOE Planning and Execution • • • • •

Set Objective Select process variables Select experimental design Execute Check consistency of data with the experimental assumptions • Analyze and interpret resuls • Use/present the results (may lead to further runs or DOE's).

NOTE: we will provide you a more detailed list of steps

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Practical Considerations • • • • • • • • • • •

Check performance of gauges/measurement devices first.

Keep the experiment as simple as possible.

Check that all planned runs are feasible.

Watch out for process drifts and shifts during the run.

Avoid unplanned changes (e.g., swap operators at halfway point).

Allow some time (and back-up material) for unexpected events.

Obtain buy-in from all parties involved.

Maintain effective ownership of each step in the experimental plan.

Preserve all the raw data--do not keep only summary averages!

Record everything that happens.

Reset equipment to its original state after the experiment. (1)

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Road Map to DOE Analysis

From NIST/Sematech (see references)

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Good practices for DOE using Minitab

by Eng. Rodrigo Picado, ASQ CSSBB, CQE,CQIA Email: rpicado@rawlings.com 2006

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

BoK Terminology

Design and analysis of one-factor experiments Design and analysis of full and fractional factorial

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

DOE Anovas and regression techniques are good for identifying significant differences between treatments and levels.

Analysis of variance help to determine the source for differences without making changes to the process.

However, Anova and regression techniques sometimes do not describe the most important process improvement activities

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

“To find out what happens to a system when you interfere with it you have to interfere with it (not just passively observe it)�

George Box(1966)

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

DOE offers a structured approach for changing many factor settings within a process at once and observing the data collectively for improvements or degradations.

These experiments can address all possible combinations of a set of input factors (full factorial) or a subset of all combinations ( a fractional factorial)

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

The need for DOE It is very common in industry even in these days to set up experiments at the nominal values and then at other values/levels using OFAT(one-at-atime) approach, which is very inneficient.

Instead, we want to understand the effect that factors had collectively on a product so changes can be made to adjust the process and reduce its variability

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Setting up a DOE Key people with knowledge in the process must be in the brainstorming session for factor determination.

Factors that are believed to be important should be included in order to have meaningful results.

Factors that are not included should be held constant or blocked An MSA must be run to assure the measurement system is reliable

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

The sequence of trial must be randomized in order to reduce the risk of an unknown occurrence to bias the accuracy of the results And finally care should be exercised to ensure there is minimal error in the measurements for each trial

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Many people think that is better to run multi-level experiements, but the truth is that this can be very expensive and complex, it is better for sure to reduce the number of levels always to two.

In order to have a very strong but economical DOE, we need to set up an experiment that can manage to 2 factor interactions, more than two factor interactions in a process is very rare and most of the time negligeble. A resolution V is a resolution where 2 factor are not alised within each other.

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Good practices for setting up a DOE 1.

2.

3.

4.

5.

6.

7.

8.

9.

10.

11.

12.

13.

14.

15.

16.

List the objectives of the experiment. It’s the experiment to understand mean effects or variability reduction opportunities?

List the process assumptions.

List the factors that might be consider.

Choose the factors and levels to consider in the experiment.

List the factors that will not be consider and will be held constant.

Reduce many-factor levels to 2-levels.

Choose the number of trials and resolution of the experiment.

Determine if one of the factor needs to be blocked.

Try to use continuous responses instead of discrete.

Determine if center points will be used for curvature check.

Determine if trials will be replicated or repeated.

Choose a sample size for the number of repetitions and replications needed.

Determine a random order sequence to use.

Determine what can be done to reduce experimental error.

Plan a confirmation and a follow-up DOE if a higher resolution is needed or other factors/levels need to be assessed.

Run the experiment

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Important considerations for variability reduction DOEs • Use the standard deviation as a response in order to identify small variation opportunities.

• If there is less than 5 repetitions it is better to use the range as a variation metric.

• It is important to transform the variation metric using natural logarithm “ln”.

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Experimentation traps 1.

2.

3.

4.

Measurement error Lack of randomization Confused effects Independently design and conducting an experiment

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

References • • • • • • •

1. Engineering Statistics Handbook. NIST Sematech.

2. MINITAB Stat Guide. MINITAB Release 14 3. Wikipedia Internet 4. Implementing Six Sigma. Smarter Solutions Using Statistical Methods. Forrest W. Breyfogle III 5. Design and Analysis of Experiments. Fifth Edition. Douglas Montgomery 6. PXS-Performance Excellence Solutions White Papers.

7. MINITAB Answers Knowledbase

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Visit Us at the WEB www.SixSigmaCostaRica.com Find application cases, e-groups, tips, and more about MINITAB 14, Quality Companion, among many other topics. www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Disclaimer • Blackberry & Cross has compiled the information in this document. The intention is to provide didactic material for MINITAB and industrial statistics enthusiasts.

Blackberry & Cross is not responsible for any situation derived from then application of the concepts contained in this document.

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735

Blackberry & Cross San Josè, Costa Rica (506) 297-4735 P.O.Box: 1372-2150 soporte@blackberrycross.com www.blackberrycross.com www.sixsigmacostarica.com

www.blackberrycross.com / soporte@blackberrycross.com/ 506-297-4735