Advanced DOE for Improving a High Performance Racecar Scott Kowalski October 25, 2007 Minitab Inc.

Design of Experiments (DOE) Basic idea is to simultaneously study the impact of several factors on the response(s) of interest Sequential Approach from screening to optimization Interactions among factors are important Surfaces can be linear or quadratic

Guidelines for DOE State the problem and clearly define the objectives of the study Choose the factors to be studied and their levels Determine the responses and how to measure them Determine the appropriate experimental design

Guidelines for DOE Execute the design Statistically analyze the data Verify results using confirmatory runs Make recommendations

Racecar Experiment Introduction Wind tunnel experiment to characterize aerodynamic performance and develop improvements Typical factors are vehicle attitude, ride height, yaw angle, and vehicle geometry Responses include lift (downforce), drag, and lift to drag ratio Goal is often to minimize drag while maintaining a specified level of downforce

Racecar Experiment Introduction The experiment was performed in the wind tunnel at Langley Air-force Base (also used extensively for aircrafts) Four factors were considered • • • •

Front end height Rear end height Yaw angle Grill cover

Racecar

CL-rear CD

CL-front

Yaw

Grille with tape

Racecar Experiment Introduction They were concerned about possible curvature in the response A previous experiment had been performed (36 runs) • Replicated 24 design with 4 center points

Problems: • When either the front end or rear end height level is changed, then the other end also changes (needs adjusting) • Only way to change height is by shutting down the wind tunnel (30-45 minutes to get back to equilibrium ) • Other factors have easy to change levels (3 minutes) • (30 hours to complete the experiment)

Racecar Experiment Introduction Can we use some other advanced DOE tools to reduce the total time of the experiment? Can we collect more overall data using an advanced tool? How does using the advanced tool affect the analysis of the data? Answer to all 3 is YES!!!

Transition We will come back to the racecar example later We need to cover some general information about the advanced design that we will use

Treatments Factors have different levels used in the experiment Treatments: the combination of the factor levels used in the experiment Example: Temperature 100, 200 Material A, B, C â€˘ Treatments are â€˘ 100, A

200, A

100, B

200, B

100,C

200,C

DOE Basics Three Principles of DOE • Randomization—randomly assign the treatments to the units of interest • Replication--assign the treatments to more than one unit • Local Control—control for known sources of variation through blocking

We will focus on all three of these in some fashion

DOE Units There are two types of units in a DOE â€˘ Experimental Unit: the smallest unit to which a treatment can be applied independently of all other treatments â€˘ Observational Unit: the unit we take measurements on

Most times these units are the same It is important that we understand that experimental error comes from variation in running the same treatment on more than one experimental unit

DOE Units Consider an example of spraying pesticides on trees: two brands and two amounts with 2 replicates

DOE Units We spray the trees, then take several leaves from each tree and count the number of bugs

Thus we have two types of units • Experimental Unit: TREE Replicate • Observational Unit: LEAF Repeat

Randomization Let’s talk about the randomization principle • Randomization is done to “average” out the effects of lurking variables • A fundamental philosophy---textbooks assume for almost all techniques that the design is randomized • Most software for DOE automatically randomizes the runs • Unfortunately, random run order often results in changes to factor settings after each run for many of the factors in the DOE • What should be done then if, one or more of these factors cannot be easily or quickly changed?

Types of Factors Actually it is common in industry to have one or more factors that are not easily randomized Examples include temperatures, pressures, prototype factors and change over factors These factors are often called Hard-to-Change (HTC) factors while the rest of the factors in the design are referred to as Easy-to-Change (ETC) factors Many people ignore the impact of the HTC factors

Split-Plot Design Split-plot design and it originated in agriculture

I1

I2 I3 I2 I1 I3

F1 F2 F3 F1 F4 F3

Experimental Units:

F4 F1 F4 F3 F1 F2

Irrigation is Column

F2 F3 F2 F4 F2 F1 F3 F4 F1 F2 F3 F4

Fertilizer is Plot

Industrial Example 1 Printing Press Blanket (image carrier)

Blanket cylinder

Cylinder gap Paper

Impression cylinder

Industrial Example 2 Baking a cake • Oven Temperature, Egg Powder, Flour, Sugar • How do we conduct the experiment? • Mix up a cake with some level of Egg Powder, Flour, Sugar then bake it at a certain temperature • Notice this will take a long time to carry out

Another idea: fix the temperature at a level, then bake all the cakes involving Egg Powder, Flour and Sugar Then change the temperature and again bake all the cakes

Baking a Cake This looks something like High Temp Low Temp

The experimental unit for Temperature is the Oven The experimental unit for other factors is a Cake Cakes are observational units for Temperature

Baking a Cake We average the observational units (repeats) to get the response for the experimental unit Therefore to get the response for Temperature at High, we would average the 8 cakes involving the different combinations of Egg Powder, Flour, Sugar Hence, we only have 2 observations for Temperature: one at High and one at Low To get an estimate of error, we would need to run the Temperature at High twice and at Low twice

Baking a Cake In addition to two different experimental units, there is two randomizations

We randomly assign the Temperature to the oven, then randomly assign the cakes within the oven

Therefore, there are two error terms â€˘ One for testing Temperature â€˘ One for testing Egg Powder, Flour, Sugar and all the interactions

Baking a Cake The final design looks like High Temp Low Temp

Blocking Why is the design is not a block design? • A block is a collection of similar experimental units • Temperature is a factor applied to the experimental units • There is interest in the interactions with Temperature • The resulting design has two errors from two kinds of experimental units

However, we will be able to take advantage of the fact that it looks like a block in order to construct the design

Baking a Cake In Minitab, we construct the subplot design in â€œblocksâ€? to get the right structure of the design Stat > DOE > Factorial > Create Factorial Design

Baking a Cake Fill in the ETC Factor Information

Baking a Cake It is common to rename blocks --- Rep To create the Temperature column â€˘ Calc > Make Patterned Data > Simple Set of Numbers

Baking a Cake The analysis involves a small trick to get the right error for Temperature since by default Minitab only has one error term Consider a simple One-Way ANOVA case The error is a nested term We use this knowledge to trick Minitab to get the correct error for Temperature

Baking a Cake Ignoring the two error terms â€˘ Use smaller error for Temp (Type I error) â€˘ Use larger error for other terms (Type II error) 1 Error

Incorrect

2 Errors

Correct

Temp

Signif.

Temp

Not

Egg

Signif.

Egg

Signif.

Flour

Signif.

Flour

Signif.

Sugar

Not

Sugar

Not

T*E

Not

T*E

Signif.

T*F

Signif.

T*F

Signif.

T*S

Not

T*S

Not

E*F

Not

E*F

Signif.

E*S

Not

E*S

Not

F*S

Not

F*S

Not

Back to the Racecar Experiment

CL-rear CD

CL-front

Yaw

Grille with tape

Racecar Experiment Recall that we have 4 factors â€˘ 2 HTC factors (front and rear heights) â€˘ 2 ETC factors (yaw and grill cover)

So a replicated 22 gives 8 runs for the heights (1 center point in the heights was included for a total of 9 runs) This means only changing the heights 9 times In each HTC run, 5 ETC combinations are carried out (22 plus one center run in Yaw and Grill Cover)

Racecar Experiment 1. randomly select the ride height factor levels of the car 2. at the factor levels from step 1, running all combinations of yaw and grille tape in random order 3. randomly selecting another ride height combination 4. again running all combinations of yaw angle and grille tape in random order 5. repeat the steps until all ride height combinations have been tested

Racecar Design Rep 1 Front RH − Rear RH −

Front RH − Rear RH +

Front RH + Rear RH −

Front RH + Rear RH +

+

+

+

+

Tape

Tape

Tape

Tape

−

−

−

− Yaw +

− Yaw +

− Yaw +

−

− Yaw +

Rep 2 Front RH − Rear RH −

− Yaw +

Front RH − Rear RH +

Front RH + Rear RH −

Front RH + Rear RH +

+

+

+

+

Tape

Tape

Tape

Tape

−

−

−

− Yaw +

− Yaw +

Front RH 0 Rear RH 0 + Tape − Yaw +

−

− Yaw +

−

Racecar Experiment This leads to a total of 45 runs But it only took about 10 hours to complete So more data, in about 1/3 of the time Wind tunnel time is very expensive so this was a huge savings Analysis is more complicated than Cake example

Racecar Experiment We have two error terms We also have center points But since we use ANOVA for the analysis, Minitab will think there are 3 distinct levels instead of 2 levels with center points We need to create a bunch of terms in the calculator (interactions and center points)

Racecar Analysis The analysis needs to be done in two stages • First is to do the HTC factor analysis using the means of the 5 ETC combinations from each of the 9 runs of the HTC factors • This gives the correct tests for the HTC factors • Second is to use a categorical factor with 5 levels (representing the 5 combinations of the HTC factors) to account for the correct SS and df from the HTC factors • Doing this gives along with the nested trick from earlier gives all the correct tests for the other terms

Racecar Coefficient Table 45-run split-plot Term

Coefficient

p-value

Constant

0.40117

FRH

0.00858

0.0000

RRH

0.00898

0.0000

FRHxRRH

0.00013

0.7247

yaw

-0.01167

0.0000

tape

-0.00494

0.0000

FRHxyaw

0.00047

0.0910

FRHxtape

-0.00016

0.5640

RRHxyaw

-0.00047

0.0910

RRHxtape

0.00078

0.0070

tapexyaw

-0.00056

0.0360

ssq

0.00058

0.0480

wsq

0.00031

0.5592

Summary DOE is a great tool for learning about and optimizing products/processes Many applications of DOE involve HTC factors Using a split-plot design saves time and money Analysis is more complicated MINITAB-16 will have 2-level split-plot designs

References Montgomery DC. Design and Analysis of Experiments, 6th ed., John J. Wiley & Sons, New York, 2004. Kowalski, S. M.; Parker, P. A.; and Vining, G. G. (2007). “Tutorial on Split-Plot Experiments”. Quality Engineering 19, pp. 1-16. Simpson, J. R.; Kowalski, S. M.; and Landman, D. (2004). “Experimentation With Randomization Restrictions: Targeting Practical Implementation”. Quality and Reliability Engineering International 20(5), pp. 481-495.

GRACIAS