Technical FAQ's

What is DOE?
DOE stands for "Design Of Experiments." DOE helps you to design an experiment that will provide you with the most information possible with the least amount of work.

Everyone wants to perform experiments that provide a lot of valuable information with as little work as possible. So why doesn't everyone use DOE?

Many people have never heard of it. Many people who have heard of it don't really understand it.

You can learn DOE in a fun, informative, non-mathematical workshop.

Learn more about the workshops.


What model should I use?
Choosing a model doesn't have to be difficult. Just remember that you will almost certainly not choose the "best" possible model. All you really need is a model that is good enough to lead you to your Sweet Spot.

With that in mind, let your budget guide you to your model.

If your budget is too tight to study all of your factors, you can screen out the factors least likely to be important. When your goal is to screen factors, choose a Main Effects model. The Main Effects model requires the fewest experimental trials. The trade-off is that it will provide you with the least information.

If your budget is tight, but you can study all of the factors, choose an interaction model. The interaction model will provide you with information about factor interactions as well as main effects. If it doesn't predict well enough, you can always augment it with more data to fit a full quadratic model.

If you can afford it, choose a full quadratic model at the outset. This model will provide information about main effects, interactions, and domes and basins. If you choose an I-Optimal design you can collect data to fit a quadratic model with little more work than you need to fit an interaction model.


What design should I use?
Your choice of design is largely dictated by your choice of model.

If you have chosen a Main Effects model, you need to use a screening design. Two important types of screening designs are Plackett-Burman designs and D-Optimal designs.

If you have chosen an Interaction model, you will need an interaction design. Important types of interaction designs are Factorial and Fractional Factorial designs.

If you have chosen a Full Quadratic model, you will need a quadratic design. Important quadratic designs are the Central Composite, the Face Centered Cubic, and I-Optimal designs.

If you have chosen another model, you will need a custom design to fit your model.


I've collected all of my data, but I don't remember which model to use. What can I do?
Since models and designs are chosen to work together, you can often work backwards from your design to figure out which model you originally chose.

Do you recognize the design? If so, you can use these guidelines to determine the model:

Main Effects designs (Plackett-Burman or D-Optimal) fit Main Effects models.

Interaction designs (Factorial or Fractional Factorial) fit Interaction models.

Quadratic designs (Central Composite, Face Centered Cubic, and I-Optimal) fit Full Quadratic models.

If you don't recognize the design, try this: Try to analyze your data with a a Full Quadratic model. If you get an error, try an interaction model. If you get an error, try a Main Effects model. If that still doesn't work, call for help!


I get a message saying X'X is singular. What's wrong?
When you see this error, it means you don't have enough data for the model you have chosen. Perhaps you originally chose a simpler model and forgot. Maybe you were unable to collect data for every trial in your design. Perhaps some data was entered incorrectly.

To eliminate this error, you will have to use a simpler model or collect more data. You can also call for help.


I get a message saying that the log condition number is greater than 15. What's wrong?
When you see this error, it means you don't have enough data for the model you have chosen. Perhaps you originally chose a simpler model and forgot. Maybe you were unable to collect data for every trial in your design. Perhaps some data was entered incorrectly.

To eliminate this error, you will have to use a simpler model or collect more data. You can also call for help.


What is an Optimal Design?
The simplest way to select trials for an experiment is to spread them out by placing them on corners, centers of faces etc. Optimal designs use a different method for spreading the points out that can result in more efficient experiments.

First, an "optimal" property must be chosen. D-Optimal designs, for instance, focus on finding the best b-coefficients for a model. I-Optimal designs focus on finding the best predictions from a model.

Second, this optimality must be stated mathematically. D-Optimal designs minimize the determinant of the variance-covariance matrix. I-Optimal designs minimize the integrated variance for the region of interest.

Finally, experimental trials can be chosen to satisfy the appropriate mathematical criterion.

This method can be very complicated and is generally executed by a computer. Gosset is a program that generates optimal designs on a routine basis.

Optimal designs do not have their trials restricted to corners, face centers, etc, so they may look quite different from standard designs.


What is an I-Optimal Design?
I-Optimal designs have trials selected to produce models that will provide the best predictions. Unlike standard designs, I-optimal designs are not confined to corners, centers of faces, etc. This allows I-Optimal designs to be more efficient than their standard counterparts.

"I" stands for "Integrated variance." I-Optimal designs provide the lowest integrated or average variance for a design.


What is a D-Optimal Design?
D-Optimal designs have trials selected to produce models that will provide the best estimates of the effects or b-coefficients. D-Optimal designs are best used when screening factors.

"D" stands for "Determinant." D-Optimal designs provide the lowest determinant for the variance-covariance matrix for the b-coefficients.


What are the differences among various designs?
The principle behind good experiment designs is that the trials are well-spread-out from each other. The way the trials are spread out and the model to be fitted create different types of designs.

Factorial Designs: These are probably the best known designs. They are intended to fit interaction models. The trials are spread out geometrically -- they are placed at all of the cube or hypercube corners.

Fractional Factorial Designs: These designs are intended to fit interaction models with the higher order interactions ignored. They typically fit all of the 2-factor interactions. The trials are placed at corners, but not all of the corners are used, producing more efficient designs.

Central Composite Designs: Central Composite designs are intended to fit full quadratic models. They place their trials at corners, the center point, and on a sphere enclosing the cube above the centers of faces.

Face-Centered Cubic Designs: FCC designs are intended to fit full quadratic models. These designs place the trials at the corners, center, and centers of faces.

Uniform Shell Designs: Uniform Shell designs are intended to fit full quadratic models. They place the trials on the surface of a sphere. Because they aren't tied to cube corners, they require fewer trials than Central Composite or Face-Centered Cubic designs.

D-Optimal Designs: D-Optimal designs spread the trials to get the best estimates of the effects or b-coefficients. They can be generated for any model. They are best used for screening with Main Effects models.

I-Optimal Designs: I-Optimal designs spread the trials in an attempt to provide the best predictions. They can be generated for any model. They can have the smallest number of trials possible for any selected model. I-Optimal designs have many properties that make them excellent choices for industrial experimentation.


How do I get my b-coefficients into Excel?
From STATISTICA:

1. Create a table with the b-coefficients. In STATISTICA 6.0 you can do this by selecting the ANOVA/Effects tab and clicking the Regression Coefficients button.

2. Highlight the b coefficients. Choose Edit \ Copy.

3. Paste them into your Excel Spreadsheet.

From STRATEGY: (Thank you Ed Vawter!)

1. Run your regression.

2. Find the b-coefficients table and highlight it all.

3. Choose Edit \ Copy.

4. Paste into your Excel Spreadsheet.

5. in Excel, choose Data \ Text to Columns... \ Select Fixed width


Why doesn't my software recognize my replicates?
DOE software looks at all of the decimal places of a trial level. If even one digit is different, or missing, the software will count it as a different trial. To avoid this problem, cut and paste your replicates rather than typing them in.


Why do I need replicates?
All measurements vary. If you do not see the variation, your measurement device is not sensitive enough for your work. You need to collect replicate measurements to determine the amount of variation in your measurements. Without repeated measurements (extra replicates) you can't know your uncertainty, and so you can't know the risk you are taking.


How many replicates do I need?
I recommend collecting enough replicates to provide you with 5 degrees of freedom on your pooled standard deviation. Dave Doehlert recommended collecting enough to provide 8 degrees of freedom. This means you should repeat 5 or 8 of your trials.

Five extra replicates is often enough to judge whether a response transformation is required. In those cases when it is not enough, you can return to the lab and measure 3 more replicates.


What is blocking?
Blocking is a technique used to eliminate the effects of "nuisance factors," factors that are of no interest to you but may affect your responses. For example, you may want to use two different ovens in your experiment. The two ovens are not identical, but the differences are simply a nuisance to you -- you are not studying the ovens. You can run two blocks of carefully selected experiments, one on the first oven and the other on the second. Doing so will prevent the nuisance factor from misleading you.


I've lost some of my data. What can I do?
The best thing to do is to return to the lab and collect the missing data.

Sometimes this is not possible. The first thing to do is try to analyze the data you do have. If you don't get an error, take a look either at the correlation matrix of effects or the VIF's. If they look OK (no off diagonal elements > 0.95 nor < -0.95 in the matrix, or no VIF greater than 5) you got lucky. You should be able to proceed with your analysis.

If you get an error, your next step will depend on the reason you can't collect the data.

If you can't run the trial requested by the design, move the trial until you can collect data. Move it as little as possible. Replace the requested trial with this one and analyze as usual.

If it is economically unfeasible to collect more data, you may have to settle for a simpler model. For example, if you wanted to fit a full quadratic model, you might have to settle for fitting an interaction model.

Never make up data to fill in the blanks. Don't use averages, estimates, etc. Doing poses a great risk of misleading yourself as to the effects of factors and of missing the Sweet Spot.


I couldn't run exact replicates. What should I do?
DOE analysis assumes that the variations observed are entirely due to response variation. This is never true -- everything varies. When you set factor levels, the settings vary from time to time. The assumption that all variation comes from responses is close enough in general that you still find the right answer.

Your best bet is to use the average levels for the factors as the trial that was replicated. This forces the software to make the assumption that all variation was in the responses.

If you find this unsatisfactory, there are more complex methods of regression that take variations in the factors into account.


I can't find a design that fits my experiment. What can I do?
Take a look at the I-Optimal Design Library. It has many designs that are not commonly available.

You can learn to use Gosset. Gosset is a general purpose program for generating experiment designs. It allows you to create a design tailored to your specific requirements.

You can hire Math Options to generate a custom design for you. This frees you from the complications of doing it yourself.

Call (866) 683-6173 for pricing.

You can visit http://www.WebDOE.com. WebDOE is a free online service that allows you to generate your own custom experiment designs for up to 3 factors.


What is Gosset?
Gosset is the consummate program for generating Optimal experiment designs.

Gosset allows you to specify various factor types, including process, mixture, discrete, and restricted, in the same experiment.

It allows you to specify constraints on your experiment -- limitations that would prevent some trial from running. This allows you to create designs that don't ask you to run trials that are impossible.

It allows you to augment existing data. Tell Gosset the trials you already have and it will tell you what you need to run to turn this data into a good experiment design.

It allows you to specify designs in blocks to eliminate the effects of nuisance factors.

Gosset allows you to specify the Optimality of your design.

You can learn to use Gosset in your work in the workshop, Creating Custom Experiment Designs.


What is a constraint?
A constraint is a limitation on an experiment.

For example, suppose you want to find the best combination of ingredients for a cooling solution. The sum of the ingredients must add to 100%. This limitation, or constraint, makes standard designs useless. Standard designs will ask for 100% of each ingredient, for instance. This constraint requires a special design -- a mixture design. Mixture designs have the levels for each trial add to 100%.


What is a mixture factor?
A mixture factor is a factor subject to the mixture constraint. This constraint requires that all ingredients in a mixture must add to 100%.

You can learn to experiment with mixtures in the workshop, Objective Experiments for Mixtures and Discrete Factors.

How can I study discrete factors?
Discrete factors can pose many challenges. Simple problems with a 2-level discrete factor, such as light -- ON or OFF -- can be handled using an interaction design. More complex factors, such as material -- metal, plastic, wood, cotton, or nylon -- are most easily treated as a mixture of their levels with the added constraint that each component of the mixture (discrete level) must be either all or nothing.

Because discrete factors are treated as mixtures, it is important to understand mixtures when working with discrete factors.

You can learn to experiment with discrete factors in the workshop, Objective Experiments for Mixtures and Discrete Factors.


Does DOE really work or is it just the latest fad?
DOE has been in use since the 1920's. While it was the province of Statisticians it successfully solved problems in industry for decades. Now that modern computers and software have brought DOE into the province of Engineers and Scientists it is being used to solve problems on an even wider scale.

DOE is not just a fad -- it is a time-proven, powerful technique.

Call Paula Bartolomei at (866) 683-6173 to find out how.