Design of Experiments (DOE) can be used to create an experimental design and then analyze the responses to develop a model between the input(s) and the output. There are several types of design one can use to perform the experiments. The following flowchart shows the types of experiments you can run using Sigma Magic software.
It is recommended that you run a Full Factorial design for up to 6 factors and run a Fractional Factorial design if you have more than 6 factors. It is a good idea to add center points to check if your model is linear. If your model is not linear, you can consider running a General Factorial design with more than 2 levels or create a response surface design. You can also run the experiments sequentially. First, you run the Fractional Factorial design with a large number of factors and narrow the list of factors to the most important ones. Next, you can run a Full Factorial design with center points added. Based on the analysis results, you can choose to run either a General Factorial or a Response Surface design.
This tool can be added to your active workbook by clicking on Stats and then selecting Design of Experiments > Factorial Design.
Inputs
Click on Analysis Setup to open the menu options for this tool.
Create Design
A sample screenshot of the create design menu is shown below.
1
Num Factors:
You can analyze up to 15 factors in this software.
2
Design Type:
Specify the type of design you want to create. The following options are available:
Option
Description
Full Factorial
Create a Full Factorial design. It is recommended that you use a Full Factorial design for up to six factors.
Fractional Factorial
Create a Fractional Factorial design. Use a Fractional Factorial design if you have more than six factors.
General Factorial
Use a General Factorial design when you have more than 2 levels for one or more factors. Typically used for models with text factors.
Response Surface
Use a response surface design if you want to build a non-linear model with continuous factors.
3
Center Points:
Specify the number of center points you want to add to your model. If you specify "None" then no center points are added and only a linear model is developed. For Response surface designs, select the number of center points to "Auto" for the system to automatically determine the number of center points required. However, you can feel free to change this number based on your needs. Center points can help you check for model linearity and increase the power of your designed experiments.
4
Axial Points:
The axial points option can only be specified for the Response Surface Methodology designs. The two options available here are Central Composite Design (CCD) and Face Centered Design (FCD). If you choose the CCD design, the system will automatically calculate the alpha value for you. However, you can feel free to change this value based on your specific needs. If you choose the FCD design, the alpha value defaults to 1.0.
5
Replicates:
It is a good idea to replicate the design so that you can get an estimate of the experimental error. Replication helps you improve the power of your experimental design and calculate the P values for each term so that you can determine model significance. Increasing the number of replicates comes with the cost of having to perform a greater number of tests which can add to the time and cost required for performing the designed experiment.
6
Blocks:
Known noise factors should be blocked, and the entire design should be randomized to handle unknown noise factors. If you have specific noise factors that you feel could impact your experimental design, then you could block out this factor from impacting your experimental results. Note that currently, you can only block on the replicates.
7
Randomize:
Specify if you want to randomize your runs. The purpose of randomization is to handle unknown noise factors.
8
Variable Name:
Specify a name for each of your variables. Make sure the names are unique and don't involve special characters.
9
Low Level:
Specify the low level for each factor. If you use numeric values for the levels, Sigma Magic software will assume that the factor is numeric. If you use non-numeric (text) levels for either low or high, then Sigma Magic will assume that the factor is text. Note that for Response surface designs, the levels must be numeric. However, for the other designs, the factors could be either text or numeric. Make sure that the factor levels and levels are unique for each factor.
10
High Level:
Specify the high level for each factor. If you use numeric values for the levels, Sigma Magic software will assume that the factor is numeric. If you use non-numeric (text) levels for either low or high, then Sigma Magic will assume that the factor is text. Note that for Response surface designs, the levels must be numeric. However, for the other designs, the factors could be either text or numeric. Make sure that the factor levels and levels are unique for each factor.
11
Flowchart:
Click on this button to view the flowchart for Design of Experiments.
12
Help Button:
Click on this button to open the help file for this topic.
13
Create Design:
Click on the checkbox "Create design & save to worksheet" to create the design and save the design to the worksheet. Once the design is created and you have entered the response data on the worksheet, change the checkbox to "Analyze design stored in the worksheet" to generate analysis results.
14
Cancel Button:
Click on this button to cancel all changes to the settings and exit this dialog box.
15
OK Button:
Click on this button to save all changes and compute the outputs for this analysis.
A sample worksheet of the DOE design is shown in the figure below.
Once you enter the inputs and press okay, the DOE design is created in the worksheet. If your dialog box tab is on "Create Design". Once you create the run, you would need to conduct each of these tests in random order and then upload the results in the response column. If you want to build a robust model that minimizes the impact of variation on the design, then you may consider repeating each test and then calculating the mean and standard deviation of the numbers. You can then enter either the mean or the standard deviation of the data you have collected in the response column.
When all the responses have been entered in the worksheet, click on Analysis Setup and then click on Analyze Design tab to generate the results.
Analyze Design
A sample screenshot of the analyze design menu is shown below.
1
Terms:
Specify how you want to perform the analysis. You will need to select the option for the model terms. You have the following options:
Option
Description
Auto
The software will start off with all the terms in the model and progressively keep dropping terms until only the significant terms remain in the odel.
All Terms
The software will include all the available terms in the model for analysis. It will not drop any terms. You will have to manually perform model reduction if required.
Up to 1st Order
The software will only estimate the main effects.
Up to 2nd Order
The software will only estimate the main and 2nd order interactions.
Manual
You can specify which terms to include in the model. To select the terms either double click on the available terms to select the term for analysis
or single click on the desired term in the list of available terms and click on the right arrow to select the term. If you want to remove an already selected term, just select the term that you want to delete and then click on the left click arrow.
2
Available Terms:
This listbox lists all the terms available in your model to pick for analysis. Note that the factor names are represented in the short form (A, B, C) etc. to keep it easy to understand.
3a
Select All Terms:
Click on this button to move all the available terms to the terms included for analysis which is shown on the listbox on the right. This is equivalent to using all terms for your analysis.
3b
Deselect Term:
Click on this button to remove the selected term from the Terms Included column. You can use this to drop any terms from your model that you are no longer interested in.
3c
Select Term:
Click on this button to move the term that has been selected in the list of available terms to the included terms on the right. You can use this to add new terms to your model for analysis.
3d
Deselect All Terms:
Click on this button to remove all terms from the included terms listbox. You will have to start from scratch and add terms that you are interested in later.
4
Terms Included:
This listbox lists all the terms that will be used to build your model.
5
Blocks:
Click on this checkbox to include blocks in your analysis. You need to initially include blocks in your analysis and if it is not statistically significant, you can drop the block terms.
6
Center Points:
Click on this checkbox to include center points in your analysis. You need to initially include center points in your analysis and if it is not statistically significant, you can drop the center point terms.
Do note that not all terms you include in the included terms may be included by the software analysis results. For example, if you are performing a Fractional Factorial design, some terms may be confounded with others. In which case, each term cannot be independently estimated by the model. For General Factorial designs, fourth-order and higher interactions are not included in the analysis. When you click on OK the software will analyze the DOE and compute the outputs and plot any graphs as specified.
Plot Results
A sample screenshot of the plot results menu is shown below.
Make sure that the tab is on Plot Results if you want to update analysis charts. Do note that when you use the Plot Results tab, the analysis results are not regenerated. The software will use previous analyses that you have run to generate the plots. If you make any changes to your settings, make sure to keep your tab on Analysis Results and click on OK button to update results.
If you would like to look for an optimal target value in the results, specify the optional target value in the dialog box. This value will be used to plot a horizontal line on the main effects plot. The intersection of the horizontal line along with the main effects plot will give an idea of value of the input variables that can result in this value of the output. The target value is also used to fashion the contours so that we can easily look for the target values in the contour and/or the surface plots.
1
Main Effect:
Click on the checkbox if you would like to plot the main effects plot. Click on the Setup button to setup the main effects plot. Under the dropdown box for Select plots, specify which main effects you want to plot. The following options are available:
Option
Description
None
No main effect plots are generated.
All
All possible combinations of main effect plots are generated
Significant
Only significant main effect plots are generated. Note that this requires that you have already analyzed the DOE. It will use results from the last updated DOE run.
A, B, ...
Only the factor you select here will be plotted.
Under the graph type and specify the type of plots to generate. The available options are:
Option
Description
Single
Plot all the main effects on a single plot. This makes it easier to compare multiple plots.
Multiple
Plot each main effects plot on a separate plot.
2
Interaction Effect:
Click on the checkbox if you would like to plot the interaction effects plot. Click on the Setup button to setup the interaction effects plot. Specify which interaction effects you want to plot. The following options are available:
Option
Description
None
No interaction effect plots are generated.
All
All possible combinations of interaction effect plots are generated
Significant
Only significant main effect plots are generated. Note that this requires that you have already analyzed the DOE. It will use results from the last updated DOE run.
Click on the graph type and specify the type of plot to generate. The available options are:
Option
Description
Single
Plot all the main effects on a single plot. This makes it easier to compare multiple plots.
Multiple
Plot each main effects plot on a separate plot.
Click on dropdown box for the X-axis variable to specify which variable you want to use for the X-axis. This option is only available if you are plotting a single interactione effects plot. The following options are available:
Option
Description
Auto
The software will automatically pick the X-axis for you (usually the first variable).
A, B, ...
The factor you specify here will be used as the X-axis.
3
Contour or Surface Plot:
Click on the checkbox if you would like to plot the contour or surface plot. Click on the Setup button to setup the contour plot. Specify which interaction effects you want to plot. The following options are available:
Option
Description
None
No contour plots are generated.
AB, AC, ...
Select the interaction effect for which you want to generate the contour plot. Note that both variables have to continuous for this plot.
Click the variable graph type to specify the type of plot to generate. The available options are:
Option
Description
Contour
Plot the data as a contour plot.
Surface
Plot the data as a surface plot.
Click on the X-axis factor setting to specify the factor use for the X-axis. By default, the first variable is chosen for the X-axis. Next, specify the levels to be used for the other factors. The following options are available:
Option
Description
Low
All the remaining variables will be set at their low settings.
High
All the remaining variables will be set at their high settings.
Custom
You can specify the settings to use for the remaining variables.
Next, specify how you want to enter the other settings for the contour variable. This setting only is applicable when you pick the "Custom" option for the previous selection. The following options are available:
Option
Description
Coded
You want to enter the settings in the coded space (-1 to 1).
Uncoded
You want to enter the settings in the uncoded space (example: 60 to 80).
4
Other Plots:
Click on the checkbox if you would like to generate the other plots - Pareto effects plot and the residual plots. Click on the Setup button to setup these plots. Under the Pareto plot and the Residual plots, specify if you would like to show or hide these plots. The available options are:
Option
Description
Show
Plot the Pareto and/or the Residual plots.
Hide
Do not plot the Pareto and/or the Residual plots.
Outputs
When all inputs have been entered, click on OK to generate the outputs. Click on Compute Outputs to update the output calculations if required. A sample screenshot of the outputs is shown in the figure below.
In the notes section, the analysis first summarizes the inputs specified for the analysis under the section Input Summary. Check this to make sure all your inputs are correctly specified. Make sure that the variables' names, variable levels are as intended. The ANOVA table lists the overall ANOVA of the model and the P values. This P-value is used to determine if we have a statistically significant model or not. The R^2 and R^2(adj) of the model are also listed here which can be used to get an idea of the goodness of fit of the model to the experiments. Finally, the model coefficients are shown in the coded space. Check which model terms are significant and which ones are not. The Variance Inflation Factors (VIF) are also listed for each model. If these values are less than 5, then we are okay - not a cause for worry. You can drop the non-significant terms and redo the analysis if required until a statically significant model remains. The model equation in the uncoded space is listed below which can be used for predictions.
The graphs section will show the graphs for this analysis as requested. The Pareto chart of significant effects will highlight those effects that are significant by plotting a critical limit line in red. Any effects that are greater than this red line are considered to be significant. Note that if you have an un-replicated design there it is not possible to determine the P values, the critical red line is based on Lenth's formula to determine the PSE along with a T-factor of 2.0. If you have selected the option of only showing significant graphs, then the graph is plotted only if the term is significant. Interaction plots and surface plots are shown if you have requested these plots in your input options.
The analysis will also print out a histogram of the residuals along with the residuals plotted for the run order and residuals plotted with respect to the fit. Make sure that the residuals are normally distributed from the histogram and check to make sure that the residuals with the run order and fit are randomly distributed. If not, there may be a problem with your modeling assumptions and you will not be able to rely on the developed model.
If we were analyzing a Fractional Factorial design, then a sample screenshot of the output is shown below. The analysis notes and graphs are similar to the above Full Factorial design and the only difference is that not all terms are estimated since some terms are confounded with others due to aliasing.
If we were analyzing a General Factorial design, then a sample screenshot of the output is shown below. The analysis results are similar to the one shown for the Full Factorial design except that there is no model equation shown.
If we were analyzing a Response Surface sign, then a sample screenshot of the output is shown below. The analysis results are similar to the Full Factorial design except that now we can have quadratic factors in our model.
Notes
Here are a few pointers regarding this analysis:
You are currently limited to 15 factors and 10 levels for each factor in this analysis.
Using the Auto method for analysis will use the backward model reduction approach. It will first develop a full model with all the factors and then keep dropping factors from the bottom-up which are not statistically significant until only a statistically significant model remains.
Using the plot results tab will not recompute the analysis results but will use the results from the last performed analysis to update the graphs.
The plot of the histogram of the model residuals is currently not available for the General Factorial but is displayed for the other types of designs when you compute the analysis results (not when updating the graphs).
Examples
Following examples can be found in the Examples folder.
Create a 2-level factorial design for 3 factors with no blocks and no center points. The data for the exercise is given in the file. (DOE 1.xlsm).
Analyze the 2-level factorial design based on the results given in the data file. Plot the significant main and interaction effects. (DOE 1.xlsm).