Sigma Magic Help Version 17

Compare Samples

Overview

Compare Samples can be used to perform hypothesis testing analysis in cases where the output Y is continuous or discrete, and the input X(s) is discrete. The analysis automatically determines the right test to use and applies that test to the input data points.

If the primary metric, Y data, is continuous and if there is only one data set that needs to be compared to an external standard, then the analysis either applies a 1-sample t-test if the data is normal or a 1-sample sign test or a 1-sample Wilcoxon test if the data is not normal. If both the data sets are normal, then a 2-sample t-test is used. If they are not normal, then a Mann-Whitney test or the Kruskal-Wallis test is used. If tests are conducted in a paired fashion, then a paired test is used. If there are more than 2 data sets and the data sets are normal, then ANOVA analysis is used; otherwise, a Mood's Median test is performed. A flowchart of the different methodologies is shown in the figure below.

The flowchart below shows the flowchart for comparing the means of continuous data. The flowchart below shows the flowchart for comparing the standard deviations of continuous data. If the primary metric, Y data, is discrete and we have one or two sets of data to compare, then if we are working with defects, the Poisson rate tests can be selected; otherwise, if we are working with defectives, then the Proportion tests can be selected. If there are more than 2 data sets, the Chi-Square tests can be selected.

The flowchart below shows the flowchart for comparing the defects for discrete data. This tool can be added to your active workbook by clicking on Stats and then selecting Compare Samples.

Analysis setup

Click on Analysis Setup to open the menu options for this tool.

Setup

A sample screenshot of the setup menu for continuous data is shown below.

1	Data Type: Specify the type of data you are dealing with. The available options are: Option Description Continuous Continuous data refers to measurements on a continuum or scale that can be meaningfully subdivided into infinitely small increments, depending on the precision of the measurement system. Examples of continuous measurements are time, temperature, weight, currency, etc. Defects Defects are any item or service that exhibits a departure from specifications. A defect does not necessarily mean the product or service cannot be used. Defectives Defectives refer to the entire product or service and refer to the condition that the product or service is not usable. A product may have many defects - not all of these defects may cause the product to be defective.
2	Comparison: Specify the type of comparison you want to make. Option Description Mean or Median Continuous data refers to measurements on a continuum or scale that can be meaningfully subdivided into infinitely small increments, depending on the precision of the measurement system. Examples of continuous measurements are time, temperature, weight, currency, etc. Std Dev Defects are any item or service that exhibits a departure from specifications. A defect does not necessarily mean the product or service cannot be used. Proportion Defectives refer to the entire product or service and refer to the condition that the product or service is not usable. A product may have many defects - not all of these defects may cause the product to be defective.
3	Num Samples: Specify how many data sets you are trying to compare. This is typically the number of X variables you have in your analysis. For example, if you are trying to compare the mean value delivery times of Supplier A and Supplier B, then the number of data sets is two as we compare two data sets.
4	Hypothesis Test: Specify the hypothesis test that you want to perform for your analysis. You can leave this setting at Auto, and the software will pick the appropriate hypothesis test for your case. Use the flowchart to determine which would be the best test to use for your analysis.
5	Null Hypothesis: Specify the null hypothesis. If you compare one data set with an external standard, the null hypothesis value should be the external standard. Note that for proportions, the null hypothesis should be between 0 and 1. If you are comparing multiple sets of data with each other, enter a value of 0 to check if the mean or proportion is equal, and enter a value of 1 to check if the standard deviations are equal.
6	Alt Hypothesis: Specify the alternate hypothesis (less than, greater than, or not equal). The default setting is Not Equal. The software will perform a one-sided hypothesis test if you select either less than or greater than.
7	Hypothesis Statements: The hypothesis test that you are performing is shown as a hypothesis statement. Ensure you closely read this statement to ensure this is the test you want to perform. If not, make appropriate changes to the dialog box to reflect the test you want to perform. The hypothesis statement assumes some default settings for the null and alternative hypotheses. If you want to change these settings, you must go to the Hypothesis tab.
8	Flowchart: Click on this button to open and view the flowchart for this tool. You can also click and choose the test you want to perform directly from this flowchart.
9	Help Button: Click on this button to view the help files for this topic.
10	Cancel Button: Click on this button to cancel your changes and exit from changing the settings.
11	OK Button: Click on this button to save your settings and try to compute the analysis results. Note that your analysis results will not be computed if all the required data are not specified.

A sample screenshot of the setup menu for discrete data is shown below.
Most of the menu options are similar to the continuous case. The two changes you will find are highlighted below:

1	Data Type: It would be best if you changed the data type to defects or defectives for discrete data.
2	Comparison: For discrete data, the comparison changes to proportion. This refers to comparing the proportion of defects or defectives between different data sets.

Options

A sample screenshot of the options menu for continuous data is shown below. The menu options for the discrete case are very similar as well.

1	Input Data: Specify the format for your input data. If you need to change your input data format, click the Format Data button. Option Description Group Data Group data means that there are two columns with group description in one column and the data in another column. Enter the data column under the box labeled Analysis Variables and the group column under the box labeled Categorical Variables. Raw Data Raw data implies that each data set is in a separate column. Enter the columns you want to analyze in the box labeled Analysis Variables. Summarized Data Summarized data refers to properties of the raw data such as mean, standard deviation, etc. Enter the summarized data in the appropriate text boxes in the Data tab.
2	Delta: For this analysis, you must specify the difference that is practically important for you. For example, if you compare the time it takes to repair TV sets, you may be interested in detecting any difference greater than 0.5 hours. If you leave the delta value blank, the software will not compute the minimum sample size required for this analysis.
3	Confidence Level: Enter the confidence level required for your analysis. This controls your Type I or Alpha error (1 - Confidence Level). The default value for this is 95%.
4	Power: Specify the Power required for your test. This value controls your Type II or Beta error (1 - Power). The default value is 90%.

Data

You will see the following dialog box if you click the Data button. Here, you can specify the data required for this analysis.

1	Search Data: The available data displays all the columns of data that are available for analysis. You can use the search bar to filter this list and speed up the search for the right data for analysis. Enter a few characters in the search field, and the software will filter and display the filtered data in the Available Data box.
2	Available Data: The available data box contains the list of data available for analysis. If your workbook has no data in tabular format, this box will display "No Data Found." The information displayed in this box includes the row number, whether the data is Numeric (N) or Text (T), and the name of the column variable. Note that the software displays data from all the tables in the current workbook. Even though data within the same table have unique column names, columns across different tables can have similar names. Hence, it is crucial that you not only specify the column name but also the table name.
3	Add or View Data: Click on this button to add more data to your workbook for analysis or to view more details about the data listed in the available data box. This button opens the Data Editor dialog box, where you can import more data into your workbook. You can also switch from the list view to a table view to see the individual data values for each column.
4	Required Data: The code for the required data specifies what data can be specified for that box. An example code is N: 2-4. If the code starts with an N, select only numeric columns. You can choose numeric and text columns if the code begins with a T. The numbers to the right of the colon specify the min-max values. For example, if the min-max values are 2-4, you must select a minimum of 2 columns of data and a maximum of 4 columns in this box. If the minimum value is 0, then no data is required to be specified for this box.
5	Select Button: Click on this button to select the data for analysis. Any data you choose for the analysis is moved to the right. To select a column, click on the columns in the Available Databox to highlight them and then click on the Select Button. A second method to choose the data is to double-click on the columns in the list of Available Data. Finally, you can drag and drop the columns you are interested in by holding down the select columns using your left mouse key and dragging and dropping them in one of the boxes on the right.
6	Selected Data: The list box header will be black if the right number of data columns is specified. If sufficient data has not been specified, then the list box header will be displayed in red color. Note that you can double-click on any of the columns in this box to remove them from the box. The data you specify for this analysis depends on the options in the Setup tab. Option Description 1 If you want to analyze a single column of data, you can specify that column under Analysis Variables and leave the Grouping Variable blank. 2 If you want to analyze multiple columns of data, you can specify the multiple columns under Analysis Variables and leave the Grouping Variable blank. 3 If you are working with grouped data, then you enter the variable data under the Analysis Variables and the column that contains grouping information under Grouping Variables.
7	View Selection: Click on this button to view the data specified for this analysis. The data can be viewed in a tabular format or a graphical summary.

If you have specified the input data format as summarized data, your data tab will look similar to the following dialog box: For summarized data, you can specify a data file for entry or type the values within the dialog box.

1	Summarized Data: You must specify the average, standard deviation, and number of data points for continuous data. For discrete defects or defectives data, you must specify the count of the number of defects or defectives and the total number of samples tested.
2	Load from File: Click on Load from Table and specify the data columns that contain the data you are interested in. The software will fill up your summarized data table based on the specified data. Note that you can always edit and overwrite the values on the screen, which will be used for analysis.

Click on OK to save the dialog box options and compute the analysis results.

Charts

You will see the following dialog box if you click the Charts button.

0	Pick Charts: Select the charts you would like to display for this analysis.
1	Title: The system will automatically pick a title for your chart. However, if you want to override that with your title, you can specify a title for your chart here. Note that this input is optional.
2	Sub Title: The system will automatically pick a subtitle for your chart. However, if you want to override that with your subtitle, specify a subtitle for your chart here. Note that this input is optional.
3	X Label: The system will automatically pick a label for the x-axis. However, if you would like to override that with your label for the x-axis, you can specify a different label here. Note that this input is optional.
4	Y Label: The system will automatically pick a label for the y-axis. However, if you would like to override that with your label for the y-axis, you can specify a different label here. Note that this input is optional.
5	X Axis: The system will automatically pick a scale for the x-axis. However, if you would like to override that with your values for the x-axis, you can specify them here. The format for this input is to specify the minimum, increment, and maximum values separated by a semi-colon. For example, if you specify 10;20, the minimum x-axis scale is set at 10, and the maximum x-axis scale is set at 20. If you specify 10;2;20, then, in addition to minimum and maximum values, the x-axis increment is set at 2. Note that this input is currently disabled, and you cannot change this setting.
6	Y Axis: The system will automatically pick a scale for the y-axis. However, if you would like to override that with your values for the y-axis, you can specify them here. The format for this input is to specify the minimum, increment, and maximum values separated by a semi-colon. For example, if you specify 10;20, the minimum y-axis scale is set at 10 and the maximum y-axis scale is set at 20. If you specify 10;2;20, then, in addition to minimum and maximum values, the y-axis increment is set at 2. Note that this input is optional.
7	Horizontal Lines: You can specify the values here to add a few extra horizontal reference lines to your chart. The format for this input is numeric values separated by semi-colon. For example, if you specify 12;15, two horizontal lines are plotted at Y = 12 and Y = 15, respectively. Note that this input is optional.
8	Vertical Lines: You can specify the values here if you want to add a few extra vertical reference lines on top of your chart. The format for this input is numeric values separated by a semi-colon. For example, if you specify 2;5, two vertical lines are plotted at X = 2 and X = 5, respectively. Note that this input is optional.

Verify

If you click the Verify button, the software will perform some checks on the data you entered. A sample screenshot of the dialog box is shown in the figure below. The software checks if you have correctly specified the input options and entered the required data on the worksheet. The results of the analysis checks are listed on the right. If the checks are passed, they are shown as green-colored checkmarks. If the verification checks fail, they are shown as a red-colored cross. The verification checks are shown in the orange exclamation mark if the verification checks result in a warning. Finally, any checks required to be performed by the user are shown as blue info icons.

1	Item: The left-hand side shows the major tabs and the items checked within each section
2	Status: The right-hand side shows the status of the checks.
3	Overall Status: The overall status of all the checks for the given analysis is shown here. The overall status check shows a green thumps-up sign if everything is okay and a red thumps-down sign if any checks have not passed. Note that you cannot proceed with generating analysis results for some analyses if the overall status is not okay.

Outputs

Click on Compute Outputs to update the output calculations. A sample screenshot of the worksheet is shown below.

1-Sample t Test

The following analysis compares the mean values for one data set to an external standard using a one-sample t-test.

1	Input Summary: Shows you a summary of the inputs used for this analysis. The critical point is entering raw data in a column for this analysis. We have one set of data that we are comparing to an external standard (specified as 10 in Ho), and the alternative hypothesis is not equal. In effect, we are checking if the mean of the proc time data is 10 or not. The methodology is set at Auto so that the software will pick the proper method for this analysis.
2	Assumption Checks: Lists any assumptions checked for this analysis and the status of the checks. A check mark indicates that the assumptions were satisfied, and a cross mark indicates that the assumptions were not met. In our case, the software checks if the data is normally distributed and determines that the data is normal.
3	Hypothesis Test: Lists the null and alternative hypothesis that is being conducted based on the inputs specified in the dialog box. Always make sure that you have correctly specified the hypothesis for your test.
4	Analysis Results: The analysis's results include test statistics, P values, and confidence levels. The T statistic is 0.43, and the P value is 0.673. Since the P value is greater than alpha, we can conclude that there is no reason to believe that the mean of the data is different from 10. The confidence interval specifies that, with 95% confidence, the mean values could vary between 9.42 and 10.88.
5	Conclusions: This study's primary conclusion is that the mean is not different from 10.
6	Graphs: Shows the graphs generated for this analysis, such as hypothesis P plot, confidence interval plot, box plots, etc. In our case, we have the confidence interval plot and the box plot of the data.

Mann-Whitney Test

The following is a case where we compare two data sets using a non-parametric test such as the Mann-Whitney test.

1	Input Summary: Shows you a summary of the inputs used for this analysis. In our case, we are comparing two data sets, the processing time for the north and east zones. Note that we are interested in comparing the mean/median values, and the methodology is set to Auto, which means the software will pick the best method for this analysis.
2	Assumption Checks: Lists any assumptions checked for this analysis and the status of the checks. A check mark indicates that the assumptions were satisfied, and a cross mark indicates that the assumptions were not met. In our case, the data is NOT normally distributed.
3	Hypothesis Test: Lists the null and alternative hypothesis that is being conducted based on the inputs specified in the dialog box. Note that for the non-parametric tests, the comparison is for the median values. Here, we compare whether the medians are the same or different.
4	Analysis Results: Lists the results of the analysis, including any test statistic, P values, and confidence levels. The W statistic for the Mann-Whitney test was 919, corresponding to a P value of less than 0.001.
5	Conclusions: Lists the primary conclusion from this study. Since the P value is very low, we can safely conclude that the median values between the two populations differ.
6	Graphs: Shows the graphs generated for this analysis, such as hypothesis P plot, box plots, etc. Note that only the box plot is shown between the two groups. The confidence interval plots are only shown for the parametric tests.

Levene Test

We compare the standard deviation values for four data sets using Levene's test in the following case.

1	Input Summary: Shows you a summary of the inputs used for this analysis. Note that this example compares the standard deviations of four data sets (North, East, West, and South). The methodology is set at Auto so that the system will pick the right methodology for this case.
2	Assumption Checks: Lists any assumptions checked for this analysis and the status of the checks. A check mark indicates that the assumptions were satisfied, and a cross mark indicates that the assumptions were not met. In our case, at least one of the data sets is not normal.
3	Hypothesis Test: Lists the null and alternative hypothesis that is being conducted based on the inputs specified in the dialog box. The null hypothesis is that all the standard deviations are equal, and the alternative hypothesis is that at least one standard deviation is different.
4	Analysis Results: Lists the results of the analysis, including any test statistic, P values, and confidence levels. The W statistic is computed for Levene's test, and this is, in turn, used to determine the P value. The P value was determined to be less than 0.001.
5	Conclusions: Lists the primary conclusion from this study. From this study, we can conclude that the standard deviations are different between the four data sets.
6	Graphs: Shows the graphs generated for this analysis, such as hypothesis P plot, box plots, etc. Since this is a non-parametric test, no confidence intervals are displayed. The box plot shows the differences in the variations between groups.

ANOVA Test

In the following case, we compare the average values for four data sets using the ANOVA Test.

1	Input Summary: Shows you a summary of the inputs used for this analysis. Note that our data is in the Raw data format, and we are comparing the mean values of the four data sets, namely North, East, West, and South. We have specified the methodology as ANOVA and not let it be selected as Auto, as we want this specific methodology to be used for this analysis.
2	Assumption Checks: Lists any assumptions checked for this analysis and the status of the checks. A check mark indicates that the assumptions were satisfied, and a cross mark indicates that the assumptions were not met. In our case, at least one of the data sets is not normal. Ideally, we should not be using the ANOVA test for this case.
3	Hypothesis Test: Lists the null and alternative hypothesis that is being conducted based on the inputs specified in the dialog box. The null hypothesis states that all the means are the same, and the alternative hypothesis states that at least one mean is different.
4	Analysis Results: Lists the results of the analysis, including any test statistic, P values, and confidence levels. The ANOVA test results in an F value of 85.47, which translates to a P value of less than 0.001.
5	Conclusions: Lists the primary conclusion from this study. Since the P value is less than alpha, we can conclude that at least one means is different.
6	Graphs: Shows the graphs generated for this analysis, such as hypothesis P plot, box plots, etc. The graphs section plots the confidence interval of the means for each group along with the box plot of the data.

Chi-Square Test

We compare the proportion values for three data sets using the Chi-Square Test in the following case.

1	Input Summary: Shows you a summary of the inputs used for this analysis. Note that we have specified summarized data for this study. There are 3 data sets that we want to compare and two categories (defectives and nondefectives).
2	Assumption Checks: Lists any assumptions checked for this analysis and the status of the checks. A check mark indicates that the assumptions were satisfied, and a cross mark indicates that the assumptions were not met. For this analysis, we require each data set with at least 5 satisfied data points.
3	Hypothesis Test: Lists the null and alternative hypothesis that is being conducted based on the inputs specified in the dialog box. Note that for the non-parametric tests, the comparison is for the median values. The null hypothesis is that all the proportions are equal, and the alternative hypothesis is that at least one proportion is different.
4	Analysis Results: Lists the results of the analysis, including any test statistic, P values, and confidence levels. The test statistic is the Chi-Square value, which is evaluated at 0.04. This translates to a P value of 0.98.
5	Conclusions: Lists the primary conclusion from this study. Since the P value is high (compared to alpha of 0.05), we can conclude there is no reason to believe that the proportions are different.
6	Graphs: Shows the graphs generated for this analysis, such as hypothesis P plot, box plots, etc. The graph shown in this case is a 3D bar chart. We can use the bar chart to identify where the Chi-square values contribution is coming from.

Notes

Here are a few pointers regarding this analysis:

• You can specify your tests in the Analysis Setup menu. However, it is recommended that you set the determination of the test to "Auto" so that Sigma Magic can pick the right test for you based on certain checks it performs on the data sets.
• The minimum number of data points depends on several factors, such as the difference you are interested in detecting, the amount of type 1 and type 2 errors you are willing to tolerate, and the amount of variation you have in your data. Use the sample size calculator to determine the minimum sample size required.

Examples

The following examples are in the Examples folder.

• Data is collected for the hours worked to process a claim. Determine if there is reason to believe the processing time is over 10 hours (Compare Data Set 1.xlsx).
• Compare the number of defects generated by the department for the last year and determine if it is greater than the industry average of 2% (Compare Data Set 2.xlsx).
• Compare the performance of the north region with the east region in the meantime to repair data and determine if there are any differences between the two groups (Compare Data Set 3.xlsx).
• Compare the performance of all the regions of the country (north, south, east, west) for the mean repair time and determine if there are any differences between them (on average) (Compare Data Set 3.xlsx).
• Compare the performance of all the regions of the country (north, south, east, west) for the mean repair time and determine if there are any differences in variation between the data sets (Compare Data Set 3.xlsx).

---

© Rapid Sigma Solutions LLP. All rights reserved.