Sigma Magic Help Version 15

Generalized Linear Model

Overview

Ordinary linear regression predicts the expected value of a dependent variable as a linear combination of a set of observed values (predictors). This implies that a constant change in a predictor leads to a constant change in the response variable (for a linear response model). One of the assumptions of the ordinary linear regression is that the error terms (the difference between the data and the fitted model) follow a normal distribution. This model works fine when the response variable can vary essentially indefinitely in either direction or, more generally, for any quantity that only varies by a relatively small amount.

However, these assumptions are inappropriate for some types of response variables. For example, a model that predicts a probability of making a yes/no choice (a Bernoulli variable) is not suitable as a linear-response model since probabilities are bounded on both ends (they must be between 0 and 1). If a linear model is fit for this situation, the predicted response values can be less than 0 or greater than 1! In these situations, instead of fitting a model directly between the output Y and the input X, a link function is used where a model is fit between the probability of Y vs. the input variable X. By using a well-defined link function, the outputs will always lie between 0 and 1.

Generalized linear models cover all these situations by allowing for response variables that have arbitrary distributions (rather than simply normal distributions). They generalize the possible distributions of the residuals to a family of distributions called the exponential family. This family includes the normal as well as the Binomial, Poisson, Gamma distributions, among others. In addition, the generalized model relaxes the requirement of a linear relationship between the dependent and independent variables. When you change the distribution of the residuals, it turns out that the relationship between Y and the model parameters is no longer linear. However, for each distribution in the exponential family, there exists at least one function of the mean of Y, whose relationship with the model parameters is linear. This function is called the link function, which is used to develop this model.

This functionality is provided by the R software using the function glm within the Stats package. Note that this functionality requires that RScript software be installed on your computer and linked to the Sigma Magic software within the options menu. Generalized Linear Model worksheet can be added to your active workbook by clicking on Analytics and then selecting Generalized Linear Model.

Inputs

Click on Analysis Setup to open the menu options for this tool. A sample screenshot of the menu is shown below.

1	Data Type: The data type for this analysis can be either continuous data or categorical data. Option Description Continuous Pick this option is your Y metric is continuous data. Categorical Use this option if your Y metric is categorical data.
2	Training Size: Specify what % of the data points should be picked for the training set. These are used to build the model, and the remaining points are used to test the model.
3	Training Set: Specify the data to be used for training the algorithms. Option Description Auto Random method for selection of the training data set is used./td> Random A random number algorithm is used to select the training set. Sequential A sequential set of samples are picked. For example, 1st point, 3rd point, 5th point, etc.
4	Link Function: Each family has a set of link-functions that can be defined. The following options are available: Option Description Binomial Use the link functions: Logit, Probit, or CLogLog. Gaussian Uses the Identify link function. Gamma Uses the Inverse or the Log link function. Inv Gaussian Uses the Musquare link function. Poisson Use the Identity, Log, or Sqrt link function.
5	Family: Specify the description of the error distribution and link function to be used in the model. There are several options available: Option Description Binomial Use this when predicting binary outcome from a set of continuous predictor variables. It is usually preferred over discriminant analysis because of less restrictive assumptions. Poisson (or Quasi-Poisson) Use this when predicting an outcome of a variable representing counts from a set of continuous predictor variables. Gaussian Use this for predicting outcomes of a continuous primary metric. Inverse Gaussian Use this when the response variable follows an inverse gaussian distribution.
6	Tolerance: Specify the tolerance value for GLM model convergence. The software will stop trying to find a better solution once the tolerance value is reached.
7	Max Iterations: Specify the maximum number of iterations for GLM model convergence. The software will stop trying to find a better solution once the number of iterations is exceeded.
8	Formula: Specify the formula to use to build the GLM model. There are multiple formulae used to build the model between input(s) and output. You can specify the interaction between variables using the ":" keyword. For example, if X1, X2, X3 are the input variables and Y is the output variable, you can specify the formula as Y ~ X1^2 + X2 + X3:X1 (where you have a quadratic term for X1, a linear term for X2, and an interaction term between X1 and X3. Clearly, the generalized linear model is very powerful to model all types of equations between the input(s) and the output.
9	Help Button: Click on the Help Button to view the help documentation for this tool.
10	Cancel Button: Click on the Cancel Button to discard your changes and exit this menu.
11	OK Button: Click on the OK Button to save your changes and try to execute the program. Note that you will need to specify the required data in order to complete the analysis and generate outputs. If there are any missing data, then the software will remind you to specify the data and click on Compute Outputs to generate analysis results.

Data

If you click on the Data button, you will see the following dialog box. Here you can specify the data required for this analysis.

1	Search Data: The available data displays all the columns of data that are available for analysis. You can use the search bar to filter this list and to speed up finding the right data to use for analysis. Enter a few characters in the search field and the software will filter and display the filtered data in the Available Data box.
2	Available Data: The available data box contains the list of data available for analysis. If your workbook does not have any data in tabular format, this box will display "No Data Found". The information displayed in this box includes the row number, whether the data is Numeric (N) or Text (T), and the name of the column variable. Note that the software displays data from all the tables in the current workbook. Even though data within the same table have unique column names, columns across different tables can have similar names. Hence, it is important that you not only specify the column name but also the table name.
3	Add or View Data: Click on this button either to add more data into your workbook for analysis or to view more details about the data listed in the available data box. When you click on this button, it opens up the Data Editor dialog box where you can import more data into your workbook, or you can switch from the list view to a table view to see the individual data values for each column.
4	Required Data: The code for the required data specifies what data can be specified for that box. An example code is N: 2-4. If the code starts with an N, then you will need to select only numeric columns. If the code starts with a T, then you can select both numeric and text columns. The numbers to the right of the colon specify the min-max values. For example, if the min-max values are 2-4, then you need to select a minimum of 2 columns of data and a maximum of 4 columns of data in this box. If the minimum value is 0, then no data is required to be specified for this box.
5	Select Button: Click on this button to select the data for analysis. Any data you select for the analysis is moved to the right. To select a column, click on the columns in the Available Databox to highlight them and then click on the Select Button. A second method to select the data is to double click on the columns in the list of Available Data. Finally, you can also drag and drop the columns you are interested in by holding down the select columns using your left mouse key and dragging and dropping them in one of the boxes on the right.
6	Selected Data: If the right amount of data columns has been specified, the list box header will be displayed in the black color. If sufficient data has not been specified, then the list box header will be displayed in the red color. Note that you can double-click on any of the columns in this box to remove them from the box.
7	View Selection: Click on this button to view the data you have specified for this analysis. The data can be viewed either in the tablular format or you can view a graphical summary of the data selected.

Program

If you click on the Program button, the software will display the program code - an example screenshot is shown in the figure below.

1	R Program: You can view the R program that will be executed here. This program is usually automatically generated from the options you have specified in the setup earlier. This is the program that will be executed by the R program to generate analysis outputs. If you like, you can edit this program.
2	Auto Mode: If the radio button is selected as Auto, then the software will automatically update this code based on any changes you make in the input dialog box. We recommend that you use this option to generate the R program so that all your input settings are used to generate analysis results.
3	Manual Mode: If you use the Manual option, then you will be allowed to edit the R program before the program is executed. Make sure that you specify a syntactically correct program; otherwise, the R program may report errors.

Verify

If you click on the Verify button, the software will perform some checks on the data you have entered. A sample screenshot of the data is shown in this figure.

1

Verify Checks: The objective of this analysis as well as any checks that are performed are listed in this dialog box. For example, the software may check if you have correctly specified the input options and if you have specified the data correctly for analysis.

2

Check Status: The results of the analysis checks are listed here. If the checks are passed, then they are shown as a green-colored checkmark. If the verification checks fail, then they are shown as a red-colored cross. If the verification checks result in a warning, they are shown in the orange color exclamation mark and finally, any checks that are required to be performed by the user are shown as blue info icons.

Outputs

Click on Compute Outputs to update the results on the worksheet. A sample screenshot of the worksheet is shown below.

1

Notes Section: The notes section provides a summary of the input data, and the analysis results section shows the name of the response variable along with the names of the input variables. Information is provided on the residuals and the model fit. By reviewing the P-values for each of the selected model coefficients, we can determine which of the terms in the model are significant. For this analysis, we can see that the only disp is statistically significant (P-value of 0.03) and the other variables have P values greater than alpha (0.05). We can run a model reduction and only build a model between am and disp. The predictions of the output variable based on this model are given in the prediction column.

2

Graph Section: The graph section shows the standard residuals vs. model leverage and Cook's distance. Leverage is a measure of how far the independent variable values of observation are from those of other data points. A data point has high leverage if it is extreme and may influence model results. Cook's distance D is used to estimate the influence of a data point when performing the least-squares regression analysis. The histogram of the residuals is also shown. We need to ensure that we can trust the model results by reviewing these charts. The residuals need to be close to zero, normally and randomly distributed.

Notes

Here are a few pointers regarding this analysis:

• This analysis requires that the R software needs to be installed on your computer. Further, you will need to provide a link to the RScript executable file under Sigma Magic Options so that the software can use the R software to generate analysis results.

Examples

Following examples can be found in the Examples folder.

• For the data given in the file, determine the generalized linear model for am as the output. Note that am has only 2 values - use a Binomial distribution for this analysis. (Generalized Linear Model 1.xlsm)

References

For more information on this topic, please refer to the following articles. Do note that if any external links are mentioned below, they are for reference purposes only.

• GLM Overview: http://www.statmethods.net/advstats/glm.html
• GLM in R: https://www.r-bloggers.com/what-does-a-generalized-linear-model-do/