Home / DOE / Analyze / Nominal Logistic Regression

Nominal Logistic Regression¶

Nominal logistic regression is the engine Run Regression uses when an output has three or more outcomes with no order between them: a colour, a day of the week, a failure mode. For an output with exactly two outcomes use Binary Logistic Regression, which is better suited to it.

When it does not converge¶

The message appears in red at the top of the regression table, and for a nominal fit it is that one sentence on its own. The separation sentences and the last-iteration sentence belong to the binary engine, not this one:

Model failed to converge. The results of the model are in question.

Running out of half steps is reported as a success instead, with the message Model Converged (Half stepping limit reached)., so that line is worth reading rather than skipping.

The commonest cause with a nominal response is an outcome that occurs in too few runs, or that is perfectly predicted by one factor level. An outcome observed once or twice cannot support its own set of coefficients.

What the report contains¶

Coefficients, one set per non-baseline outcome, each compared with the baseline.

Odds ratios, the exponentiated coefficients, with confidence bounds obtained by exponentiating the bounds of the coefficient:

\[ \text{OR} = e^{b_{jk}} \qquad \text{CI} = e^{\text{CI}(b_{jk})} \]

The log likelihood and its value at each iteration, with RSquaredU, AIC and BIC beside it, and the G statistic with its degrees of freedom and p value.

A Goodness of Fit Test block with two rows, Pearson and Deviance, each with its chi-square, degrees of freedom and p value. As with the binary engine, a large p value is the good outcome: it says the fitted probabilities are consistent with the observed counts.

A predicted probability table, two columns wide with one row per outcome, the baseline included. The baseline's cell is not a subtraction from the others: it holds the reference-level formula directly, which is what makes the column of probabilities total exactly 1.

Which outcome is the baseline¶

The baseline is the outcome with no coefficients of its own. It is not left out of the model: every other outcome's coefficients are measured against it, so it is the reference the whole report is built on.

This matters when reading the report, because the same data with a different baseline gives different coefficients describing the same fit. The predicted probabilities do not change.

Predictions¶

For a nominal output with \(L\) non-reference levels there is one sum per level, \(\eta_k = \sum_j c_j b_{jk}\), and the probability of level \(k\) is

\[ p_k = \frac{e^{\eta_k}}{1 + \sum_{l=1}^{L} e^{\eta_l}} \]

The \(1\) in the denominator is the reference level, whose \(e^{\eta}\) is \(1\) by construction. So the reference level's own probability is

\[ p_{\text{ref}} = \frac{1}{1 + \sum_{l=1}^{L} e^{\eta_l}} \]

and the probabilities across all levels sum to \(1\).

Full details: Prediction Equation.

When charting a nominal output you choose which outcome's probability to plot, and each outcome gets its own chart, because there is no single response value to draw.

The mathematics¶

The model, the fit and what is estimated are on Math Details.

References¶

Agresti, A. (2013). Categorical Data Analysis, 3rd edition. Wiley.
Hosmer, D. W., Lemeshow, S., and Sturdivant, R. X. (2013). Applied Logistic Regression, 3rd edition. Wiley.
McCullagh, P., and Nelder, J. A. (1989). Generalized Linear Models, 2nd edition. Chapman and Hall.