01f-regression-categorical.qmd

title	Introduction to Statistical Modeling
subtitle	Categorical variables
author	Joris Vankerschaver

```{r include = FALSE} birnh <- read.table("./datasets/01-linear-regression/birnh.txt", header = T, sep = "\t", dec = ".") ``` ## Example: BIRNH study - Epidemiological follow-up study in the mid 80s where nutritional and health data in Belgium were measured ($n=5,815$) - **Goal**: effect of smoking on cholesterol - Since people from different provinces might have a different smoking and dietary behaviour, we want to correct for province - Possible values of this variable: - 1: West Flanders - 2: East Flanders - 3: Flemish Brabant - 7: Antwerp - 8: Limburg ## A first analysis ... \footnotesize ```{r echo=FALSE} model_birnh <- lm(TCHOL ~ SMOKING + AGE + I(AGE^2) + I(AGE^3) + SEX + PROVINCE, data = birnh) summary(model_birnh)$coefficients ``` \normalsize Model implicitly assumes that mean difference in cholesterol - between Limburg and West-Flanders is 7 times as large as - the one between East- and West-Flanders ## Dummy variables - Create 4 **dummy variables** \begin{align*} P_2 & = \left\{\begin{array}{ll} 1 & \textrm{East Flanders}\\ 0 & \textrm{other} \end{array}\right.\\ P_3 & = \left\{\begin{array}{ll} 1 & \textrm{Flemish Brabant}\\ 0 & \textrm{other} \end{array}\right.\\ P_7 & = \left\{\begin{array}{ll} 1 & \textrm{Antwerp}\\ 0 & \textrm{other} \end{array}\right.\\ P_8 & = \left\{\begin{array}{ll} 1 & \textrm{Limburg}\\ 0 & \textrm{other} \end{array}\right.\\ \end{align*} ## Dummy variables - These 4 dummy variables carry same information as variable PROVINCE: - In West Flanders: $(P_2,P_3,P_7,P_8)=(0,0,0,0)$ - In East Flanders: $(P_2,P_3,P_7,P_8)=(1,0,0,0)$ - In Flemish Brabant: $(P_2,P_3,P_7,P_8)=(0,1,0,0)$ - In Antwerp: $(P_2,P_3,P_7,P_8)=(0,0,1,0)$ - In Limburg: $(P_2,P_3,P_7,P_8)=(0,0,0,1)$ - Each categorical variable with $k$ levels can be transformed into $k-1$ dummy variables by choosing 1 level as **reference**: - In \texttt{R}: ```{r} #| echo: true m <- lm(TCHOL ~ SMOKING + AGE + I(AGE^2) + I(AGE^3) + SEX + factor(PROVINCE), data = birnh) ``` ## Analysis with dummy variables \scriptsize ```{r echo=FALSE} m <- lm(TCHOL ~ SMOKING + AGE + I(AGE^2) + I(AGE^3) + SEX + factor(PROVINCE), data = birnh) summary(m)$coefficients ``` \phantom{Necessary to test if multiple coefficients are zero} ## How to test for effect of province? \scriptsize ```{r echo=FALSE} m <- lm(TCHOL ~ SMOKING + AGE + I(AGE^2) + I(AGE^3) + SEX + factor(PROVINCE), data = birnh) summary(m)$coefficients ``` \normalsize Necessary to test if multiple coefficients are zero ## F-test for effect of province Compare two **nested models** (as in slide deck on predictivity): - **Complex model** with $p_\mathrm{complex} = 10$ parameters: $$ E(Y|X,P)=\beta_0+\beta_1X+\beta_2P $$ - **Simple model** with $p_\mathrm{simple} = 6$ parameters: $$ E(Y|X,P)=\beta^*_0+\beta^*_1X $$ - Testing $H_0:\beta_2=0$ is equivalent to testing if both models fit the data equally well. ## F-test for effect of province - Use the $F$-test to compare residual sums of squares: $$ f = \frac{\frac{SSE_\mathrm{simple}- SSE_\mathrm{complex}}{p_\mathrm{complex} - p_\mathrm{simple}}}{\frac{SSE_\mathrm{complex}}{n - p_\mathrm{complex} }} \sim F_{p_\mathrm{complex} - p_\mathrm{simple}, n - p_\mathrm{complex}}. $$ ## F-test in \texttt{R} \footnotesize ```{r} mC <- lm(TCHOL ~ SMOKING + AGE + I(AGE^2) + I(AGE^3) + SEX + factor(PROVINCE), data = birnh) mR <- lm(TCHOL ~ SMOKING + AGE + I(AGE^2) + I(AGE^3) + SEX, data = birnh) anova(mR,mC) ```