Differences in emmeans and marginaleffects outputs

[nodice73]

Hello! Thank you for the amazing code and book. I have been using emmeans extensively for a project, but I think there are aspects of marginaleffects that would be useful so I am trying to learn how to use it. I have read "Model to Meaning", but I don't understand something about the differences between the calculations in emmeans and marginaleffects.

As stated in chapter 3.2.4, one difference between emmeans and marginaleffects is that emmeans uses what marginaleffects calls a "balanced grid" by default. So, I was expecting the following code to have the same output:

library(emmeans)
library(marginaleffects)

dat = get_dataset("thornton")

mod = glm(outcome ~ incentive + agecat + distance, data = dat, family = binomial)

(emmeans(mod, ~ agecat, type="response", infer=TRUE, adjust = "none"))

# agecat    prob     SE  df asymp.LCL asymp.UCL null z.ratio p.value
# <18      0.554 0.0329 Inf     0.489     0.617  0.5   1.615  0.1063
# 18 to 35 0.562 0.0166 Inf     0.530     0.595  0.5   3.718  0.0002
# >35      0.615 0.0178 Inf     0.580     0.649  0.5   6.230  <.0001

avg_predictions(mod, newdat = "balanced", by = "agecat", hypothesis = 0.5)

#   agecat Estimate Std. Error    z Pr(>|z|)    S 2.5 % 97.5 %
# <18         0.542     0.0261 1.61    0.108  3.2 0.491  0.593
# 18 to 35    0.549     0.0135 3.62   <0.001 11.7 0.522  0.575
# >35         0.591     0.0151 6.02   <0.001 29.1 0.561  0.621

The point estimates are similar, but they don't seem to be close enough to be due to numerical error. Also, in section 36.1.1, the examples you show of contrast vs comparisons give identical output (to at least the third decimal place).

Similarly, based on my understanding of the emmeans "weights" argument, I thought this code would give the same output:

(emmeans(mod, ~ agecat, type="response", infer=TRUE, adjust = "none", weights = "cells"))

# agecat    prob     SE  df asymp.LCL asymp.UCL null z.ratio p.value
# <18      0.686 0.0282 Inf     0.628     0.739  0.5   5.966  <.0001
# 18 to 35 0.689 0.0137 Inf     0.661     0.715  0.5  12.445  <.0001
# >35      0.744 0.0138 Inf     0.716     0.770  0.5  14.666  <.0001

avg_predictions(mod, by = "agecat", hypothesis = 0.5)

#   agecat Estimate Std. Error     z Pr(>|z|)     S 2.5 % 97.5 %
# <18         0.670     0.0240  7.07   <0.001  39.2 0.623  0.717
# 18 to 35    0.673     0.0116 14.91   <0.001 164.6 0.650  0.695
# >35         0.720     0.0121 18.21   <0.001 243.7 0.697  0.744

Again, similar but not the same.

I see that in section 36.1.2, something like this is discussed, but without the math I cannot quite follow. For example, I am not sure what "Make adjusted predictions in each cell of the prediction grid" means (what is an "adjusted prediction" in this context?).

I apologize in advance if you covered this somewhere and I missed it.

[vincentarelbundock]

this is a very good question. I added a short section to the vignette to explain the difference:

https://marginaleffects.com/bonus/alternative_software.html#back-transformation-order-of-operations

[nodice73]

Amazing! Thank you. This explains it perfectly. I see your justification for transforming-then-averaging, but do you happen to know the reason one might want to take the emmeans approach of averaging-then-transforming?

Also, you have a typo in both of your lists (point 2: "grind" instead of "grid").

[vincentarelbundock]

I'm not sure what the statistical justification is, but from a computational perspective, it almost certainly makes it a lot easier and faster to compute standard errors. That's because, in the linear/link case, the jacobian is the same as the model matrix.