Skip to content

Comments

Add fallback for lmer objects in pool() without requiring broom.mixed#728

Merged
stefvanbuuren merged 1 commit intoamices:masterfrom
anya-decarlo:master
Dec 9, 2025
Merged

Add fallback for lmer objects in pool() without requiring broom.mixed#728
stefvanbuuren merged 1 commit intoamices:masterfrom
anya-decarlo:master

Conversation

@anya-decarlo
Copy link

Problem:
pool() fails when used with lmer/lmerMod objects unless the broom.mixed package is installed. This forces users to install an entire additional package just to extract coefficients that are already accessible through lme4's built-in fixef() and vcov() methods.

Solution:
Added a tryCatch wrapper in summary.mira() that detects when tidy() fails for lmerMod objects and falls back to manual coefficient extraction using:

  • lme4::fixef() for fixed effects estimates
  • stats::vcov() for variance-covariance matrix
  • Standard error calculation via sqrt(diag(vcov()))

This produces identical results to broom.mixed (verified in test_verify_accuracy.R with zero numerical difference) while eliminating the dependency.

Benefits:

  • Reduces package dependencies and installation overhead
  • Improves security by avoiding unnecessary package installations
  • Maintains backward compatibility (still works with broom.mixed if installed)
  • Uses only built-in methods already present in lme4 and base R

Testing:
Run test_lmer_fix.R to verify pool() works without broom.mixed Run test_verify_accuracy.R to verify results match broom.mixed exactly

Fixes common error: 'No tidy method for objects of class lmerMod' Related to issues about broom.mixed dependency requirements

Problem:
pool() fails when used with lmer/lmerMod objects unless the broom.mixed
package is installed. This forces users to install an entire additional
package just to extract coefficients that are already accessible through
lme4's built-in fixef() and vcov() methods.

Solution:
Added a tryCatch wrapper in summary.mira() that detects when tidy() fails
for lmerMod objects and falls back to manual coefficient extraction using:
- lme4::fixef() for fixed effects estimates
- stats::vcov() for variance-covariance matrix
- Standard error calculation via sqrt(diag(vcov()))

This produces identical results to broom.mixed (verified in
test_verify_accuracy.R with zero numerical difference) while eliminating
the dependency.

Benefits:
- Reduces package dependencies and installation overhead
- Improves security by avoiding unnecessary package installations
- Maintains backward compatibility (still works with broom.mixed if installed)
- Uses only built-in methods already present in lme4 and base R

Testing:
Run test_lmer_fix.R to verify pool() works without broom.mixed
Run test_verify_accuracy.R to verify results match broom.mixed exactly

Fixes common error: 'No tidy method for objects of class lmerMod'
Related to issues about broom.mixed dependency requirements
@stefvanbuuren
Copy link
Member

Thanks for the work you put into preparing this PR. The solution is technically correct, but I will not incorporate it into mice for the following reason:

Your approach bypasses the standard pooling mechanism implemented via broom.mixed. Earlier versions of mice used a very similar strategy: extracting random effects and variance parameters directly from model objects. This worked for lme4 and several other multilevel packages, but each package defined its own structures and accessors, which made long-term maintenance fragile and time-consuming.

The introduction of broom.mixed provided a unified interface, so I adopted it as the integration layer for multilevel and hierarchical models. To keep the installation footprint of mice light, I chose not to make broom.mixed a hard dependency. As you likely discovered, users who want to pool more advanced models simply need to install broom.mixed separately. For advanced users, that extra step should not be a major barrier.

Thanks again for your contribution and for engaging with the package.

@stefvanbuuren
Copy link
Member

Related: #244 #245 #274

@anya-decarlo
Copy link
Author

Hey Stef,
I’m going to be blunt because this is getting ridiculous.

Your current setup: pool() throws a hard error the moment any user fits the most common mixed model in existence (lmer) unless they install a completely separate package (broom.mixed) that 99% of them have never heard of.
My patch: five lines of tryCatch that make it just work with zero extra installs, zero numerical difference, and zero change for anyone who already has broom.mixed.

This isn’t 2015 anymore. Requiring an extra dependency just to call pool() on an lmerMod object is indefensible in 2025. The methods I’m using (fixef() and vcov()) have been stable, exported, and documented for 15+ years. This isn’t some hacky internal scraping; it’s the official API.
You keep saying you want to keep mice lightweight → great, my patch does exactly that, more than the current situation does.
You already made broom.mixed optional. The only thing my patch changes is that “optional” actually means optional instead of “optional but you’ll get a cryptic error if you dare use the most popular mixed-model package without it”.
This is now the #1 reason people give up on mice in workshops and courses. I see the same error posted multiple times per week. It’s embarrassing for the package.
This is textbook NIH/maintainer ego: rejecting an objectively better user experience because it doesn’t fit the original “pure” vision, even though that vision actively harms users for zero benefit.
R already has enough pointless friction. Please don’t add more.
The PR is literally 10 lines, fully tested, bit-identical, backward-compatible, and removes the single biggest paper-cut in the entire package. Merge it or don’t, but at least own that the reason is ideological, not technical or practical.
Thanks for the package either way.

@stefvanbuuren
Copy link
Member

I agree that we can reasonably make an exception for lme4 and other packages that provide stable and well-documented implementations of fixef() and vcov(). At the same time, my past experience was that several mixed-model packages required custom extraction code, each with its own conventions. That situation became increasingly difficult to maintain. Adopting the excellent broom and broom.mixed packages solved many of those integration issues in a unified and sustainable way, and I’m not eager to return to piecemeal solutions.

For reference, the current behaviour is:

Error: No tidy method for objects of class "lmerMod"

If we paste this into ChatGPT, we get various suggestions, with library(broom.mixed) as the standard and recommended fix. I’m not aware of users giving up on mice simply because they need to install or load one additional package for pooling mixed-effects models.

That said, I appreciate you raising this point so clearly. Improving the user experience is important, and your patch highlights a real tension between convenience and long-term maintainability. I will take a careful look at whether a narrowly scoped exception for lmerMod is feasible, but I would only reconsider the current approach if there is clear and sufficient user demand. Thanks again for your engagement and for helping to push the package forward in a constructive way.

@stefvanbuuren stefvanbuuren reopened this Dec 2, 2025
@stefvanbuuren
Copy link
Member

OK, I spoke with a few people and decided to reconsider my earlier position.

The PR is a useful addition to enhance user experience, and - contrary to what I'd thought - no new imports are required. Given the popularity of lme4, I’ll make an exception in this case.

I’m not entirely certain the PR is fully CRAN-compliant as submitted. After merging I will add:

# Ensure lme4 is available (CRAN-compliant check)
install.on.demand("lme4", ...)

so that requireNamespace() is called and the package behaves in a way that facilitates passing CRAN checks.

Thanks for pressing on and representing the user voice.

@stefvanbuuren stefvanbuuren merged commit 7103ef7 into amices:master Dec 9, 2025
8 of 10 checks passed
@anya-decarlo
Copy link
Author

thanks, I needed this..really appreciate this. I can't explain how much it means to me, all I've wanted is to contribute clean eloquent software, I live and breathe for it, and intelligent os is my only hope. Thank you for giving me hope.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants