GLM Support

[s3alfisc]

This PR

• Adds an abstract Feglm class from which different type of GLM families can inherit.
• Adds a pf.feglm(, family = "logit") front end to call these functions.
• The implementation closely follows this paper by Stamann: https://arxiv.org/pdf/1707.01815. Equations and algo are implemented "verbatim" - should be easy to match code to paper
• Families to support for now: (unconditional) logit, probit, an alternative Poisson implementation.

Other PR

• Later: anything else of interest. Clogit (different estimator). Negbin?
• Incidental Param Problem Bias Corrections

TODO:

• Implement abstract GLM base class (inheriting from Fepois for now, later this should be reversed) with IWLS algo and step halfing
• Implement Logit class based on Stamman (not yet converging, bug somewhere)
• store IWLS parameters so that they can be used in post estimation classes (i.e. for computing vcov's etc). See Fepois.get_fit() method for details.
• tests against fixest::feglm()

Goal:

import pyfixest as pf
data = pf.get_data(model ="Fepois")
data["Y"] = np.where(data.Y > 0, 1, 0)
fit = pf.feglm("Y ~ X1 | f1", data = data, family = "logit")

Should all be doable before the year ends (hopefully!)

Fyi @leostimpfle @apoorvalal @gbekes.

[codecov]

Codecov Report

Attention: Patch coverage is 32.80757% with 213 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
pyfixest/estimation/feglm_.py	23.95%	146 Missing ⚠️
pyfixest/estimation/estimation.py	5.00%	19 Missing ⚠️
pyfixest/estimation/fegaussian_.py	50.00%	16 Missing ⚠️
pyfixest/estimation/FixestMulti_.py	33.33%	12 Missing ⚠️
pyfixest/estimation/felogit_.py	60.00%	10 Missing ⚠️
pyfixest/estimation/feprobit_.py	61.53%	10 Missing ⚠️

Flag	Coverage Δ
core-tests	75.00% <32.80%> (-3.10%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
pyfixest/__init__.py	81.81% <ø> (ø)
pyfixest/estimation/__init__.py	100.00% <100.00%> (ø)
pyfixest/estimation/feiv_.py	98.18% <ø> (ø)
pyfixest/estimation/feols_.py	83.57% <100.00%> (-0.07%)	⬇️
pyfixest/estimation/fepois_.py	87.26% <ø> (ø)
pyfixest/estimation/felogit_.py	60.00% <60.00%> (ø)
pyfixest/estimation/feprobit_.py	61.53% <61.53%> (ø)
pyfixest/estimation/FixestMulti_.py	76.27% <33.33%> (-3.93%)	⬇️
pyfixest/estimation/fegaussian_.py	50.00% <50.00%> (ø)
pyfixest/estimation/estimation.py	81.51% <5.00%> (-15.46%)	⬇️
... and 1 more

[s3alfisc]

Status: implementations seem to work for logit and probit without fixed effects (coefficients match, SEs is another story):

import statsmodels.api as sm
import pyfixest as pf 

spector_data = sm.datasets.spector.load_pandas()
spector_data.exog = sm.add_constant(spector_data.exog)

# fit via statsmodels
logit_mod = sm.Logit(spector_data.endog, spector_data.exog)
probit_mod = sm.Probit(spector_data.endog, spector_data.exog)
logit_res = logit_mod.fit()
probit_res = probit_mod.fit()

logit_params = logit_res.params
probit_params = probit_res.params
print(logit_params)
#          Iterations 6
# const   -13.021347
# GPA       2.826113
# TUCE      0.095158
# PSI       2.378688
# print(probit_params)
# const   -7.452320
# GPA      1.625810
# TUCE     0.051729
# PSI      1.426332


# fit via pyfixest 
fit_logit = pf.feglm("GRADE ~ GPA + TUCE + PSI", data = spector_data.data, family = "logit")
fit_probit = pf.feglm("GRADE ~ GPA + TUCE + PSI", data = spector_data.data, family = "probit")

pf.etable([fit_logit, fit_probit], digits = 6)

[apoorvalal]

wait you closed this without merging? is this in a different branch now?

[s3alfisc]

Yes it's here: #763. Branch was named "GLM" but I checked out "glm" at some point ...🤦‍♂️

[gbekes]

Thanks! Gonna test it
@adamvig96

Clogit would be great next....

[s3alfisc]

Cool, thanks @gbekes! If you could choose, would you rather start with the clogit or implement the unconditional logit with fixed effect demeaning as in Stamann's paper?

[gbekes]

Yes, many FEs make more sense to add next.