LongMixEM

LongMixEM estimates finite mixture (latent-class) models for longitudinal data via the Expectation–Maximization (EM) algorithm. It targets a common problem in panel/repeated-measures analysis: unobserved subject-level heterogeneity that may be correlated with the covariates, which biases standard random-effects estimators. Instead of assuming a parametric (e.g. Normal) random-effects distribution, LongMixEM approximates it non-parametrically with a small number of latent classes (NPML), and lets that class structure optionally depend on covariates or be split into within/between components.

Why this is not a trivial EM implementation

• Likelihood stability. The E/M updates use log-space computation, clipped probabilities, and a probit link fit via statsmodels GLM, avoiding the numerical underflow that a naive implementation of mixture likelihoods runs into.
• Multi-start for local maxima. Finite-mixture log-likelihoods are generally non-concave and riddled with local maxima. multi_run and select_models fit many random initializations (reproducibly, via seeds spawned from a master seed) and keep the best run — a single EM run is not a reliable estimate.
• Within/between decomposition. The QP specification separates each covariate into its within-subject deviation and its subject-level mean before it enters the model, which is what lets the estimator disentangle a genuine covariate effect from confounding with the latent class structure.

Install

pip install -e .

Quickstart

from longmixem import LongMixEM, dgp_scenario

df, _ = dgp_scenario(n=300, T=6, rho_target=0.5, seed=0)
model = LongMixEM(df, type_="B")
result = model.fit(k=2, seed=42, flag="P")
print(result["likeli"], result["AIC"], result["BIC"])

The three specifications

• P (plain) — class prior probabilities are constant across subjects.
• C (covariate-dependent) — class priors depend on subject-level covariates (a finite-mixture analogue of correlated random effects).
• QP (within/between) — covariates are decomposed into within- and between-subject components to separate genuine effects from confounding with the latent classes.

Methodological reference

LongMixEM implements, in Python, the class of structured latent-class models for longitudinal data described by Alfò & Rocci (finite mixtures / NPML approaches to unobserved heterogeneity in panel data).