Repository navigation
r2_nakagawa() is **NOT** location invariant when COV is not modelled (and is wrong?) #428
Description
This is some weird stuff.
library(lme4)
#> Loading required package: Matrix
library(performance)
r2_nakagawa2 <- function(model) {
y <- insight::get_response(model)
c(
Cond. = cor(predict(model, re.form = NULL), y)^2,
Marg. = cor(predict(model, re.form = NA), y)^2
)
}
# With COV =================================
m1 <- lmer(Reaction ~ Days + (Days|Subject),
data = sleepstudy)
# Change location -------
sleepstudy$Days0 <- sleepstudy$Days - 10
m2 <- lmer(Reaction ~ Days0 + (Days0|Subject),
data = sleepstudy)
#> Warning in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
#> Model failed to converge with max|grad| = 0.00422952 (tol = 0.002, component 1)
r2_nakagawa(m1)
#> # R2 for Mixed Models
#>
#> Conditional R2: 0.799
#> Marginal R2: 0.279
r2_nakagawa(m2) # same
#> # R2 for Mixed Models
#>
#> Conditional R2: 0.799
#> Marginal R2: 0.279
# Validate:
r2_nakagawa2(m1)
#> Cond. Marg.
#> 0.8271703 0.2864714
r2_nakagawa2(m2)
#> Cond. Marg.
#> 0.8271702 0.2864714
# Without COV =================================
m1b <- lmer(Reaction ~ Days + (Days||Subject),
data = sleepstudy)
# Change location -------
m2b <- lmer(Reaction ~ Days0 + (Days0||Subject),
data = sleepstudy)
r2_nakagawa(m1b) # different
#> # R2 for Mixed Models
#>
#> Conditional R2: 0.702
#> Marginal R2: 0.415
r2_nakagawa(m2b) # even more different
#> # R2 for Mixed Models
#>
#> Conditional R2: 0.864
#> Marginal R2: 0.183
r2_nakagawa2(m1b) # same as with cov
#> Cond. Marg.
#> 0.8279419 0.2864714
r2_nakagawa2(m2b) # same as with cov
#> Cond. Marg.
#> 0.8244819 0.2864714Created on 2022-06-03 by the reprex package (v2.0.1)
Activity
[-]`r2_nakagawa()` is **NOT** location invariant when COV is not modelled?[/-][+]`r2_nakagawa()` is **NOT** location invariant when COV is not modelled (and is wrong?)[/+]strengejacke commentedon Jun 9, 2022
Let's look at some more output:
Created on 2022-06-09 by the reprex package (v2.0.1)
strengejacke commentedon Jun 9, 2022
The formulas for R2 are:
This explains why
m1andm2return same results, while we find those differences form1bandm2b.m1be equal tom1b.bwiernik commentedon Jun 9, 2022
Okay that explains it.
It makes sense that the intercept variance is a lot different when the predictor is re-centered. The huge slope-intercept correlation the re-centering induces compensates for that. But if that's constrained to zero, there is no place for the extra variance to go
mattansb commentedon Jun 9, 2022
A fix for the marginal R2 would be:
The issue is how do we get a correct estimate of the variance accounted for by random effects?
(Other than using the
predict(..., re.form=)solution above?)Note that this also affects the validity of the
icc()output.jmgirard commentedon Oct 10, 2026
I looked into this again with the current development versions (performance from main, insight 1.5.4.24). There are two separate things going on.
First, there is a bug in
insight::get_variance(). For uncorrelated random slopes ((Days || Subject)), lme4 splits the terms intoSubjectandSubject.1.var.randomonly includes the intercept variance and leaves out the slope variance. Form1b,var.randomis 627.6, which equalsvar.interceptalone. The fix for easystats/insight#304 in 2021 correctedvar.interceptandvar.slopebut notvar.random. The fix is open in easystats/insight#1263, because performance only uses the variance components that insight returns.With Johnson's (2014) mean random-effect variance, which
r2_nakagawa()says it uses (seeR/r2_nakagawa.R, details section), the value is:With easystats/insight#1263,
get_variance()returns this value. Conditional R2 is then 0.797 and marginal R2 is 0.283 form1b. Those values are close to the correlated model (0.799 and 0.279), andr2_nakagawa()currently returns 0.702 and 0.415.Second, even after that fix,
m2b(centeredDays0) gives different values: 0.887 conditional and 0.152 marginal. I do not think that this part is a bug. bwiernik explained it above: when the slope-intercept correlation is fixed at zero, shifting the predictor changes the model itself. The intercept variance (3377.7 vs. 627.6) cannot be absorbed by the correlation, so the variance decomposition really changes. Models with uncorrelated random slopes are not location invariant. A note in the docs forr2_nakagawa()andicc()could say that.Replacing the denominator with
var(y), as suggested above, would move away from the Nakagawa et al. (2017) definition, so that is a separate design question.mattansb commentedon Oct 11, 2026
Thanks for looking into this @jmgirard - does this mean that once easystats/insight#1263 is resolved this issue should be closed? Or is there any fix required on the performance side?
jmgirard commentedon Oct 11, 2026
Mostly yes. The numeric bug is in insight.
r2_nakagawa()andicc()usevar.randomfrominsight::get_variance()as is. So after easystats/insight#1263 merges, both return the corrected values form1b(conditional R2 0.797, marginal R2 0.283).On the performance side, I suggest two small changes in one PR after that merge, and that PR can close this issue:
DESCRIPTIONto the release with #1263, and add a regression test form1b.r2_nakagawa()andicc(): models with uncorrelated random slopes are not location invariant. A shift of a predictor changes the model, so R2 and the ICC change too.On
var(y)as the denominator: probably not. It makes the marginal R2 stable (0.286 form1bandm2b), but it divides model estimates by the observed spread of the data. The parts then do not add up to the total, and the conditional R2 form2bbecomes 1.669. For non-Gaussian models,var.fixedandvar.randomare on the link scale, andvar(y)is not. The cause is the model itself, so the docs note is the better fix.mattansb commentedon Oct 11, 2026
Yes to 1+2. Thanks @jmgirard !
I agree about not using$Var[Y]$ - that's too messy.
strengejacke commentedon Oct 11, 2026
I think we should just add a Remotes-field to the DESCRIPTION file.