Convergence, bootstrap, p-values by Maximilian-Stefan-Ernst · Pull Request #316 · StructuralEquationModels/StructuralEquationModels.jl

Maximilian-Stefan-Ernst · 2026-03-09T22:15:11Z

convergence assessment
- adds the method converged for SemFit objects
- now, converged returns whether the model is converged in general
- convergence returns the convergence flags and can be used to inspect why the model converged (or not)
boostrap refactoring
- bug fix: each boostrap sample is now checked for convergence
- bootstrap can now use multiple threads by setting parallel = true
- there is now an additional generic bootstrap function that can be used to boostrap any statistic that can be extracted from a fitted model
convenience functions
- z_test to compute p-values for individual parameters
- normal_CI to compute a normal theory confidence interval

Maximilian-Stefan-Ernst · 2026-03-09T22:16:59Z

@alyst this is a relatively big PR - would you like to review it before I merge?

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

alyst · 2026-03-09T22:43:27Z

@Maximilian-Stefan-Ernst Please proceed as you planned, I think these are nice changes. As I have mentioned before, I am finishing preparing the PR that potentially enhances the support for multi-group and "multi-regularization" SEMs. Because of those changes, I also have to update replace_observed() and bootstrap_se(), which relies on it. I will rebase that WIP PR on the new devel branch once you merge this one, so any suggestions to this PR could be a part of my upcoming PR. :)

…bserved, fix bs + tests

github-actions · 2026-03-12T10:07:11Z

src/implied/RAM/generic.jl

+            observed = observed,
+            gradient_required = !isnothing(implied.∇A),
+            meanstructure = MeanStruct(implied) == HasMeanStruct,
+            kwargs...)


[JuliaFormatter] _{reported by reviewdog 🐶}

Suggested change

kwargs...)

kwargs...,

)

github-actions · 2026-03-12T10:07:11Z

src/implied/RAM/symbolic.jl

+        observed = observed,
+        vech = implied.Σ isa Vector,
+        gradient = !isnothing(implied.∇Σ),
+        hessian = !isnothing(implied.∇²Σ),
+        meanstructure = MeanStruct(implied) == HasMeanStruct,
+        approximate_hessian = isnothing(implied.∇²Σ),
+        kwargs...)


[JuliaFormatter] _{reported by reviewdog 🐶}

Suggested change

observed = observed,

vech = implied.Σ isa Vector,

gradient = !isnothing(implied.∇Σ),

hessian = !isnothing(implied.∇²Σ),

meanstructure = MeanStruct(implied) == HasMeanStruct,

approximate_hessian = isnothing(implied.∇²Σ),

kwargs...)

observed = observed,

vech = implied.Σ isa Vector,

gradient = !isnothing(implied.∇Σ),

hessian = !isnothing(implied.∇²Σ),

meanstructure = MeanStruct(implied) == HasMeanStruct,

approximate_hessian = isnothing(implied.∇²Σ),

kwargs...,

)

github-actions · 2026-03-12T10:07:11Z

src/loss/ML/ML.jl

+        observed = observed,
+        approximate_hessian = HessianEval(lossfun) == ApproxHessian,
+        kwargs...)


[JuliaFormatter] _{reported by reviewdog 🐶}

Suggested change

observed = observed,

approximate_hessian = HessianEval(lossfun) == ApproxHessian,

kwargs...)

observed = observed,

approximate_hessian = HessianEval(lossfun) == ApproxHessian,

kwargs...,

)

github-actions · 2026-03-12T10:07:11Z

src/loss/WLS/WLS.jl

+update_observed(lossfun::SemWLS, observed::SemObserved; kwargs...) = SemWLS(;
+    observed = observed,
+    meanstructure = MeanStruct(kwargs[:implied]) == HasMeanStruct,
+    kwargs...)


[JuliaFormatter] _{reported by reviewdog 🐶}

Suggested change

kwargs...)

kwargs...,

)

github-actions · 2026-03-12T11:08:11Z

test/examples/helper.jl

+        model_fit,
+        spec;
+        compare_hessian = true,
+        rtol_hessian = 0.2,
+        compare_bs = true,
+        rtol_bs = 0.1,
+        n_boot = 500)


[JuliaFormatter] _{reported by reviewdog 🐶}

Suggested change

model_fit,

spec;

compare_hessian = true,

rtol_hessian = 0.2,

compare_bs = true,

rtol_bs = 0.1,

n_boot = 500)

model_fit,

spec;

compare_hessian = true,

rtol_hessian = 0.2,

compare_bs = true,

rtol_bs = 0.1,

n_boot = 500,

)

github-actions · 2026-03-12T11:08:12Z

test/examples/helper.jl

+    if compare_bs
+        bs_samples = bootstrap(model_fit, spec; n_boot = n_boot)
+        @test bs_samples[:n_converged] > 0.95*n_boot
+        bs_samples = cat(bs_samples[:samples][BitVector(bs_samples[:converged])]..., dims = 2)


[JuliaFormatter] _{reported by reviewdog 🐶}

Suggested change

bs_samples = cat(bs_samples[:samples][BitVector(bs_samples[:converged])]..., dims = 2)

bs_samples =

cat(bs_samples[:samples][BitVector(bs_samples[:converged])]..., dims = 2)

github-actions · 2026-03-12T20:25:19Z

test/examples/multigroup/build_models.jl

 )

-model_ls_multigroup = SemEnsemble(model_ls_g1, model_ls_g2; optimizer = semoptimizer)
+model_ls_multigroup = SemEnsemble(model_ls_g1, model_ls_g2; groups = [:Pasteur, :Grant_White], optimizer = semoptimizer)


[JuliaFormatter] _{reported by reviewdog 🐶}

Suggested change

model_ls_multigroup = SemEnsemble(model_ls_g1, model_ls_g2; groups = [:Pasteur, :Grant_White], optimizer = semoptimizer)

model_ls_multigroup = SemEnsemble(

model_ls_g1,

model_ls_g2;

groups = [:Pasteur, :Grant_White],

optimizer = semoptimizer,

)

github-actions · 2026-03-12T20:25:19Z

test/examples/multigroup/build_models.jl

            lav_groups = Dict(:Pasteur => 1, :Grant_White => 2),
        )

+


[JuliaFormatter] _{reported by reviewdog 🐶}

Suggested change

alyst · 2026-03-12T23:33:12Z

src/frontend/fit/standard_errors/bootstrap.jl

+)
+```
 """
 function se_bootstrap(


It should be possible to make se_bootstrap() a wrapper around bootstrap()

alyst · 2026-03-12T23:40:17Z

src/frontend/fit/standard_errors/bootstrap.jl

-                lock(lk) do
-                    n_failed[] += 1
-                end


To avoid locking, you can also use Threads.@atomic n_failed[] += 1, where Threads.atomic_add!(n_failed, 1) (see atomic operations)

alyst · 2026-03-12T23:48:02Z

src/frontend/fit/standard_errors/bootstrap.jl

+            put!(model_pool, thread_model)
+        end
+    end
+    return Dict(


I think it's better to use NamedTuple here.
Also, while it might be fragile to use Base.return_types() to infer the result of statistic() before it is called, here it should be in principle possible to use T = mapreduce(typeof, promote_type, out) and Vector{T}(out) to help with the type inference and faster code generation downstream.

alyst · 2026-03-12T23:53:17Z

src/frontend/fit/standard_errors/bootstrap.jl

+    data = prepare_data_bootstrap(data, fitted.model)
+    start = solution(fitted)
+    # pre-allocations
+    out = []


You can also preallocate out = Vector{Any}(nothing, n_boot) and conv = fill(false, n_boot) (or conv = Vector{Union{Missing, Bool}}(missing, n_boot)), then you can avoid locks and just set the results directly out[i] = ....
The failed iterations should have out[i] === nothing, you can filter them out when generating the result.
If you are using conv with true/false/missing logic, you also don't have to count failed iterations as they will have conv[i] === missing.

Maximilian-Stefan-Ernst added 5 commits March 9, 2026 23:05

add converged method

e80389c

refactor bootstrap and add bootstrap for any statistic

1623119

add CI and p values

8f2eaf1

add exports

6f1246c

add boostrap, CI and p-values tests

20c8b63

Maximilian-Stefan-Ernst changed the title ~~Add pvalue~~ Convergence, boostrap, p-values Mar 9, 2026

Maximilian-Stefan-Ernst and others added 2 commits March 9, 2026 23:20

Apply suggestions from formatter

0eb04ef

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

fix bootstrap tests

5e5575b

fix bootstrap tests

c27706d

Maximilian-Stefan-Ernst changed the title ~~Convergence, boostrap, p-values~~ Convergence, bootstrap, p-values Mar 9, 2026

fix nobs var check in update_observed, keep previous args in update_o…

24a7947

…bserved, fix bs + tests

github-actions bot reviewed Mar 12, 2026

View reviewed changes

Maximilian-Stefan-Ernst added 2 commits March 12, 2026 11:58

increase tolerance for bootstrap test

aa62557

increase tolerance for bootstrap test

fdca879

github-actions bot reviewed Mar 12, 2026

View reviewed changes

Maximilian-Stefan-Ernst added 3 commits March 12, 2026 19:02

remove bootstrap try-catch and update tests

ccf8c0d

fix bootstrap

1adb1fa

fix bootstrap

9478d55

github-actions bot reviewed Mar 12, 2026

View reviewed changes

fix bootstrap

2e37710

alyst reviewed Mar 12, 2026

View reviewed changes

	bs_samples = cat(bs_samples[:samples][BitVector(bs_samples[:converged])]..., dims = 2)
	bs_samples =
	cat(bs_samples[:samples][BitVector(bs_samples[:converged])]..., dims = 2)

Conversation

Maximilian-Stefan-Ernst commented Mar 9, 2026

Uh oh!

Maximilian-Stefan-Ernst commented Mar 9, 2026

Uh oh!

alyst commented Mar 9, 2026

Uh oh!

github-actions bot Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

alyst Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

alyst Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

alyst Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

alyst Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

alyst Mar 12, 2026 •

edited

Loading