Skip to content

perf: trim static scope map allocation#837

Merged
stephenamar-db merged 1 commit into
databricks:masterfrom
He-Pin:perf/static-scope-map-allocation
May 11, 2026
Merged

perf: trim static scope map allocation#837
stephenamar-db merged 1 commit into
databricks:masterfrom
He-Pin:perf/static-scope-map-allocation

Conversation

@He-Pin
Copy link
Copy Markdown
Contributor

@He-Pin He-Pin commented May 11, 2026

Motivation:
Reduce optimizer allocation overhead in static scope construction without changing name-resolution semantics.

Key Design Decision:
Keep the existing immutable scope representation and only replace intermediate collection construction with a direct loop.

Modification:
ScopedExprTransform now builds the child scope mapping with a while loop instead of zipWithIndex.map, avoiding tuple and intermediate collection allocation.

Benchmark Results:

Benchmark master PR #837 Delta Notes
OptimizerBenchmark.main JMH 0.432 +/- 0.004 ms/op 0.422 +/- 0.004 ms/op -2.31% Targeted optimizer benchmark from exploration run.
Scala Native hyperfine realistic2.jsonnet 83.438 +/- 1.304 ms 83.871 +/- 1.111 ms +0.52% Output matched master; end-to-end CLI guard is neutral/noisy.

Analysis:
This is intentionally smaller than the historical mutable-scope idea; it keeps the safer immutable invariant while trimming hot setup work. The Native CLI guard does not show a material end-to-end win, but the targeted optimizer benchmark is positive.

References:
Source exploration commit: He-Pin/sjsonnet b1a42ad8.

Result:
Local ./mill --no-server -j 1 __.reformat and ./mill --no-server -j 1 __.test passed on this split branch (2066/2066).

Motivation:
Reduce allocation overhead in the static expression transform scope setup without changing optimizer semantics.

Modification:
Build the immutable scope mapping with a direct while loop instead of zipWithIndex/map intermediate tuples and arrays.

Result:
Optimizer setup keeps the same immutable-scope behavior while doing less temporary allocation.
@He-Pin He-Pin marked this pull request as ready for review May 11, 2026 09:35
@stephenamar-db stephenamar-db merged commit d0aec7d into databricks:master May 11, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants