You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Strengthen the defuse tiers for JavaScript, with a precision audit #113
Is your feature request related to a problem? Please describe.
The defuse linker was designed as a backstop behind tsc, and on TypeScript that is exactly what it is. On JavaScript it silently becomes the primary resolver, while still being shaped like a fallback.
Measured provenance split, analyzer v1.1.0, -a 2:
corpus
tsc
import
defuse
vscode (TypeScript, 9,351 modules)
84.8%
8.4%
6.8%
nodejs/node whole repo (1,777 modules)
28.0%
18.6%
53.3%
nodejs/node lib/ (pure JS, 406 modules)
16.1%
17.6%
66.1%
odoo web frontend (650 modules)
10.1%
11.7%
78.2%
On Node's standard library — high-quality, heavily JSDoc'd JavaScript — the checker resolves about one edge in six. Two thirds of the graph comes from tiers whose job was to catch what tsc missed, not to carry it.
This is not a parsing problem. tsc parsed all 21,811 JS files in nodejs/node with zero load failures, at lib/ scale (406 modules, 9,248 callables, 33,837 edges) in 15.5s / 2.4 GB.
Describe the solution you'd like
An assignment-driven type-propagation tier that runs when tsc resolution is weak, in the spirit of Joern's XTypeRecovery (joern-cli/frontends/x2cpg/.../XTypeRecovery.scala): flow-insensitive, SSA-symbol-table style, a fixed small iteration count rather than a fixpoint (Joern defaults to 2), so it stays fast and deterministic. Scan signatures, walk assignments to infer variable types, and when a variable is a call receiver set the call's type from it.
A precision audit over the result, not just a coverage number: sample resolved JS edges and verify each against source. The model is the existing Joern ledger's exception classification, applied to our own output rather than to theirs.
Re-run the Joern comparison on a JavaScript corpus (nodejs/node/lib is a good size: 406 modules, self-contained, no vendored tree) and record it in docs/design/specs/defuse-linker-joern-ledger.md.
Caveat the existing superset claim in that ledger as a TypeScript result (see below).
Describe alternatives you've considered
Not stated in the original issue.
Additional context
Scope boundary
Strengthen the existing tiers in src/semantic_analysis/defuseLinker.ts for the low-tsc-resolution case, and audit the precision of what they produce.
Explicitly not in scope:
Replacing or supplementing tsc with Babel. Joern uses astgen + @babel/parser because it is a JVM tool with no TypeScript checker available; that is why it needs XTypeRecovery at all. Babel contributes no type information, so adopting it would trade 16-28% real resolution for 0% and then require rebuilding the difference. We have no parsing problem to solve.
Adopting Joern's fan-out. Its recovery emits candidate lists; the vscode ledger classes 131,710 rows as "multi-candidate callee lists ... candidate enumeration, not resolution" (one .toString() row links a 131 KB candidate string). cants resolves to a single callee, which is what makes an edge mean something. Keep that.
Caveats and known risks
More heuristic resolution means more wrong edges, and we have a worked example of the reverse. The allowJs fix in feat(callgraph)!: defuse linker, propagation tiers, and the repository-artifact layer #103 removed 89 edges from the vscode graph, 74 of them prov: defuse. They were not lost coverage: extensions/media-preview/media/imagePreview.js is a standalone webview IIFE with no imports whose e.preventDefault() on a DOM event had been matched by name to vs/base/browser/keyboardEvent.ts's StandardKeyboardEvent. Letting tsc type the receiver deleted the guess. This issue pushes in the opposite direction, so the precision audit is not optional — it is the acceptance criterion.
The ledger's 99.72% superset figure is a TypeScript result. It was measured on vscode, where tsc carries 84.8% of edges — a checker-backed analyzer against one with no checker. On JS we fall back to the same heuristic class Joern occupies while Joern loses nothing, so the margin should be expected to narrow and may not hold. It is currently stated without that qualification.
Determinism must survive.--jobs N output is byte-identical to --jobs 1 today and that gate must keep passing; a propagation tier with iteration-order sensitivity would break it.
Joern is not currently installed on the dev machine used for these measurements, so the comparison run needs it re-provisioned (v4 distribution, jssrc2cpg).
Cost: node/lib is 15.5s / 2.4 GB today. A propagation pass over every assignment will not be free, and the JS repos where it matters most are often the largest.
Definition of done
A measurable rise in resolved JS edges on nodejs/node/libtogether with a precision audit showing the false-positive rate did not increase. More edges alone is not success — the failure mode this issue risks is precisely a larger, more confident, wronger graph.
Targeting data (measured, v1.1.0, nodejs/node/lib, -a 2)
406 modules, 9,248 callables, 31,465 call sites in bodies.
Headroom: 17,951 of 31,465 call sites (57.1%) carry callee: null.
Classified by call shape:
shape
count
share
examples
<bare ident>()
7,770
43.3%
debug('SOCKET ERROR on FREE socket:', ...)
<ident>.<method>()
4,648
25.9%
socket.destroy(), socket.emit('agentRemove')
new <X>()
2,472
13.8%
new ERR_INVALID_ARG_TYPE(...)
this.<method>()
1,538
8.6%
this.on('newListener', maybeEnableKeylog)
<chain>.<method>()
1,104
6.2%
this[owner_symbol].destroy()
other
419
2.3%
this[kDetachAbortSignal]()
The top two shapes are 69% of the shortfall and are exactly what assignment-driven propagation
targets: a bare identifier bound by an earlier assignment, and a method call whose receiver's type
comes from one.
Ruled out: module-resolution settings
Hypothesis: defaultCompilerOptions sets module: ESNext + moduleResolution: Bundler, while
node/lib is CommonJS — so require() bindings might be failing to resolve for configuration
reasons rather than analysis ones.
Tested with a variant binary (module: CommonJS, moduleResolution: Node10):
Byte-identical. No resolver setting helps — node's require('internal/...') specifiers are
build-system paths that do not exist on disk, so they are unresolvable under every mode. The
shortfall is analysis, not configuration.
prov decision
The new tier reuses the existing defuse token. No new vocabulary, so no python lockstep and no
schema movement — this issue is maintenance-level work, not structural. Recorded because the
alternative was live: python already carries ssa and points-to, and coining a third term for the
same concept is the parity failure worth avoiding.
Trade-off accepted: propagation-derived edges will not be distinguishable from alias-chase or CHA
edges in the output, which makes the precision audit harder to slice. If that proves limiting,
adopting python's ssa is the fallback.
Is your feature request related to a problem? Please describe.
The defuse linker was designed as a backstop behind tsc, and on TypeScript that is exactly what it is. On JavaScript it silently becomes the primary resolver, while still being shaped like a fallback.
Measured provenance split, analyzer v1.1.0,
-a 2:lib/(pure JS, 406 modules)On Node's standard library — high-quality, heavily JSDoc'd JavaScript — the checker resolves about one edge in six. Two thirds of the graph comes from tiers whose job was to catch what tsc missed, not to carry it.
This is not a parsing problem. tsc parsed all 21,811 JS files in nodejs/node with zero load failures, at
lib/scale (406 modules, 9,248 callables, 33,837 edges) in 15.5s / 2.4 GB.Describe the solution you'd like
XTypeRecovery(joern-cli/frontends/x2cpg/.../XTypeRecovery.scala): flow-insensitive, SSA-symbol-table style, a fixed small iteration count rather than a fixpoint (Joern defaults to 2), so it stays fast and deterministic. Scan signatures, walk assignments to infer variable types, and when a variable is a call receiver set the call's type from it.nodejs/node/libis a good size: 406 modules, self-contained, no vendored tree) and record it indocs/design/specs/defuse-linker-joern-ledger.md.Describe alternatives you've considered
Not stated in the original issue.
Additional context
Scope boundary
Strengthen the existing tiers in
src/semantic_analysis/defuseLinker.tsfor the low-tsc-resolution case, and audit the precision of what they produce.Explicitly not in scope:
astgen+@babel/parserbecause it is a JVM tool with no TypeScript checker available; that is why it needsXTypeRecoveryat all. Babel contributes no type information, so adopting it would trade 16-28% real resolution for 0% and then require rebuilding the difference. We have no parsing problem to solve..toString()row links a 131 KB candidate string).cantsresolves to a single callee, which is what makes an edge mean something. Keep that.Caveats and known risks
allowJsfix in feat(callgraph)!: defuse linker, propagation tiers, and the repository-artifact layer #103 removed 89 edges from the vscode graph, 74 of themprov: defuse. They were not lost coverage:extensions/media-preview/media/imagePreview.jsis a standalone webview IIFE with no imports whosee.preventDefault()on a DOM event had been matched by name tovs/base/browser/keyboardEvent.ts'sStandardKeyboardEvent. Letting tsc type the receiver deleted the guess. This issue pushes in the opposite direction, so the precision audit is not optional — it is the acceptance criterion.--jobs Noutput is byte-identical to--jobs 1today and that gate must keep passing; a propagation tier with iteration-order sensitivity would break it.jssrc2cpg).node/libis 15.5s / 2.4 GB today. A propagation pass over every assignment will not be free, and the JS repos where it matters most are often the largest.Definition of done
A measurable rise in resolved JS edges on
nodejs/node/libtogether with a precision audit showing the false-positive rate did not increase. More edges alone is not success — the failure mode this issue risks is precisely a larger, more confident, wronger graph.