Skip to content

fix: stop endless recursion in resolveValues on assignment cycles - #530

Merged
san-zrl merged 1 commit into
mainfrom
fix/issue-525-resolvevalues-cycle
Aug 31, 2026
Merged

fix: stop endless recursion in resolveValues on assignment cycles#530
san-zrl merged 1 commit into
mainfrom
fix/issue-525-resolvevalues-cycle

Conversation

@n1ckl0sk0rtge

Copy link
Copy Markdown
Contributor

Fixes #525.

What was wrong

JavaDetectionEngine.resolveValues follows a variable's assignments and its initializer to find its value. It had no cycle check. Code like this made it recurse forever:

private String algorithm = "AES/ECB/PKCS5Padding";
private String copy = algorithm;   // initializer edge: copy -> algorithm

public void swap() {
    algorithm = copy;              // assignment edge: algorithm -> copy
}

Resolving copy jumps between the assignment branch (line 337) and the initializer branch (line 353) forever. This is exactly the alternating stack trace in #525. The existing selections.size() > 15 guard never fires, because identifier-to-identifier hops do not grow selections.

The bug is old and exists in 1.6.0 too. But 1.6.1 exposed it: since e1fdab3 the engine resolves the arguments of every method call (to build detached call records), not only the arguments of matched crypto calls. So a cycle anywhere in the scanned code now crashes the scan. The same change caused the "Detected constructor definition has more then one argument" log flood from the issue.

The fix

The variable branch of resolveValues moves into a new method resolveVariableValues. It tracks which variable symbols are being resolved on the current path:

  • Before following a variable, its symbol is added to a resolvingVariables set.
  • A variable that is already on the path is a cycle. It returns an empty list instead of recursing.
  • When the variable is done, a finally block removes it again.

The release step matters: it cuts only real cycles. Two sibling branches (for example two array elements) may still resolve through the same shared variable. A global "visited" set without release would drop the second value.

Red-green test

JcaCipherGetInstanceCyclicAssignmentTest reproduces the issue:

Not in this PR

  • The noisy LOGGER.info for constructors with more than one argument. Fixes StackOverflow in JavaDetectionEngine:resolveValues #529 already demotes it to trace; that part fits better there.
  • The wider question of how much work buildDetachedCall does per call. That is a performance topic, not a crash, and deserves its own issue.

Verified with mvn test -pl engine,java: 164 tests, 0 failures. Spotless and checkstyle clean.

Fixes #525.

resolveValues follows a variable's assignments and its initializer to
find its value. It had no cycle check. Code like

    String algorithm = "AES";
    String copy = algorithm;
    ...
    algorithm = copy;

made it recurse forever and crash the scan with a StackOverflowError.
The existing selections.size() > 15 guard never fires here, because
identifier-to-identifier hops do not grow the selections list.

The bug is old, but 1.6.1 exposed it: since e1fdab3 the engine
resolves the arguments of every method call (for detached call
records), not only the arguments of matched crypto calls. So a cycle
anywhere in the scanned code now reaches resolveValues.

The fix moves the variable branch into resolveVariableValues and
tracks which variables are being resolved on the current path. A
variable that is already on the path is not followed again. The
variable is released again when its resolution is done, so two
sibling branches may still resolve through the same variable.

A new red-green test reproduces the exact alternating stack trace
from the issue and checks that the value still resolves through the
cycle to the constant initializer.
@san-zrl

san-zrl commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

log msg demoted to trace in #531 (merged), CBOM generation disabled in #532.

@san-zrl san-zrl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. This also fixes the StackOverflow in the CI pipeline when scanning
BlockCipherReorganizer.java with a simpler call stack:

[main] ERROR org.sonar.java.ast.JavaAstScanner - A stack overflow error occurred while analyzing file: 'src/main/java/com/ibm/mapper/reorganizer/rules/BlockCipherReorganizer.java'
java.lang.StackOverflowError
	at com.ibm.engine.language.java.JavaDetectionEngine.resolveValues(JavaDetectionEngine.java:318)
	at com.ibm.engine.language.java.JavaDetectionEngine.resolveValues(JavaDetectionEngine.java:337)
	at com.ibm.engine.language.java.JavaDetectionEngine.resolveValues(JavaDetectionEngine.java:337)
	at com.ibm.engine.language.java.JavaDetectionEngine.resolveValues(JavaDetectionEngine.java:337)
...

@san-zrl
san-zrl merged commit 908d08e into main Aug 31, 2026
0 of 2 checks passed
@san-zrl
san-zrl deleted the fix/issue-525-resolvevalues-cycle branch August 31, 2026 14:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Regression in 1.6.1 (compared to 1.6.0): StackOverflowError and excessive logging

2 participants