Skip to content

fix(test): reduce ASAN wall-clock rate in BoundMethodHandleProfilerTest to avoid OOM - #630

Merged
rkennke merged 1 commit into
mainfrom
fix/bound-method-handle-asan-oom
Jul 3, 2026
Merged

rkennke merged 1 commit into
mainfrom
fix/bound-method-handle-asan-oom

Conversation

@rkennke

@rkennke rkennke commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Problem

BoundMethodHandleProfilerTest fails on aarch64 under :ddprof-test:testAsan — it runs very long and OOMs.

Root cause

The test samples at wall=100us (~10k samples/s). Under ASAN the bound-method-handle workload runs for minutes, and because wall-clock sampling is time-based, a slower aarch64 CI runner accumulates a huge number of datadog.MethodSample events for the same workload. verifyEvents() then loads the entire recording into a JMC object model inside the 512MB ASAN test heap (-Xmx512m), which OOMs — or the multi-minute runtime trips the CI job timeout.

The pre-existing 1000-handle aarch64 workaround was fragile because reducing the work count only partly reduces runtime; it doesn't decouple sample count from runner speed.

Fix

  • Coarsen sampling to wall=1ms under ASAN (10× fewer samples, independent of runner speed).
  • Drop the aarch64+asan workload from 1000 → 500 handles for margin.

Verification (on aarch64)

Config datadog.MethodSample events Result
Pre-fix (wall=100us, 1000) 57,735 passed here (CI runner OOMs)
Fixed (wall=1ms, 500) 275 passed

🤖 Generated with Claude Code

…st to avoid OOM

BoundMethodHandleProfilerTest sampled at wall=100us (~10k samples/s). Under
ASAN the bound-method-handle workload runs for minutes, and because wall-clock
sampling is time-based, a slow aarch64 CI runner accumulates a huge number of
MethodSample events. verifyEvents() then loads the entire recording into a JMC
object model inside the 512MB ASAN test heap (-Xmx512m), causing an OOM (or the
multi-minute runtime trips the CI job timeout).

Coarsen the sampling to wall=1ms under ASAN (10x fewer samples, independent of
runner speed) and drop the aarch64+asan workload from 1000 to 500 handles for
margin. Verified on aarch64: MethodSample count drops from 57,735 to 275 while
the test still passes.

Environment: Datadog workspace

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@rkennke
rkennke requested a review from a team as a code owner July 3, 2026 10:51
@dd-octo-sts

dd-octo-sts Bot commented Jul 3, 2026 •

Copy link
Copy Markdown
Contributor

CI Test Results

Run: #34249236363 | Commit: d130f69 | Duration: 0s (longest job)

✅ All 0 test jobs passed

Summary: Total: 0 | Passed: 0 | Failed: 0


Updated: 2026-09-08 16:39:21 UTC

@rkennke
rkennke merged commit d130f69 into main Jul 3, 2026
155 of 183 checks passed
@rkennke
rkennke deleted the fix/bound-method-handle-asan-oom branch July 3, 2026 11:37
@github-actions github-actions Bot added this to the 1.46.0 milestone Jul 3, 2026
@zhengyu123 zhengyu123 added the fixed_version: v_1.46.0 Version this PR's fix shipped in label Sep 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fixed_version: v_1.46.0 Version this PR's fix shipped in trivial

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants