Skip to content

Conversation

@marianotepper
Copy link
Contributor

Relax the threshold of OnHeapGraphIndexTest.testIncrementalInsertionFromOnDiskIndex_withIdentityOrdinalMapping as it was flaky

@github-actions
Copy link
Contributor

github-actions bot commented Jan 20, 2026

Before you submit for review:

  • Does your PR follow guidelines from CONTRIBUTIONS.md?
  • Did you summarize what this PR does clearly and concisely?
  • Did you include performance data for changes which may be performance impacting?
  • Did you include useful docs for any user-facing changes or features?
  • Did you include useful javadocs for developer oriented changes, explaining new concepts or key changes?
  • Did you trigger and review regression testing results against the base branch via Run Bench Main?
  • Did you adhere to the code formatting guidelines (TBD)
  • Did you group your changes for easy review, providing meaningful descriptions for each commit?
  • Did you ensure that all files contain the correct copyright header?

If you did not complete any of these, then please explain below.

@marianotepper marianotepper changed the title Relax the threshold for a flaky test Relax the threshold of a flaky test Jan 20, 2026
@marianotepper
Copy link
Contributor Author

marianotepper commented Jan 22, 2026

I run 10k instances with this threashold on my machine and it seemed to have done the job. However, PRs are still failing because of this test:
OnHeapGraphIndexTest.testIncrementalInsertionFromOnDiskIndex_withIdentityOrdinalMapping:274 Recall mismatch, recallFromReconstructedAllNodeOnHeapGraphIndex: 0.686000 != recallFromAllGraphIndex: 0.606000 expected:<0.6859998> but was:<0.6059998>
The gap is 0.08 instead of 0.06, which is the value I used. We could change it to 0.07, but it looks like 8% recall drop is pretty significant. This to me indicates that there are deeper problems with the functionality behind this test. This is related to #568.

@marianotepper
Copy link
Contributor Author

The functionality that this test is evaluating is not working correctly and is tagged as experimental. The test remains flaky and the level of relaxation needed to get rid of the flakiness is starting to get unacceptable. Marking the test as @Ignore until the functionality receives proper attention.

Copy link
Collaborator

@tlwillke tlwillke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to ignore the test for now. We will decide what to do with the related contribution. We can then revisit the recall threshold or permanently remove the test and related work.

@tlwillke tlwillke merged commit 977c21d into main Jan 26, 2026
8 checks passed
@tlwillke tlwillke deleted the fix_testIncrementalInsertionFromOnDiskIndex_withIdentityOrdinalMapping branch January 26, 2026 18:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants