test: add e2e regression tests for issue #1138 — AI insert between existing lines#1270
Open
svarlamov wants to merge 1 commit into
Open
test: add e2e regression tests for issue #1138 — AI insert between existing lines#1270svarlamov wants to merge 1 commit into
svarlamov wants to merge 1 commit into
Conversation
Three tests using the TestRepo harness with custom checkpoints: 1. test_ai_insert_between_existing_methods_issue_1138 — PASSES AI inserts a single method block between two existing methods. All inserted lines are correctly attributed as AI. 2. test_ai_insert_single_line_between_existing_lines_issue_1138 — PASSES AI inserts a single line between existing lines. Correctly attributed. 3. test_ai_insert_multiple_blocks_between_existing_issue_1138 — FAILS (ignored) AI inserts two method blocks at different positions in a single edit. The tail lines of the second insertion block are misattributed as 'Test User' instead of 'mock_ai', confirming the lineRanges bug described in #1138. The third test is marked #[ignore] since it reproduces the bug. Refs: #1138 Co-Authored-By: Sasha Varlamov <sasha@sashavarlamov.com>
Contributor
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds three e2e tests using the
TestRepoharness with custom checkpoints to verify whether #1138 still reproduces.Finding: The bug reproduces. When AI inserts content at multiple positions in a single edit, the line ranges for later insertion blocks are truncated, causing the tail lines to be misattributed as unknown/untracked instead of AI.
Tests added
test_ai_insert_between_existing_methods_issue_1138test_ai_insert_single_line_between_existing_lines_issue_1138test_ai_insert_multiple_blocks_between_existing_issue_1138Test Userinstead ofmock_ai, confirming the lineRanges bug.Blame output from the failing test
The third test is marked
#[ignore]so CI passes. Remove it once the underlying line-range calculation is fixed.Review & Testing Checklist for Human
task test TEST_FILTER=test_ai_insert_multiple_blocks EXTRA_TEST_BINARY_ARGS="--ignored" NO_CAPTURE=trueNotes
fs::write+ manualcheckpoint human/checkpoint mock_aicalls (notset_contents) to precisely replicate real AI agent checkpoint flows.Link to Devin session: https://app.devin.ai/sessions/c78741bb99c14c0ca7ce1536292df196
Requested by: @svarlamov